Hmm but surely corners can add more information to game state - there may be two games that are 1-0, except the first game has 10 corners by the home team whereas the second only 2 by the home team. If we think about expected goals in the second half, doesn't the 10 corners indicate aggressive play and so more of an expectation of goals in the first game rather than the second?
Football Data (CSV, JSON) - UPDATED 16/08/17
Corner = A ball going over the goal line but last touched by the defending team. So usually via clearance or a save by the goalie - both of which indicate that the attacking team are in dangerous positions. So more corners = more attempts to attack by a team, whereas the number of actual goals may not be so indicative as shown by my example.
-
- Posts: 165
- Joined: Thu May 09, 2013 5:57 pm
Corners can be misleading imo the best way to predict goals or expect goals is to divide the number of dangerous attacks by the number of attacks to give you a % > than 70% high expectancy of a goal.
Enjoying having a play about with this SB,, just wondering if your still going too add more search options too it at some stage?spreadbetting wrote: ↑Wed Aug 23, 2017 10:18 pmI stuck welshboy's data up on a simple website, I'll try and make it a big more searchable when I figure out php7
-
- Posts: 3
- Joined: Thu Nov 09, 2017 4:34 pm
Hi Guys,
I'm doing a bit of analysis on stats at halftime/fulltime but I'm falling short trying to find halftime data without scraping it manually. Does anyone have halftime shots/shotsOT/Odds data to hand? Or perhaps know where I might be able to get hold of this? I'm only really interested in the top leagues. Would be seriously useful
P.s Loving this thread, loads of interesting chat going on
I'm doing a bit of analysis on stats at halftime/fulltime but I'm falling short trying to find halftime data without scraping it manually. Does anyone have halftime shots/shotsOT/Odds data to hand? Or perhaps know where I might be able to get hold of this? I'm only really interested in the top leagues. Would be seriously useful
P.s Loving this thread, loads of interesting chat going on
- Kafkaesque
- Posts: 886
- Joined: Fri Oct 06, 2017 10:20 am
The best part of being a relative newbie on here is these brilliant topics from before my time which pop up every so often
Brilliant work, Welshboy!
An ever so slightly off topic question for ya. Whoscored.com has some stats, I would love to delve more into. Before I try to teach myself to scrape data, though, am I correct in assuming that they've specifically coded the site to make it highly problematic to scrape it for data?
Brilliant work, Welshboy!
An ever so slightly off topic question for ya. Whoscored.com has some stats, I would love to delve more into. Before I try to teach myself to scrape data, though, am I correct in assuming that they've specifically coded the site to make it highly problematic to scrape it for data?
-
- Posts: 3
- Joined: Thu Nov 09, 2017 4:34 pm
Whoscored happens to be the website I value the most but yes, unfortunately, it is very difficult to scrape!
Betfair do offer what I am trying to find (historical HT odds) but I am not really sure I am willing to pay £200/month for it at this stage in my trading career!
Ah well...
Betfair do offer what I am trying to find (historical HT odds) but I am not really sure I am willing to pay £200/month for it at this stage in my trading career!
Ah well...
Thanks to the poster on this one good stuff for us newbies.
I have a question if I am looking to back the draw in a certain match and want to work out roughly when the first 1st goal would be scored.
Would you seasoned football traders look at the specific games involving the 2 teams (as this could be only 2 matches) or would you look at all games played by each team in this league and then take an average from that wider sample?
I have a question if I am looking to back the draw in a certain match and want to work out roughly when the first 1st goal would be scored.
Would you seasoned football traders look at the specific games involving the 2 teams (as this could be only 2 matches) or would you look at all games played by each team in this league and then take an average from that wider sample?
-
- Posts: 4
- Joined: Mon Nov 06, 2017 4:47 pm
Link not working.welshboy06 wrote: ↑Sun Aug 13, 2017 9:45 amHi All,
UPDATED 16/08/17
ADDED CSV VERSIONS OF ALL LEAGUES
I've decided to start collecting football data. Mainly because I have an interest in the sport and also because of a certain jonnyg throwing stats around in a very hard to read format.
The data is quite simple and was just scraped from HKJC.
The data is in json format, but you should be able to convert it to csv and import it to Excel. I chose json as it reduces data duplication, is structured and can be read reasonably well by the human eye. It also works really well with Python (Which I'm using to scrape and also analyse the data)
The data contains the following info.I've managed to scrape ALL leagues and Seasons from HKJC (Where specific goal time data is provided)Code: Select all
League Season Game Date Home Team Away Team And also Goals and Red cards (Player, Time and Team) ----- Soon: Total FT/HT Goals and Red Cards for HT and AT (Included in CSV's)
I may decide to scrape more data, but for most leagues there is 10+ years worth of games, with goal and red card time data.
The data is a little too big to post here, and I didn't fancy splitting everything into multiple zip files. So I've uploaded and shared to my Dropbox
Not sure if the mods/admins could increase the file size limit, so I can add directly to the thread.
The below is a direct link to the folder containing all the data I've scraped.
Each league is in a separate text file, and I've also included a .zip of ALL the files.
https://www.dropbox.com/sh/7sif9n90ehys ... bd6ba?dl=0
Contents of the above filesOkay so there are now two sets of CSV files.Code: Select all
Number of Countries: 20 Country: Argentine - Leagues: 1 League Name: Argentine Division 1 - Seasons: 11 (2009-10(Au), 2010-11 (Sp), 2010-11(Au), 2011-12(Au), 2012-13 (In), 2012-13(Fi), 2013-14 (In), 2013-14(Fi), 2014-15 (In), 2015, 2016) Country: Australian - Leagues: 2 League Name: Australian Division 1 Playoffs - Seasons: 7 (2008-09, 2009-10, 2010-11, 2011-12, 2012-13, 2013-14, 2014-15) League Name: Australian Division 1 - Seasons: 9 (2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) Country: Belgian - Leagues: 3 League Name: Belgian Division 1 Championship Playoffs - Seasons: 1 (2016) League Name: Belgian Division 1 UE Cup Playoffs - Seasons: 1 (2016) League Name: Belgian Division 1 - Seasons: 2 (2015-2016, 2016-2017) Country: Brazilian - Leagues: 3 League Name: Brazilian Division 1 - Seasons: 9 (2005, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016) League Name: Brazilian Paulista League Knockout stage - Seasons: 1 (2014) League Name: Brazilian Paulista League - Seasons: 1 (2014) Country: Chilean - Leagues: 1 League Name: Chilean Division 1 - Seasons: 6 (2014-15(AP), 2014-15(CL), 2015-16(AP), 2015-16(CL), 2016-17(AP), 2016-17(CL)) Country: Dutch - Leagues: 3 League Name: Dutch Cup - Seasons: 7 (2005-06, 2007-08, 2008-09, 2009-10, 2010-11, 2011-12, 2012-13) League Name: Dutch Division 1 - Seasons: 10 (2005-2006, 2006-2007, 2008-2009, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) League Name: Dutch Division 2 - Seasons: 6 (2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) Country: Eng - Leagues: 4 League Name: Eng Championship - Seasons: 13 (2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) League Name: Eng League 1 - Seasons: 8 (2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) League Name: Eng League Cup - Seasons: 6 (2006-07, 2007-08, 2009-10, 2010-11, 2012-13, 2013-14) League Name: Eng Premier - Seasons: 14 (2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) Country: French - Leagues: 4 League Name: French Division 1 - Seasons: 12 (2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) League Name: French Division 2 - Seasons: 6 (2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) League Name: French FA Cup - Seasons: 1 (2005-06) League Name: French League Cup - Seasons: 1 (2008-09) Country: German - Leagues: 3 League Name: German Cup - Seasons: 5 (2006-07, 2007-08, 2008-09, 2012-13, 2013-14) League Name: German Division 1 - Seasons: 14 (2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) League Name: German Division 2 - Seasons: 8 (2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) Country: Italian - Leagues: 2 League Name: Italian Cup - Seasons: 2 (2005-06, 2007-08) League Name: Italian Division 1 - Seasons: 14 (2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) Country: Japanese - Leagues: 3 League Name: Japanese Division 1 - Seasons: 14 (2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015-Stage 1, 2015-Stage 2, 2016-Stage 1, 2016-Stage 2) League Name: Japanese Division 2 - Seasons: 9 (2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016) League Name: Japanese League Cup - Seasons: 9 (2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014) Country: Korean - Leagues: 1 League Name: Korean Division 1 - Seasons: 1 (2016) Country: Mexican - Leagues: 1 League Name: Mexican Premier - Seasons: 6 (2014-15 (AP), 2014-15(CL), 2015-16(AP), 2015-16(CL), 2016-17(AP), 2016-17(CL)) Country: Norwegian - Leagues: 1 League Name: Norwegian Division 1 - Seasons: 12 (2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016) Country: Portuguese - Leagues: 1 League Name: Portuguese Premier - Seasons: 9 (2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) Country: Russian - Leagues: 1 League Name: Russian Premier - Seasons: 3 (2014-2015, 2015-2016, 2016-2017) Country: Scottish - Leagues: 2 League Name: Scottish League Cup - Seasons: 5 (2005-06, 2007-08, 2008-09, 2009-10, 2012-13) League Name: Scottish Premier - Seasons: 12 (2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) Country: Spanish - Leagues: 1 League Name: Spanish Division 1 - Seasons: 13 (2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017) Country: Swedish - Leagues: 1 League Name: Swedish Division 1 - Seasons: 12 (2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016) Country: US - Leagues: 2 League Name: US Football League Playoffs - Seasons: 8 (2007, 2009, 2010, 2011, 2012, 2013, 2014, 2015) League Name: US Football League - Seasons: 10 (2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
One with reds and goals separated, and one where they are together as "Events" The columns for each are as follows:
Goals and Reds SeparatedCode: Select all
SEASON, DATE, HOME, AWAY, HHT_GOALS, AHT_GOALS, HFT_GOALS, AFT_GOALS, TOTAL_GOALS, HOME_REDS, AWAY_REDS, TOTAL_REDS, RED_1_TIME, RED_1_TEAM, RED_2_TIME, RED_2_TEAM, RED_3_TIME, RED_3_TEAM, RED_4_TIME, RED_4_TEAM, RED_5_TIME, RED_5_TEAM, RED_6_TIME, RED_6_TEAM, RED_7_TIME, RED_7_TEAM, RED_8_TIME, RED_8_TEAM, RED_9_TIME, RED_9_TEAM, RED_10_TIME, RED_10_TEAM, RED_11_TIME, RED_11_TEAM, RED_12_TIME, RED_12_TEAM, RED_13_TIME, RED_13_TEAM, RED_14_TIME, RED_14_TEAM, RED_15_TIME, RED_15_TEAM, GOAL_1_TIME, GOAL_1_TEAM, GOAL_2_TIME, GOAL_2_TEAM, GOAL_3_TIME, GOAL_3_TEAM, GOAL_4_TIME, GOAL_4_TEAM, GOAL_5_TIME, GOAL_5_TEAM, GOAL_6_TIME, GOAL_6_TEAM, GOAL_7_TIME, GOAL_7_TEAM, GOAL_8_TIME, GOAL_8_TEAM, GOAL_9_TIME, GOAL_9_TEAM, GOAL_10_TIME, GOAL_10_TEAM, GOAL_11_TIME, GOAL_11_TEAM, GOAL_12_TIME, GOAL_12_TEAM, GOAL_13_TIME, GOAL_13_TEAM, GOAL_14_TIME, GOAL_14_TEAM, GOAL_15_TIME, GOAL_15_TEAM
Goals and Reds Together as EventsPlease let me know if I should make any changes to the below columns, or the CSV's at allCode: Select all
SEASON, DATE, HOME, AWAY, HHT_GOALS, AHT_GOALS, HFT_GOALS, AFT_GOALS, TOTAL_GOALS, HOME_REDS, AWAY_REDS, TOTAL_REDS, EVENT_1_TIME, EVENT_1_TEAM, EVENT_1_TYPE, EVENT_2_TIME, EVENT_2_TEAM, EVENT_2_TYPE, EVENT_3_TIME, EVENT_3_TEAM, EVENT_3_TYPE, EVENT_4_TIME, EVENT_4_TEAM, EVENT_4_TYPE, EVENT_5_TIME, EVENT_5_TEAM, EVENT_5_TYPE, EVENT_6_TIME, EVENT_6_TEAM, EVENT_6_TYPE, EVENT_7_TIME, EVENT_7_TEAM, EVENT_7_TYPE, EVENT_8_TIME, EVENT_8_TEAM, EVENT_8_TYPE, EVENT_9_TIME, EVENT_9_TEAM, EVENT_9_TYPE, EVENT_10_TIME, EVENT_10_TEAM, EVENT_10_TYPE, EVENT_11_TIME, EVENT_11_TEAM, EVENT_11_TYPE, EVENT_12_TIME, EVENT_12_TEAM, EVENT_12_TYPE, EVENT_13_TIME, EVENT_13_TEAM, EVENT_13_TYPE, EVENT_14_TIME, EVENT_14_TEAM, EVENT_14_TYPE, EVENT_15_TIME, EVENT_15_TEAM, EVENT_15_TYPE, EVENT_16_TIME, EVENT_16_TEAM, EVENT_16_TYPE, EVENT_17_TIME, EVENT_17_TEAM, EVENT_17_TYPE, EVENT_18_TIME, EVENT_18_TEAM, EVENT_18_TYPE, EVENT_19_TIME, EVENT_19_TEAM, EVENT_19_TYPE, EVENT_20_TIME, EVENT_20_TEAM, EVENT_20_TYPE, EVENT_21_TIME, EVENT_21_TEAM, EVENT_21_TYPE, EVENT_22_TIME, EVENT_22_TEAM, EVENT_22_TYPE, EVENT_23_TIME, EVENT_23_TEAM, EVENT_23_TYPE, EVENT_24_TIME, EVENT_24_TEAM, EVENT_24_TYPE, EVENT_25_TIME, EVENT_25_TEAM, EVENT_25_TYPE, EVENT_26_TIME, EVENT_26_TEAM, EVENT_26_TYPE, EVENT_27_TIME, EVENT_27_TEAM, EVENT_27_TYPE, EVENT_28_TIME, EVENT_28_TEAM, EVENT_28_TYPE, EVENT_29_TIME, EVENT_29_TEAM, EVENT_29_TYPE, EVENT_30_TIME, EVENT_30_TEAM, EVENT_30_TYPE
I've tried, best I can, to go through the data and make sure all games are there. I've not spotted any leagues with missing games as of yet. BUT PLEASE LET ME KNOW IF YOU SPOT ANY ISSUES.
Just to note: The data is all scraped from HKJC, so if there are any errors it would be down to the data they provided. (The dates I've been told are in HK time)
Cheers,
Adam
I've just tried it and its still active and working