Do you Collect Data on Betfair Markets?

Post Reply

Do you Collect Data

Yes for Loads of Sports
6
18%
Yes for a Few Sports
17
52%
No Don't Collect Any
10
30%
 
Total votes: 33
User avatar
Dallas
Posts: 22673
Joined: Sun Aug 09, 2015 10:57 pm
Location: Working From Home

Just been doing my regular backup of data etc and noticed that I had just under 18GB of data to move so thought I'd see if the community also collects any data and if so to what degree?
User avatar
Derek27
Posts: 23471
Joined: Wed Aug 30, 2017 11:44 am
Location: UK

Keep meaning to and will have to as I'm starting automation but just haven't had time as yet.
User avatar
northbound
Posts: 737
Joined: Mon Mar 20, 2017 11:22 pm

I’m not a hoarder but wrote software that takes a snapshot of metrics (ltp, best back, best lay, wap, wom, traded ladder, etc) at different points in time for a number of sports.

For example on horses I take snapshots at 10min, 3min, 1min, 0min before the off. I also integrate this data manually with other stuff, for example Racing Post tips for every race, etc.
User avatar
Archangel
Posts: 1987
Joined: Thu Jun 27, 2013 3:03 pm
Location: Polo Lounge, Beverly Hills Hotel

One question I have is how do people who collect data here analyse the data?

I am assuming there would be data in some sort of a database, so are there specific tools ye use to get what ye want out of that data to make sense of it so to speak ?
CallumPerry
Posts: 575
Joined: Wed Apr 19, 2017 5:12 pm
Location: Wolverhampton

Got about a weeks worth of greyhounds just to model some strategies and then replay them, I've built my own way of running simulations which is just recently working as I hoped it would months ago. Got about a month's worth of horses and no football yet but my hope is to deploy this full-time soon on those three sports. Not sure about how much in gb/mb I have, will have to have a look tomorrow if I get chance.
User avatar
northbound
Posts: 737
Joined: Mon Mar 20, 2017 11:22 pm

Archangel wrote:
Sun Jun 23, 2019 10:41 pm
One question I have is how do people who collect data here analyse the data?
Personally it all starts with a real world hypothesis. Stupid example: horses with low draw at Chester 5f races getting backed before the off.

Then build some software script that extracts the relevant metrics from the database and exports them into Excel / CSV format.

Play with Excel filters to see if the original hypothesis has legs. For example simulate backing said horses 10min before the off and laying them at post time.

Is it profitable? What about if I filter in only stalls 1-2? What about 1-2-3? What about only horses in those stalls and a certain price range? What happens in-play? Could I take 25p or 50p profit fir each £1 on most of these horses?

And so on...
User avatar
Archangel
Posts: 1987
Joined: Thu Jun 27, 2013 3:03 pm
Location: Polo Lounge, Beverly Hills Hotel

^ I get you. The devil is in the detail as they say
PeterLe
Posts: 3715
Joined: Wed Apr 15, 2009 3:19 pm

worth looking on udemy.com - under data science
Sounds simple to extract data, analyse and then create a strategy, but there is a danger of over fitting and many other ways you could go astray
You would need two sets of data, one out of sample, so you can test your theories etc

Anyway, Im no expert, Im only two steps in front of you :)

(By the way; if it seems too complicated...I watched a ted talk yesterday about learning. Yes it has been stated that you can become an expert of a skill in 10,000 hours, but this guy stated that with the right application and focus, yo can make massive headway with just 20 hours)...heres the link https://www.youtube.com/watch?v=5MgBikgcWnY

You may never get to the lofty heights that some achieve on here, but even if you get above average, then that should be enough
greenmark
Posts: 4948
Joined: Mon Jan 29, 2018 2:15 pm

10,000 hrs is in the ball park in my experience. Improve on that and you have a natural aptitude that you should exploit.
User avatar
firlandsfarm
Posts: 2688
Joined: Sat May 03, 2014 8:20 am

northbound wrote:
Mon Jun 24, 2019 12:19 am
Personally it all starts with a real world hypothesis … software script that extracts the relevant metrics from the database and exports them into Excel / CSV format … Play with Excel filters … Is it profitable? What about …
I basically do the same using Access as well as Excel though I'm needing to gradually upgrade to SQL Server but keeping Access as my front end. The skill is not so much the extracting and manipulating of the data, I see it that the skill is spotting the difference between a statistical freak and a trend and not over fitting. For example and building on Northbound's example, if the profit was found in stalls 1 and 3 but not in stall 2 I would say that is a statistical freak but then you have to judge which is the freak … is it the stalls' 1 and 3 return or the stall 2 return!
LinusP
Posts: 1871
Joined: Mon Jul 02, 2012 10:45 pm

3 MySQL databases, 100gb for order / market data (anything that comes from the API), 200gb backtesting database that contains processed streaming data and another which is more of a ‘data lake’ containing aggregation of everything (including logs) which I think is around 200gb.

I then use S3 for all logs (last 120 days) and raw streaming data:

- UK/IE racing since streaming began
- AUS/US racing past 12 months
- Greyhound racing past 12 months
- Tennis, major tournaments
- Other random sports markets
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

firlandsfarm wrote:
Tue Jun 25, 2019 4:56 am
The skill is not so much the extracting and manipulating of the data, I see it that the skill is spotting the difference between a statistical freak and a trend and not over fitting.
I like that :)

Does it look "beautiful"? Is it "elegant"? Does it have smooth, defined, consistent curves - LOL

I'm pretty certain someone like Peter will get joy at the "aesthetics" of his strategies

-actually here's a checklist:

A proof that uses a minimum of additional assumptions or previous results.
A proof that is unusually succinct.
A proof that derives a result in a surprising way
A proof that is based on new and original insights.
A method of proof that can be easily generalized to solve a family of similar problems.
Post Reply

Return to “Betfair Data”