Although I have no statistical proof to back this up. The feedback I have received from traders is that this is what most of them think is the case. I am sure there will be some people who think the opposite and if that makes them money then good luck to them.ShaunWhite wrote: ↑Wed Oct 25, 2017 2:23 amWould it true to say that generally, pre start markets suit the live trading approach, ie reactionary, and post start or final result markets benefit from the algo approach? Ie more prescriptive?
Strategy Development: Modelling
-
- Posts: 1075
- Joined: Fri Nov 20, 2015 9:38 am
Peters post on the masters is a very good example of building a model to identify potential trading situations: https://www.betangel.com/blog_wp/2017/0 ... s-masters/
I mentioned this approach to a friend and he said the methodology is very similar to the approach used in his longevity research. Maybe there is something to be learned from those academic types after all (or Peter)!
https://www.ed.ac.uk/igmm/news-and-even ... n-lifespan
I mentioned this approach to a friend and he said the methodology is very similar to the approach used in his longevity research. Maybe there is something to be learned from those academic types after all (or Peter)!
https://www.ed.ac.uk/igmm/news-and-even ... n-lifespan
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
However, Peter has a nice way of cutting the data.
A golf hole is categorical data.
For someone building a pre-race model, volume, price, time - pretty much all continuous data. This makes model building extremely challenging & easy to curve fit. Race type certainly could be classed as categorical data - however, this alone won't be enough to form meaningful analysis.
A golf hole is categorical data.
For someone building a pre-race model, volume, price, time - pretty much all continuous data. This makes model building extremely challenging & easy to curve fit. Race type certainly could be classed as categorical data - however, this alone won't be enough to form meaningful analysis.
-
- Posts: 3140
- Joined: Sun Jan 31, 2010 8:06 pm
I love the way people seem to assume punters are lined up in an orderly queue to place their bets at set times and use reams of data to try and prove the fact.northbound wrote: ↑Wed Oct 25, 2017 8:37 am
That’s the main reason why I’ve turned to backtesting. To get some ststistical confirmation from past data that, for example, if I see a certain setup 3min before race start, the odds end up in a certain range by the start, 70% of the time.
- northbound
- Posts: 737
- Joined: Mon Mar 20, 2017 11:22 pm
Mine was clearly an oversimplificationspreadbetting wrote: ↑Wed Oct 25, 2017 1:36 pmI love the way people seem to assume punters are lined up in an orderly queue to place their bets at set times and use reams of data to try and prove the fact.northbound wrote: ↑Wed Oct 25, 2017 8:37 am
That’s the main reason why I’ve turned to backtesting. To get some ststistical confirmation from past data that, for example, if I see a certain setup 3min before race start, the odds end up in a certain range by the start, 70% of the time.
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
Although modelling how money arrives in a pre-race market could easily prove that statement false.spreadbetting wrote: ↑Wed Oct 25, 2017 1:36 pmI love the way people seem to assume punters are lined up in an orderly queue to place their bets at set times
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
1second price data, for say the top 4 from -10min to post time, for about 6 months is a LOT of data I simply don't have.
I found things in the 200 races I collected but it didn't continue, just not enough data to slice and dice effectively.
I found things in the 200 races I collected but it didn't continue, just not enough data to slice and dice effectively.
Well as a non software developer I can say it is incredibly easy to collect data about markets using betangel as an interface. I taught my self basic VBA earlier this year but I've a data logger that can capture data using nearly all standard excel code (really only needs a few lines of vba). If you didn't want to build your own logger though you could just download them off the forum here there's plenty of great ones about. The hard part is building a data base you can query. I know what I want to look for in the data but the idea of building a VBA data base is quite daunting and I don't know where to begin at present.northbound wrote: ↑Mon Oct 23, 2017 2:58 pmInterested.
In fact, having started six months ago to trade preplay horses manually every day and NOT having become profitable yet, I’m in the process of building models to learn how certain preplay market setups traded in the past and see if I can find patterns that repeat.
Early signs are encouraging.
I gotta say that I worked as a software developer for 10+ years. Not sure how easy it would be for a non-developer to build models.
- northbound
- Posts: 737
- Joined: Mon Mar 20, 2017 11:22 pm
Agree.
In fact I recently coded a (non-Betangel) bot that collects data, but it will take a while to have enough historical data for backtesting. To accelerate the process, there's the option of purchasing 6 months' worth of Betfair historical data, but it's very expensive...
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
Peter, in your video "How to be a profitable trader" https://www.youtube.com/watch?v=JXSMXwaCH44 you discus how you score a market when you're live trading. You mention an "x out of 10" way to rate your confidence. Are there actually 10 things you look for in combination and if so, roughly how long does it take you to assess them? I guess once you know what the 10 things are you can do it pretty quickly. I also wondered if the figure of 10, being nice and round, slightly disguised or simplified what you actually do?
...and do you keep repeating the whole process while you have an open position? That must be tricky as the tendancy is to get tunnel vision on your selection plus just 1 or 2 other things once the chips are down.
...and do you keep repeating the whole process while you have an open position? That must be tricky as the tendancy is to get tunnel vision on your selection plus just 1 or 2 other things once the chips are down.
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
How much do you consider is 'enough'? I've always thought you would need at least 2 or 3 years.northbound wrote: ↑Wed Oct 25, 2017 9:04 pmAgree.
In fact I recently coded a (non-Betangel) bot that collects data, but it will take a while to have enough historical data for backtesting. To accelerate the process, there's the option of purchasing 6 months' worth of Betfair historical data, but it's very expensive...
Collecting the data is one thing, organising it and building a framework that will allow many things to be tested without re-writes and re-coding for every brainwave is the hard bit.
- northbound
- Posts: 737
- Joined: Mon Mar 20, 2017 11:22 pm
Regarding the “enough” question: the more the merrier. But perhaps, for a relative newbie like myself, even just 6-12 months of data could provide, not definitive answers, but at least some pointers on ideas that sound bright but in fact are no better than 50/50.ShaunWhite wrote: ↑Fri Oct 27, 2017 5:07 amHow much do you consider is 'enough'? I've always thought you would need at least 2 or 3 years.
Collecting the data is one thing, organising it and building a framework that will allow many things to be tested without re-writes and re-coding for every brainwave is the hard bit.
It could serve as an additional tool that, combined with observing market flow every day, can accelerate my understanding of the markets.
Regarding organising data and building a framework: you’re right, the best way is to build different “abstract” algorithms (time consuming) and then only having to configure signal values for every idea you want to test (really quick).
You would be surprised, following on from the Peter's advice (Webb and Le) a few years ago, my process is the following:ShaunWhite wrote: ↑Fri Oct 27, 2017 5:07 amHow much do you consider is 'enough'? I've always thought you would need at least 2 or 3 years.
Collecting the data is one thing, organising it and building a framework that will allow many things to be tested without re-writes and re-coding for every brainwave is the hard bit.
- Have a bad day, shit all my strategies are crap
- Load up my MySQL db combined with Tableau and Jupyter notebooks (python)
- Look for a pattern or something that looks like it could be automated
- Create strategy and start backtesting
- After 20/30 markets, get bored of backtesting, if its in profit or hovering around £0 push to UAT
- Test for at least a week, normally multiple variations with the programs storing entry points / triggers
- Review profit at the end of the week filter on the entry points, triggers, course etc
- Go back to 6 or wait for another bad day
If people are interesting I can provide some data, i.e. last weeks racing, and go through the process of setting up a database, loading, querying etc.
I am sure a lot of people would be v grateful for this process.
Its a lot of work, starting from scratch, with learning curves, in a few areas.
There are some free databases etc, but, u can run into limits .. (SQL for example, has a 10 TB maximum database size, for 2016 server).
Cleaning the data, (if u buy it), then storing it in the right fashion, with the right fields need a fair amount of thought - so then u can query it
usefully.
I would be most interested, in the process of back testing. The data I have, has been bought from the historical data site .. so, I cannot share. It may be best, for this excercise, to collect your own etc, and not fall foul of betfairs rules.
Its a lot of work, starting from scratch, with learning curves, in a few areas.
There are some free databases etc, but, u can run into limits .. (SQL for example, has a 10 TB maximum database size, for 2016 server).
Cleaning the data, (if u buy it), then storing it in the right fashion, with the right fields need a fair amount of thought - so then u can query it
usefully.
I would be most interested, in the process of back testing. The data I have, has been bought from the historical data site .. so, I cannot share. It may be best, for this excercise, to collect your own etc, and not fall foul of betfairs rules.
-
- Posts: 165
- Joined: Wed Mar 01, 2017 2:06 pm
Ive started collecting data and also use python and ipython notebooks for analysis. But my process is tedious and not very creative.LinusP wrote: ↑Sat Oct 28, 2017 9:00 amYou would be surprised, following on from the Peter's advice (Webb and Le) a few years ago, my process is the following:ShaunWhite wrote: ↑Fri Oct 27, 2017 5:07 amHow much do you consider is 'enough'? I've always thought you would need at least 2 or 3 years.
Collecting the data is one thing, organising it and building a framework that will allow many things to be tested without re-writes and re-coding for every brainwave is the hard bit.
- Have a bad day, shit all my strategies are crap
- Load up my MySQL db combined with Tableau and Jupyter notebooks (python)
- Look for a pattern or something that looks like it could be automated
- Create strategy and start backtesting
- After 20/30 markets, get bored of backtesting, if its in profit or hovering around £0 push to UAT
- Test for at least a week, normally multiple variations with the programs storing entry points / triggers
- Review profit at the end of the week filter on the entry points, triggers, course etc
- Go back to 6 or wait for another bad day
If people are interesting I can provide some data, i.e. last weeks racing, and go through the process of setting up a database, loading, querying etc.
So it would be amazing if you did have time to provide the above! Im sure it would benefit a lot of people here too!