Any way to backtest an automated strategy?

User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

Euler wrote:
Wed Nov 21, 2018 2:14 pm
It sort of feels like this discussion and has splintered and become a bit amorphous.

I make extensive use of historical data, but not to create a new strategy by seeing if it works by backtesting. But to model the market and understand how it works with a view to exploiting that understanding of the market.

I honestly can't think of ever having used a backtested strategy in the market cold from a set of data. I've only ever used it to understand the way the market works and look for exploitable patterns.
If it's become amorphous, then let me (try to) steer it back on course. As mentioned earlier, I see backtesting (as part of Guardian) as being appropriate where:

1. The intention is clearly understood; you are honing a system against a diverse set of markets in order to test hypothesis
2. A rule that has evolved and has worked previously begins to lose its edge. In this case, it would be great to try to find (via historical data) when and where the change has occurred. if the issue can be identified, then potentially it can be remedied.
3. A pattern has been identified but no intention has been declared. In this case, backtesting would provide insight against a defined set of rules set up to test the rigidity of this pattern (i.e - black box testing)

In short, I'm not advocating backtesting as a way to haphazerdly *find* some silver bullet. I'm rather suggesting that the hours spent looking out the window (thanks shaun :D) could be better utilised by proactively troubleshooting issues with rulesets via instant recall on market replay. also, this would potentially mitigate the scores of BF account suspensions reported, due to practice mode being overused (in BF's opinion)
User avatar
PDC
Posts: 2272
Joined: Sun Jul 24, 2016 5:52 pm

PDC wrote:
Wed Nov 21, 2018 2:21 pm
Any chance it could be moved to the appropriate forum please, not sure why it is in Australian as it seems to have no relevance to Australian as it is causing me to get notifications on a topic not to do with Australian markets.
Thank you for moving it.
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

Euler wrote:
Wed Nov 21, 2018 2:14 pm
I can't think of ever having used a backtested strategy in the market.

I've only ever used it to look for exploitable patterns.
Can someone please explain to me the difference between those two statements, in the context of a market? Because I just don't get it!

& I don't think I'm the only one confused, because a lot of the Q&A comments seem incredibly backtest oriented also.
User avatar
Naffman
Posts: 5626
Joined: Sun Aug 11, 2013 5:46 am

ruthlessimon wrote:
Wed Nov 21, 2018 6:22 pm
Euler wrote:
Wed Nov 21, 2018 2:14 pm
I can't think of ever having used a backtested strategy in the market.

I've only ever used it to look for exploitable patterns.
Can someone please explain to me the difference between those two statements, in the context of a market? Because I just don't get it!

& I don't think I'm the only one confused, because a lot of the Q&A comments seem incredibly backtest oriented also.
+1

Sounds exactly the same I agree.

If you find backtesting works then do it, I certainly find it works
spreadbetting
Posts: 3140
Joined: Sun Jan 31, 2010 8:06 pm

I'd see them as different, backtesting is simply testing an idea against a data set i.e. I think laying every 1.01 will reap rewards so I test it against the data then start booking my holidays as this time next year we'll be millionaires with such a ROI. Looking for exploitable patterns would simply be running variable queries thru the data to find any positive outcomes then deciding if you can exploit those peaks in the real world. But only Peter can really tell you what he meant by his post.
PeterLe
Posts: 3715
Joined: Wed Apr 15, 2009 3:19 pm

Ive never used back testing either (although I know people who do to give you an 'indication')
Even if you find a pattern, lets say laying at 1.01 on certain courses..it won't necessarily work (I read this link by Smarkets with interest this week.. link posted by smarkets https://en.wikipedia.org/wiki/Adverse_selection). In the instances you are wrong, your bet will get matched and the times you are right, your bet will get matched "sometimes"
(Which incidentally, That is why you simply cant take a losing strategy and do the opposite to develop a winning strategy)
Regards
Peter
Atho55
Posts: 637
Joined: Tue Oct 06, 2015 1:37 pm
Location: Home of Triumph Motorcycles

So how many commenting on this topic are actual BA users as in using the software or just forum members using other software they have developed or had made by others.... BA user
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

spreadbetting wrote:
Wed Nov 21, 2018 7:22 pm
backtesting is simply testing an idea against a data set i.e. I think laying every 1.01 will reap rewards so I test it against the data then start booking my holidays as this time next year we'll be millionaires with such a ROI.
I've just backtested it, a £2 stake @ 1.01, 1.02, 1.03

6mths (6000+ markets)

1.01: +£51
1.02: -£68
1.03: -£187

Backtesting has allowed me to see that there's 0 leeway - 1 tick matters hugely - you won't catch me trading this £mil dollar strategy ;)

I prefer to see something working at a range of prices - just so I know if I miss the fill it's ok.

Given that same token, Peter doesn't like back-testing, & Peter is a far far better trader than me, hence why I'm fascinated in this alternate view to idea generation & testing
You do not have the required permissions to view the files attached to this post.
spreadbetting
Posts: 3140
Joined: Sun Jan 31, 2010 8:06 pm

At the end of the day you need to be playing to your own strengths not Peter's. There's always more than one way to skin a cat.
foxwood
Posts: 390
Joined: Mon Jul 23, 2012 2:54 pm

ruthlessimon wrote:
Wed Nov 21, 2018 8:50 pm
I've just backtested it, a £2 stake @ 1.01, 1.02, 1.03
And there's the difference !

Your "backtesting" process shows you could make a profit of about £100 pa by laying the field at 1.01 for £2 stakes (£240 liability) but you dismissed the strategy.

[comment: if you laid for £100 stake you could make £10k - would that be attractive based on the backtest ?]

Anyway for various reasons you probably wouldn't make the profit level as per your backtest so you are probably right to let it go. There are however people on this forum who do profit nicely from that classic strategy - think on :lol:

So what is the difference between what you did and modelling the 1.01 lay ?

To me, I would model it by thinking what would happen to the market if the certain winner suddenly looks like losing. Follow that line of thinking and also consider how to manage the delay and the risk of only one runner trading at 1.01 and you end up with a strategy that could be nicely profitable with very little downside.

Backtesting is looking at consequences - the data values and moves etc shown in the data are the result of something that happened and just because there is correlation doesn't mean common causation.

Modelling is looking at causation and using the data to confirm your view of what the consequences would be.

Simples ? Probably not - may be coat time ;)
User avatar
Euler
Posts: 24701
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

I'd say that was a good summary.
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

foxwood wrote:
Wed Nov 21, 2018 10:03 pm
To me, I would model it by thinking what would happen to the market if the certain winner suddenly looks like losing. Follow that line of thinking and also consider how to manage the delay and the risk of only one runner trading at 1.01 and you end up with a strategy that could be nicely profitable with very little downside.

Backtesting is looking at consequences - the data values and moves etc shown in the data are the result of something that happened and just because there is correlation doesn't mean common causation.

Modelling is looking at causation and using the data to confirm your view of what the consequences would be.

Simples ? Probably not - may be coat time ;)
Blimey, that's gonna take some time to get my head around (getting my head away from causation having to be linked to something fundamental)

Really really good insight there though Fox, appreciate it!

No, it defo isn't simple - no-one else was able to explain that view so concisely
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

foxwood wrote:
Wed Nov 21, 2018 10:03 pm
To me, I would model it by thinking what would happen to the market if the certain winner suddenly looks like losing. Follow that line of thinking and also consider how to manage the delay and the risk of only one runner trading at 1.01 and you end up with a strategy that could be nicely profitable with very little downside.
Here's the way the way I think - if you (or anyone) can explain the error I'd be so so grateful.

We know laying 1.01 generically, back-tested is (technically) an edge - although done IRL, probably ins't (hence the dangers).

Let's say, I watch a race live, & notice 1.01 is hit extremely early, before the race has even reached half-way - & the fav gets caught.

I subsequently make a hypothesis that 1.01s hit early in a market tend to fail - & backtest it.

What's the error, basically why isn't that a model?

Cos, that's how I think, & those are the types of ideas I generate daily (albeit using technicals, rather than fundamentals)
LinusP
Posts: 1871
Joined: Mon Jul 02, 2012 10:45 pm

I don’t think I have read a thread on this forum with so many conflicting opinions, which if you ask me is a good thing.

I backtest but looking back it has been very good at telling me what definelty doesn’t work, helpful? Yes, very. I follow Peter W’s advice In getting money through the market ASAP, however the advice being given is very much dependant on the market, sport and preplay/inplay. Without knowing what different forum members specialise in you are in the dark. The sensitivity of backtesting can vary a huge amount and you want to limit this to as close to zero as you can.
sa7med
Posts: 800
Joined: Thu May 18, 2017 8:01 am

Euler wrote:
Wed Nov 21, 2018 2:14 pm
But to model the market and understand how it works with a view to exploiting that understanding of the market.
Would you explain in a little more detail what that process is? Would love to see a video on modeling, are there any existing ones? Is the result something like soccer mystic or tennis trader? How would it work for racing? Are you modeling likely odds or momentum of trends or everything and anything?
Post Reply

Return to “Betfair Data”