Any way to backtest an automated strategy?

User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

ShaunWhite wrote:
Tue Nov 20, 2018 2:45 pm
As for a product that will unlock the treasure chest, the potential user base is smaller than the potential gains to be had keeping it to yourself.
there's a lot to be said for that. a sneaky alternative that i also mentioned a while back and was reminded of by a pm from another user (which doesn't revue market fill etc), would be to have a structure under the BA local folder that contained blank json placeholder files (horses, football, tennis etc).

given the structure of these files (if you were tech savvy) you could incorporate your own data that you'd gathered either from BA or downloaded from BF. There would therefore be no intent from BA to supply this data - only to facilitate it's use inside BA against the official FREE BF json downloads..

maybe i'm being niaive but this would allow BA to use legally sourced json inside the BA framework.
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

ShaunWhite, I agree with you, writing a good backtesting tool is really difficult. But most if us are not trading millions at once, like your FX company. If I found a tool which does not take into account fill rates, I would be very happy! This parameter is very important when there is high volatility. For big tennis games, it is not so important: odds change when score changes, and keep more or less the same value between score updates, with usually good liquidity.
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

Backtesting is a key feature in FX world used VERY widly. I think Sport betting is mature enough to use this kind of tool. I am pretty sure a lot of traders would use it.
As an exemple, I realized last year that cfd on DAX was losing value during the first minute after opening of the market. I write a robot, backtested successfully (with a very well know fx trading software) and played in real market. For 3 months, I made a 800% roi, then I lost 50% of it: the market moved, the strategy was not valid anymore. I stoped with a 400% roi. Just to say that backtesting has been really helpfull for me in fx context. I don’t see why it would not be the case in bet trading.
User avatar
wearthefoxhat
Posts: 3205
Joined: Sun Feb 18, 2018 9:55 am

I can imagine backtesting a trading platform, along with a projection of likely performance in the market too, would be quite a powerful tool. I should imagine something already exists in the big trading houses, anything that might create an edge for the analysis and research team. I wouldn't be surprised if the economists that forecast/project inflation figures have something on these lines. If they haven't, we're all doomed... :shock:

If it did exist in B.A, info like fill rates and market reaction/over reaction could be given a %error factor. (adjustable). The operator could review the market performance at close of play and compare previous projections. Reviewing a strategy performance should always be a priority and a way of learning too.

Can Bet Angel do this? I can only imagine it existing as a "bolt on" to their existing software.

Would they sell it? I know if I produced one, it would give me an edge, so I would think twice about it for sure.

One caveat though...backtesting could lead to backfitting, the bane of many systems and methods in the past.
sionascaig
Posts: 1053
Joined: Fri Nov 20, 2015 9:38 am

The gambler who cracked the horse racing code...

https://www.bloomberg.com/news/features ... acing-code

Interesting article which seems to involve a lot backtesting or at least refinements...
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

wearthefoxhat wrote:
Wed Nov 21, 2018 8:57 am
Would they sell it? I know if I produced one, it would give me an edge, so I would think twice about it for sure.

One caveat though...backtesting could lead to backfitting, the bane of many systems and methods in the past.
the way that i view this is that it's purely an endpoint substitute (the historic json data) for the live streaming data. to all intents and purposes, it would behave in exactly the same way as if you were running the automation (or manual session) against a current event that was in play (or xx minutes PRE event).

to this end, yes, it would bring a huge advantage to explore edges. i don't however agree that you would necessarily use it to backfit data (tho for sure you could cherry pick historical events that fitted your rules). the driver here is to fine tune (or discard) automation (or manual habits) by using multiple events that you can review failure points against.
User avatar
Naffman
Posts: 5626
Joined: Sun Aug 11, 2013 5:46 am

Without backtesting what do we have? If I have a hunch that a betting system may work after watching the markets for months/years how do I actually know it's profitable without backtesting it?

And what about tennis stats for example? What's the point of noting the % of first serves a player has or how many break points they win if it didn't mean anything?

If we don't look to the past what have we actually got?
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

It sort of feels like this discussion and has splintered and become a bit amorphous.

I make extensive use of historical data, but not to create a new strategy by seeing if it works by backtesting. But to model the market and understand how it works with a view to exploiting that understanding of the market.

I honestly can't think of ever having used a backtested strategy in the market cold from a set of data. I've only ever used it to understand the way the market works and look for exploitable patterns.
User avatar
PDC
Posts: 2272
Joined: Sun Jul 24, 2016 5:52 pm

Any chance it could be moved to the appropriate forum please, not sure why it is in Australian as it seems to have no relevance to Australian as it is causing me to get notifications on a topic not to do with Australian markets.
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

Naffman wrote:
Wed Nov 21, 2018 1:36 pm
Without backtesting what do we have? If I have a hunch that a betting system may work after watching the markets for months/years how do I actually know it's profitable without backtesting it?
+1
The problems only arise when people use the same data for testing that they use for edge hunting. That's their problem not a problem with the underlying methodology.

There's probably a degree of cost/benefit going on here, the advocate for just having a go in live probably regards their time backtesting as being more valuable than the costs involved in trialing a system with cash. That type of conflict is typical of the difference between methods that are suitable for a fledgling trader and ones that are suitable for a trader who's had a fair degree of success. It's also more of a dialema for a manual trader who has to eschew paid work for speculative research, rather than for someone using automation who would otherwise be just looking out of the window daydreaming.
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

Euler wrote:
Wed Nov 21, 2018 2:14 pm
It sort of feels like this discussion and has splintered and become a bit amorphous.

I make extensive use of historical data, but not to create a new strategy by seeing if it works by backtesting. But to model the market and understand how it works with a view to exploiting that understanding of the market.

I honestly can't think of ever having used a backtested strategy in the market cold from a set of data. I've only ever used it to understand the way the market works and look for exploitable patterns.
If it's become amorphous, then let me (try to) steer it back on course. As mentioned earlier, I see backtesting (as part of Guardian) as being appropriate where:

1. The intention is clearly understood; you are honing a system against a diverse set of markets in order to test hypothesis
2. A rule that has evolved and has worked previously begins to lose its edge. In this case, it would be great to try to find (via historical data) when and where the change has occurred. if the issue can be identified, then potentially it can be remedied.
3. A pattern has been identified but no intention has been declared. In this case, backtesting would provide insight against a defined set of rules set up to test the rigidity of this pattern (i.e - black box testing)

In short, I'm not advocating backtesting as a way to haphazerdly *find* some silver bullet. I'm rather suggesting that the hours spent looking out the window (thanks shaun :D) could be better utilised by proactively troubleshooting issues with rulesets via instant recall on market replay. also, this would potentially mitigate the scores of BF account suspensions reported, due to practice mode being overused (in BF's opinion)
User avatar
PDC
Posts: 2272
Joined: Sun Jul 24, 2016 5:52 pm

PDC wrote:
Wed Nov 21, 2018 2:21 pm
Any chance it could be moved to the appropriate forum please, not sure why it is in Australian as it seems to have no relevance to Australian as it is causing me to get notifications on a topic not to do with Australian markets.
Thank you for moving it.
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

Euler wrote:
Wed Nov 21, 2018 2:14 pm
I can't think of ever having used a backtested strategy in the market.

I've only ever used it to look for exploitable patterns.
Can someone please explain to me the difference between those two statements, in the context of a market? Because I just don't get it!

& I don't think I'm the only one confused, because a lot of the Q&A comments seem incredibly backtest oriented also.
User avatar
Naffman
Posts: 5626
Joined: Sun Aug 11, 2013 5:46 am

ruthlessimon wrote:
Wed Nov 21, 2018 6:22 pm
Euler wrote:
Wed Nov 21, 2018 2:14 pm
I can't think of ever having used a backtested strategy in the market.

I've only ever used it to look for exploitable patterns.
Can someone please explain to me the difference between those two statements, in the context of a market? Because I just don't get it!

& I don't think I'm the only one confused, because a lot of the Q&A comments seem incredibly backtest oriented also.
+1

Sounds exactly the same I agree.

If you find backtesting works then do it, I certainly find it works
spreadbetting
Posts: 3140
Joined: Sun Jan 31, 2010 8:06 pm

I'd see them as different, backtesting is simply testing an idea against a data set i.e. I think laying every 1.01 will reap rewards so I test it against the data then start booking my holidays as this time next year we'll be millionaires with such a ROI. Looking for exploitable patterns would simply be running variable queries thru the data to find any positive outcomes then deciding if you can exploit those peaks in the real world. But only Peter can really tell you what he meant by his post.
Post Reply

Return to “Betfair Data”