Any way to backtest an automated strategy?

User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

Euler wrote:
Tue Nov 20, 2018 11:16 am
You may even find that your presence in the market may influence the outcome.
Euler, this is true only if your stakes are big compared to available/traded volume in the market, right?
User avatar
Euler
Posts: 24701
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

The way to look at Tennis is that a player can win a point on serve. So say that's a 50% chance, then to win 40-0 you need to calculate 0.5^4

If you look at all available test data and find that the average chance of winning on serve is 50% then backtest that to all available data and odds then you will find it matches the model perfectly. So the defining characteristic actually becomes the chance of winning a point on serve in that particular match and how it is changing and nothing to do with the historic chance of winning a point on serve. The odds will move to reflect the number of paths left to finish the match. If you model it, you can plot all available paths and work to anticipate them.

I suggest the best way to test this is to model just one game of tennis then backtest to check. Tennis is specifically related to the underlying score and any change in momentum or physical condition of the player.
User avatar
Euler
Posts: 24701
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

pdenoeud wrote:
Tue Nov 20, 2018 1:01 pm
Euler wrote:
Tue Nov 20, 2018 11:16 am
You may even find that your presence in the market may influence the outcome.
Euler, this is true only if your stakes are big compared to available/traded volume in the market, right?
It would be proportional, yes.
xitian
Posts: 457
Joined: Fri Jul 08, 2011 2:08 pm

Euler wrote:
Tue Nov 20, 2018 12:48 pm
xitian wrote:
Tue Nov 20, 2018 12:27 pm
So any methodologies different to those that BetAngel supports or recommends probably won’t be received well.
That's not true or fair, that's more or less saying that I'm not allowed to have an opinion. The only reason I have opinions such as one of this thread is because of my experience in the market. It's nothing to do with promoting a viewpoint. I actually stay out of a lot of active discussions to avoid tainting them.
It may not be to do with promoting a viewpoint but you invariably do because of who you are as the BetAngel owner, feature designer, and “most profitable horse trader ever”. Perhaps you should create an anonymous profile and post from that if you truly don’t want to influence people.
pdenoeud wrote:
Tue Nov 20, 2018 12:52 pm
That is a lot of work on a path which might lead nowhere (according to Euler), so I am hesitating:-)
Exactly my point. Don’t let Peter’s weight sway you.

Peter’s wrong here, backtesting can be a hugely powerful tool if used correctly.

You already have the right skills, and that’s really the biggest time sink (learning the skills). Writing a backtesting simulation system might take a couple months depending how much time you spend on it? Imagine how many years you could use it in future though, and how many ideas you can trial. Just make sure you keep some out of sample data, and make sure you know what assumptions you’re making when you backtest/simulate.
xitian
Posts: 457
Joined: Fri Jul 08, 2011 2:08 pm

Euler wrote:
Tue Nov 20, 2018 1:06 pm
The way to look at Tennis is that a player can win a point on serve. So say that's a 50% chance, then to win 40-0 you need to calculate 0.5^4

If you look at all available test data and find that the average chance of winning on serve is 50% then backtest that to all available data and odds then you will find it matches the model perfectly. So the defining characteristic actually becomes the chance of winning a point on serve in that particular match and how it is changing and nothing to do with the historic chance of winning a point on serve. The odds will move to reflect the number of paths left to finish the match. If you model it, you can plot all available paths and work to anticipate them.

I suggest the best way to test this is to model just one game of tennis then backtest to check. Tennis is specifically related to the underlying score and any change in momentum or physical condition of the player.
So wouldn’t it make sense to have a backtesting system where you can test your model of calculating or predicting a server’s chance of winning a point?

Then I could see how accurate I had been over 1000 matches. Or tweak my model to fit 500 matches and test 500 out of sample matches?
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

Euler wrote:
Tue Nov 20, 2018 11:13 am
If you backtest against scores you will find the market is pretty much perfect in terms of pricing and scores. It's not what has happened that's going to determine your profitability, it's predicting what will happen.

Tennis Trader perfectly models Tennis scores and odds, so playing with that will allow you to plot all possible paths and your goal as a trader is to anticipate price action rather than follow it.
BUt for instance, tennis matrix (game and set) show estimated odds in the future. I guess you refer to that in your post. Those estimates can be used to start or finish a trade at "good" entry/exit points. But those estimates are themselves calculated accoring to historical data I guess. So the past helps us modeling the future and take good decisions.

Also, if "he market is pretty much perfect in terms of pricing and scores", then those "good" entry/exit points wouldn't really exist : the odds would always reflect the estimated ratio reward/risk. And so the only way to be profitable would be to consider other parameters (than score and odds) in order to take the right decisions (like how tired/motivated/emotionaly balanced/... a player seems to be for instance). Thus you would need to follow the match live in order to take the right decisions...
Last edited by pdenoeud on Tue Nov 20, 2018 1:43 pm, edited 1 time in total.
User avatar
Euler
Posts: 24701
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

It may not be to do with promoting a viewpoint but you invariably do because of who you are as the BetAngel owner, feature designer, and “most profitable horse trader ever”. Perhaps you should create an anonymous profile and post from that if you truly don’t want to influence people.
TBH, If I used an anon profile then people would claim I was being underhanded. Ultimately I've learnt that If you have a high profile you can't win really even if you put your money where your mouth is, I don't really have an issue with that, it's just the way it is. All I've ever wanted is for the markets to gain broader legitimacy against the prevailing negative background they attract.

Anybody that does anything clever or innovative is to be applauded.

My views on backtesting are not that it's useless, it's just that if you just use that on its own, it's not much use. If you use it to model or simulate the market then it will be obviously useful.
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

Euler wrote:
Tue Nov 20, 2018 1:26 pm
It may not be to do with promoting a viewpoint but you invariably do because of who you are as the BetAngel owner, feature designer, and “most profitable horse trader ever”. Perhaps you should create an anonymous profile and post from that if you truly don’t want to influence people.
TBH, If I used an anon profile then people would claim I was being underhanded. Ultimately I've learnt that If you have a high profile you can't win really even if you put your money where your mouth is, I don't really have an issue with that, it's just the way it is. All I've ever wanted is for the markets to gain broader legitimacy against the prevailing negative background they attract.

Anybody that does anything clever or innovative is to be applauded.

My views on backtesting are not that it's useless, it's just that if you just use that on its own, it's not much use. If you use it to model or simulate the market then it will be obviously useful.
i think all i've ever suggested (both here and in my BAMI post) is that BA develop the core engine further to allow historical markets to be downloaded and backtested inside the current BA UI/guardian.

Apart from generating additional revenue (for BA), it would allow users to decide from the get go whether a strategy was viable or not, rather than spending weeks/months finessing something that is ultimately fragile and suceptible to minor changes in the market(s). the ability to (more) *easily* identify issues in the context of a set of historical markets, runnning against an automation rule, (whislt using all the standard BA features) is invaluable - and obviously as more features are added, all the more valuable.

[edit] - i'd go as far to say that such a piece of functionality would make the testing/validation of new features (at BA HQ) far easier to boot...
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

I think there's several things being discussed here, strategy hunting, proof of concept, refining and tuning etc. All are subtly different and all do benefit to some extent or another from looking at historical data.

'Back testing' is rather more complex than just looking at snapshots of historic data, to be effective I beleive it has to take the form of a full simulation to take into account the things Peter talks about such as fill rates. Doing so is extremely complex as you need to remember what money in the markets wouldn't be there if you'd already taken it, and what money should be in the market if you'd been putting it there previously. Even small amounts have an impact as the sports markets are tiny, often just 2 digits at any given price.

I've just come back from a weekend away at my bro-in-law's (so what you say)....he happens to be head of dev at Morgan Stanley FX which is a strange coincidence as I was in a similar position at OMAM, yet how I met my wife was nothing to do with the city...I digress. We talked bout strategy development and testing quite a lot although details were scant as security there is extreme to say the least. The message is very similar to the way we'd ideally operate; establish a concept through fundamentals, prove that concept in static data, and then use a highly sophisicated simulator (aka backtest). When that's done, forward testing usings live data and simulated orders. At no point do they consider throwing a few million at the market to see if an idea floats, that's only done when the simulation is shown to be profitable beyond the margin for error expected during implementation.

As for the tools you need then that depends what you want to achieve, you're not going to be able to do large scale HFT with excel, and if you have a system capable of that then you wouldn't use it to trade one or two specific setups a day which would be best done manually. A lot is said about how long it takes to become a profitable trader (for arguments sake call it a year), depending on the route you choose it might be the case that 10 months out of that year is learning and building a system that suits your needs. If that's not appealing, then it's perfectly possible to use a product like BA but you may have to take a more suck-it-and-see approach, testing ideas in the live markets.

It's all a balance of what type of trader you want to be, what skills you bring to the table, and what unpaid work you're prepared to put in to give you a fighting chance.... and how much money you have to invest in ideas. One talented trader I know says you should consider the costs as the price of the ticket to enter a prize draw; how likely you feel you are to win and the size of the prize you want dictates the amount you're prepared to pay for that ticket.
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

As for a product that will unlock the treasure chest, the potential user base is smaller than the potential gains to be had keeping it to yourself. And what would you charge for it, what's it worth? £10k per licence? Then data fees on top of that.
I mentioned security at MSFX, just fyi, nobody has external email, phones handed in under lock and key, no printing documentation or code fragments, instant dismissal and probably legal action if those rules are broken. It's like GCHQ.
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

ShaunWhite wrote:
Tue Nov 20, 2018 2:45 pm
As for a product that will unlock the treasure chest, the potential user base is smaller than the potential gains to be had keeping it to yourself.
there's a lot to be said for that. a sneaky alternative that i also mentioned a while back and was reminded of by a pm from another user (which doesn't revue market fill etc), would be to have a structure under the BA local folder that contained blank json placeholder files (horses, football, tennis etc).

given the structure of these files (if you were tech savvy) you could incorporate your own data that you'd gathered either from BA or downloaded from BF. There would therefore be no intent from BA to supply this data - only to facilitate it's use inside BA against the official FREE BF json downloads..

maybe i'm being niaive but this would allow BA to use legally sourced json inside the BA framework.
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

ShaunWhite, I agree with you, writing a good backtesting tool is really difficult. But most if us are not trading millions at once, like your FX company. If I found a tool which does not take into account fill rates, I would be very happy! This parameter is very important when there is high volatility. For big tennis games, it is not so important: odds change when score changes, and keep more or less the same value between score updates, with usually good liquidity.
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

Backtesting is a key feature in FX world used VERY widly. I think Sport betting is mature enough to use this kind of tool. I am pretty sure a lot of traders would use it.
As an exemple, I realized last year that cfd on DAX was losing value during the first minute after opening of the market. I write a robot, backtested successfully (with a very well know fx trading software) and played in real market. For 3 months, I made a 800% roi, then I lost 50% of it: the market moved, the strategy was not valid anymore. I stoped with a 400% roi. Just to say that backtesting has been really helpfull for me in fx context. I don’t see why it would not be the case in bet trading.
User avatar
wearthefoxhat
Posts: 3205
Joined: Sun Feb 18, 2018 9:55 am

I can imagine backtesting a trading platform, along with a projection of likely performance in the market too, would be quite a powerful tool. I should imagine something already exists in the big trading houses, anything that might create an edge for the analysis and research team. I wouldn't be surprised if the economists that forecast/project inflation figures have something on these lines. If they haven't, we're all doomed... :shock:

If it did exist in B.A, info like fill rates and market reaction/over reaction could be given a %error factor. (adjustable). The operator could review the market performance at close of play and compare previous projections. Reviewing a strategy performance should always be a priority and a way of learning too.

Can Bet Angel do this? I can only imagine it existing as a "bolt on" to their existing software.

Would they sell it? I know if I produced one, it would give me an edge, so I would think twice about it for sure.

One caveat though...backtesting could lead to backfitting, the bane of many systems and methods in the past.
sionascaig
Posts: 1053
Joined: Fri Nov 20, 2015 9:38 am

The gambler who cracked the horse racing code...

https://www.bloomberg.com/news/features ... acing-code

Interesting article which seems to involve a lot backtesting or at least refinements...
Post Reply

Return to “Betfair Data”