Any way to backtest an automated strategy?

Post Reply
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

I would like to back test automated strategies (mainly for football and tennis) but I can't find any tool to do that.
Does anybody know a tool allowing to:
- create my own bot
- back test it on historical betfair data, using scores of the events tested


That feature is THE key feature to help elaborating profitable strategies. The market is quite mature now, I would be surprised that there is not such a tool somewhere.
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

pdenoeud wrote:
Mon Nov 19, 2018 6:19 pm
I would like to back test automated strategies (mainly for football and tennis) but I can't find any tool to do that.
Does anybody know a tool allowing to:
- create my own bot
- back test it on historical betfair data, using scores of the events tested


That feature is THE key feature to help elaborating profitable strategies. The market is quite mature now, I would be surprised that there is not such a tool somewhere.
there is!! - obviously can't and won't elaborate. the market has probably moved on in many ways, but i still think BA could look around the edges of this to see how user driven historical data could be integrated (by supplying a template that allowed excel based data to be replayed thro the system). i know the real market is everything, but being able to repeatedly test and hone automation tasks (especially if mechanical, rather than stats driven) is a long way down the road to fine tuning something against a raft of markets that you know to be particularly challenging.

however, myself and many have mentioned this on many occassions and for one reason or another, it's never floated the imaginations at BA HQ, so maybe you need to create a very convincing set of criteria to show how it would drive more users to BA, rather than simply serving existing users ;)

see: viewtopic.php?f=20&t=11983
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

I'm with Jim on this; maybe if I understood a bit more about what "modelling" actually means I''d have a different view. Because, to me, a "model" is basically a steam/drift prediction - which is somehow different to a backtest?

This is why I love excel (build my own sims), I can literally replay the markets thousands of times, "what if I did that, what if I did this, what if I changed this".

Something, you cannot do in a live market. But this ("what if I did that, what if I did this, what if I changed this"), is something Peter advocates; so I don't understand why he doesn't like backtesting!

The cynic in me says the reason it doesn't exist, is cos it works (if done right)
User avatar
Euler
Posts: 24701
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

Backtesting is fitting a strategy to historic data, not seeing if it is profitable. You can spend years doing this but only by actually doing it in the market can you reliably understand what the strategy does. Modelling is a way of coming up with a way the market behaves and predicting the future using that model.

I've always prefered the latter. I spent / wasted many years in the early part of my career overfitting data only to find it wasn't replicable going forward. That is why I have a preference for the latter. You will find that whatever order you put in the market to exploit a strategy, will end up influencing the outcome.
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

I'd love to go into some real detail about this - there's one (simple) graph I'd love to send to you that I think disproves it.

I haven't traded it yet, but am about to (although I can't because it's a flat race :roll: ) - ironically kinda confirmed, because it matched a trade you took in a video (but you didn't mention (what I think) was the vital variable) 8-)

I truly am fascinated to know why it's a bad way of doing it / finding edges :)
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

Euler wrote:
Mon Nov 19, 2018 8:37 pm
Backtesting is fitting a strategy to historic data, not seeing if it is profitable. You can spend years doing this but only by actually doing it in the market can you reliably understand what the strategy does. Modelling is a way of coming up with a way the market behaves and predicting the future using that model.
I do not agree. Backtesting is not just fitting the strategy to historic data, it is checking if the strategy is profitable. And then tunning the strategy to maximize the profits.
You need to back test on a big amount of events in order to make sure positive results are not due to luck.
The issue with modeling to predict the future is that you validate your model in the future... and it usually takes a long way to apply the model against a large number of events to come. And usually, you must change/tune your model a lot before it becomes profitable...
With backtesting, you can validate (unvalidate most of the times) your strategy very quickly.
Euler wrote:
Mon Nov 19, 2018 8:37 pm
I've always prefered the latter. I spent / wasted many years in the early part of my career overfitting data only to find it wasn't replicable going forward.
When backtesting, if you need to tune many parameters in order to make the strategy profitable, there are good chances that it won't be replicable in the future. The more parameters combinaisons you test, the more chances you have to finally get a profitable combinaisons... which won't be necessarely profitable in the future.
If your strategy is profitable on a large panel of input paremeters combinaisons, you get more chances to get a strategy working in the future. I did experience it sucessfully in the DAX cfd market in the past.

Euler wrote:
Mon Nov 19, 2018 8:37 pm
You will find that whatever order you put in the market to exploit a strategy, will end up influencing the outcome.
This is true if you are a big player, if your stakes are big compared to the market volume. Or if you share your strategy and it is played by many traders.
If not, why would small stakes (compared to the market volume) influence the market?
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

pdenoeud wrote:
Mon Nov 19, 2018 10:05 pm
Backtesting is not just fitting the strategy to historic data, it is checking if the strategy is profitable.
Yes
pdenoeud wrote:
Mon Nov 19, 2018 10:05 pm
And then tunning the strategy to maximize the profits.
Yes
pdenoeud wrote:
Mon Nov 19, 2018 10:05 pm
You need to back test on a big amount of events in order to make sure positive results are not due to luck.
Yes
pdenoeud wrote:
Mon Nov 19, 2018 10:05 pm
With backtesting, you can validate (unvalidate most of the times) your strategy very quickly.
Very much, yes :)
pdenoeud wrote:
Mon Nov 19, 2018 10:05 pm
When backtesting, if you need to tune many parameters in order to make the strategy profitable, there are good chances that it won't be replicable in the future. The more parameters combinaisons you test, the more chances you have to finally get a profitable combinaisons... which won't be necessarely profitable in the future.
Yes
pdenoeud wrote:
Mon Nov 19, 2018 10:05 pm
This is true if you are a big player, if your stakes are big compared to the market volume.
Yes, cross that hurdle when we get to it ;)

-

My problem is the fact, there's a lot of info available on/around the ladders - I just cannot utilise effectively atm - & this is where the (my) back-testing collapses. Gets heniously complicated to test, as we refine the data from generic, to very specialized.

This is where I think "modelling" could help, it's just not explained very well imo. I'd defend Peter if I understood it :D
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

Does anybody know a tool allowing the creation of my own bot and back testing of the bot, with access to the score (at least for tennis)?
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

ruthlessimon wrote:
Mon Nov 19, 2018 10:21 pm

My problem is the fact, there's a lot of info available on/around the ladders - I just cannot utilise effectively atm - & this is where the (my) back-testing collapses. Gets heniously complicated to test, as we refine the data from generic, to very specialized.
Which tool do you use to back test?
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

jimibt wrote:
Mon Nov 19, 2018 8:11 pm

there is!! - obviously can't and won't elaborate.
Why can't/won't you elaborate? I am VERY interested. Can you please then tell me the tool name with a private message if you can't/don't want to share it publicly?
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

pdenoeud wrote:
Mon Nov 19, 2018 10:32 pm
Which tool do you use to back test?
I use Excel.

I just love the flexibility of being able to test almost everything. But like I said, it's so easy to quickly hit dead ends - because as we start going above 2/3 variables - back-testing becomes inefficient & dangerous (but 2/3 variables just isn't enough to get some really killa solid edges, still too generic imo) i.e. everyone would be automated if it was that easy

Hence why I love the idea, that there is an alternative - I just cannot get my head around it. Although arguably - is why it works. i.e. The fact I don't get it, means others don't get it (i.e. yourself) - so those those who do get it - have a massive edge over us

95% of all my time in a day is spent building "testing sheets", & thinking about ways to validate certain variables - I'm more of a philosopher than a trader these days ;)
User avatar
BetScalper
Posts: 1139
Joined: Sun Jul 02, 2017 10:47 pm

There is already well known software out there for building strategies and back testing them on any market.
puntingprofit
Posts: 33
Joined: Sat Apr 29, 2017 7:09 am

a touch off topic but.... I think "Building Winning Algorithmic Trading Systems" by Kevin Davey is a useful book to understand strategy development and life-cycle.
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

BetScalper wrote:
Tue Nov 20, 2018 8:53 am
There is already well known software out there for building strategies and back testing them on any market.
I wrote a list of software I tested already, but they did not fit my needs.
But that list has been removed from my post, by a forum admin I guess. I did not know this was not tolerated on this forum, I had the impression it goes beyond the scope of BetAngel tool.

My issue is to back test a tennis bot, using the historical Betfair data about odds AND the corresponding score.
I think I know the "well known software" you mention, but it does not allow to use tennis scores within the bot.
User avatar
Euler
Posts: 24701
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

Moderators may remove links to other software due to historical spamming issues and abuse. I didn't see the list so didn't see what the software was. Now and again they may remove what are considered to be legitimate links, but on balance we prefer a well-moderated forum to the sort of abuse it suffered in past.
Post Reply

Return to “Betfair Data”