Any way to backtest an automated strategy?

Post Reply
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

pdenoeud wrote:
Mon Nov 19, 2018 10:32 pm
Which tool do you use to back test?
I use Excel.

I just love the flexibility of being able to test almost everything. But like I said, it's so easy to quickly hit dead ends - because as we start going above 2/3 variables - back-testing becomes inefficient & dangerous (but 2/3 variables just isn't enough to get some really killa solid edges, still too generic imo) i.e. everyone would be automated if it was that easy

Hence why I love the idea, that there is an alternative - I just cannot get my head around it. Although arguably - is why it works. i.e. The fact I don't get it, means others don't get it (i.e. yourself) - so those those who do get it - have a massive edge over us

95% of all my time in a day is spent building "testing sheets", & thinking about ways to validate certain variables - I'm more of a philosopher than a trader these days ;)
User avatar
BetScalper
Posts: 1139
Joined: Sun Jul 02, 2017 10:47 pm

There is already well known software out there for building strategies and back testing them on any market.
puntingprofit
Posts: 33
Joined: Sat Apr 29, 2017 7:09 am

a touch off topic but.... I think "Building Winning Algorithmic Trading Systems" by Kevin Davey is a useful book to understand strategy development and life-cycle.
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

BetScalper wrote:
Tue Nov 20, 2018 8:53 am
There is already well known software out there for building strategies and back testing them on any market.
I wrote a list of software I tested already, but they did not fit my needs.
But that list has been removed from my post, by a forum admin I guess. I did not know this was not tolerated on this forum, I had the impression it goes beyond the scope of BetAngel tool.

My issue is to back test a tennis bot, using the historical Betfair data about odds AND the corresponding score.
I think I know the "well known software" you mention, but it does not allow to use tennis scores within the bot.
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

Moderators may remove links to other software due to historical spamming issues and abuse. I didn't see the list so didn't see what the software was. Now and again they may remove what are considered to be legitimate links, but on balance we prefer a well-moderated forum to the sort of abuse it suffered in past.
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

pdenoeud wrote:
Tue Nov 20, 2018 9:23 am
My issue is to back test a tennis bot, using the historical Betfair data about odds AND the corresponding score.
I think I know the "well known software" you mention, but it does not allow to use tennis scores within the bot.
This more or less highlights the issue I was speaking about. If you backtest against scores you will find the market is pretty much perfect in terms of pricing and scores. It's not what has happened that's going to determine your profitability, it's predicting what will happen.

Tennis Trader perfectly models Tennis scores and odds, so playing with that will allow you to plot all possible paths and your goal as a trader is to anticipate price action rather than follow it.
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

If you take price action at the centre of the book for example, you need to know the fill rate. No amount of backtesting will tell you that, so you need to actually put money in the market to test it under different scenarios.

The only reason I'm saying this guys, is that I've wasted soooooooooo much time coming up with really neat strategies and ideas only to find they fail at the implementation stage. Some of my best strategies were formed by putting something into the market and realising I'd spotted the reason for failure and going on to exploit the reasons for that in a positive way.

You shouldn't underestimate the amount of time you will save by moving to implementation quickly and testing it live. You may even find that your presence in the market may influence the outcome.
xitian
Posts: 457
Joined: Fri Jul 08, 2011 2:08 pm

Yeah, you’re asking in the wrong place I’m afraid.

This is indeed a forum for BetAngel (or completely unrelated stuff like investing). So any methodologies different to those that BetAngel supports or recommends probably won’t be received well.

Unfortunately there aren’t many other well documented and active communities which are truly impartial. With reference to backtesting and systematic strategies, all I can say is that it’s a perfectly valid route. You can earn as much as 7 figures a year if you’re good, and not have to click a button on a day if you don’t feel like it. Because Peter hasn’t succeeded in this area it doesn’t mean that you can’t or shouldn’t look into it. However to be really flexible I would recommend learning to program, and for many people that’s probably a big barrier to entry (but also a benefit).

There might be other software out there that does backtesting, but not sure how flexible they are. In the end it’s best to have full control and write stuff yourself. Betfair provide good documentation and sample applications, and I seem to remember LinusP has released some Python code for accessing streaming data before.

Entry costs if you’re going to program yourself are going to be higher though. You’d need to stump up for an api key (£300) then either spend ages collecting data, or paying for it and spending a few hundred pounds. You’d also want to hire a server to run stuff on, but that’s further down the line.

If you want to discuss anything more specific, let me know. Perhaps I can open a reddit topic where people can talk about platform independent stuff.
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

xitian wrote:
Tue Nov 20, 2018 12:27 pm
So any methodologies different to those that BetAngel supports or recommends probably won’t be received well.
That's not true or fair, that's more or less saying that I'm not allowed to have an opinion. The only reason I have opinions such as one of this thread is because of my experience in the market. It's nothing to do with promoting a viewpoint. I actually stay out of a lot of active discussions to avoid tainting them.

We are happy for people to discuss a wide range of topics but have suffered from the forum being abused in the past, so try to toe that line between active discussion and not letting people directly or indirectly use the forum for the wrong reasons. Given the growth in the forum and the type of people on here, I'd say we have that about right.

We don't have any objections to people discussing the pro and cons of backtesting here, I can't see why we wouldn't. It's good to have differing views and to encourage debate. Removing a backlink to somebody that previously abused the forum doesn't change that.

I've used back testing in the past, but prefer to model a market and apply ideas directly to the market. I'm just not a believer that backtesting on it's own is good enough to get confidence in a process.
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

xitian wrote:
Tue Nov 20, 2018 12:27 pm
Yeah, you’re asking in the wrong place I’m afraid.

This is indeed a forum for BetAngel (or completely unrelated stuff like investing). So any methodologies different to those that BetAngel supports or recommends probably won’t be received well.

Unfortunately there aren’t many other well documented and active communities which are truly impartial. With reference to backtesting and systematic strategies, all I can say is that it’s a perfectly valid route. You can earn as much as 7 figures a year if you’re good, and not have to click a button on a day if you don’t feel like it. Because Peter hasn’t succeeded in this area it doesn’t mean that you can’t or shouldn’t look into it. However to be really flexible I would recommend learning to program, and for many people that’s probably a big barrier to entry (but also a benefit).

There might be other software out there that does backtesting, but not sure how flexible they are. In the end it’s best to have full control and write stuff yourself. Betfair provide good documentation and sample applications, and I seem to remember LinusP has released some Python code for accessing streaming data before.

Entry costs if you’re going to program yourself are going to be higher though. You’d need to stump up for an api key (£300) then either spend ages collecting data, or paying for it and spending a few hundred pounds. You’d also want to hire a server to run stuff on, but that’s further down the line.

If you want to discuss anything more specific, let me know. Perhaps I can open a reddit topic where people can talk about platform independent stuff.
I am an experienced IT engineer, I already have a BetFait API key, I have the framework to write bots (in Java language). I could write bots in other languages actually, learning new langages would not tak eme very long.
BUT I have no way to back test my bots. Writing a back testing framework, using the Betfair data (which is costly), would take very long, it is not an easy task.
And Betfair data does not include scores. I can't see how I can easily link the evolution of odds in time with the evolution of score. Football is quite easy, but tennis is not, and I am more interested in tennis. Unless I deduce the tennis score from the odds...

That is a lot of work on a path which might lead nowhere (according to Euler), so I am hesitating:-)
User avatar
pdenoeud
Posts: 71
Joined: Tue Jun 26, 2018 10:37 am

Euler wrote:
Tue Nov 20, 2018 11:16 am
You may even find that your presence in the market may influence the outcome.
Euler, this is true only if your stakes are big compared to available/traded volume in the market, right?
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

The way to look at Tennis is that a player can win a point on serve. So say that's a 50% chance, then to win 40-0 you need to calculate 0.5^4

If you look at all available test data and find that the average chance of winning on serve is 50% then backtest that to all available data and odds then you will find it matches the model perfectly. So the defining characteristic actually becomes the chance of winning a point on serve in that particular match and how it is changing and nothing to do with the historic chance of winning a point on serve. The odds will move to reflect the number of paths left to finish the match. If you model it, you can plot all available paths and work to anticipate them.

I suggest the best way to test this is to model just one game of tennis then backtest to check. Tennis is specifically related to the underlying score and any change in momentum or physical condition of the player.
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

pdenoeud wrote:
Tue Nov 20, 2018 1:01 pm
Euler wrote:
Tue Nov 20, 2018 11:16 am
You may even find that your presence in the market may influence the outcome.
Euler, this is true only if your stakes are big compared to available/traded volume in the market, right?
It would be proportional, yes.
xitian
Posts: 457
Joined: Fri Jul 08, 2011 2:08 pm

Euler wrote:
Tue Nov 20, 2018 12:48 pm
xitian wrote:
Tue Nov 20, 2018 12:27 pm
So any methodologies different to those that BetAngel supports or recommends probably won’t be received well.
That's not true or fair, that's more or less saying that I'm not allowed to have an opinion. The only reason I have opinions such as one of this thread is because of my experience in the market. It's nothing to do with promoting a viewpoint. I actually stay out of a lot of active discussions to avoid tainting them.
It may not be to do with promoting a viewpoint but you invariably do because of who you are as the BetAngel owner, feature designer, and “most profitable horse trader ever”. Perhaps you should create an anonymous profile and post from that if you truly don’t want to influence people.
pdenoeud wrote:
Tue Nov 20, 2018 12:52 pm
That is a lot of work on a path which might lead nowhere (according to Euler), so I am hesitating:-)
Exactly my point. Don’t let Peter’s weight sway you.

Peter’s wrong here, backtesting can be a hugely powerful tool if used correctly.

You already have the right skills, and that’s really the biggest time sink (learning the skills). Writing a backtesting simulation system might take a couple months depending how much time you spend on it? Imagine how many years you could use it in future though, and how many ideas you can trial. Just make sure you keep some out of sample data, and make sure you know what assumptions you’re making when you backtest/simulate.
xitian
Posts: 457
Joined: Fri Jul 08, 2011 2:08 pm

Euler wrote:
Tue Nov 20, 2018 1:06 pm
The way to look at Tennis is that a player can win a point on serve. So say that's a 50% chance, then to win 40-0 you need to calculate 0.5^4

If you look at all available test data and find that the average chance of winning on serve is 50% then backtest that to all available data and odds then you will find it matches the model perfectly. So the defining characteristic actually becomes the chance of winning a point on serve in that particular match and how it is changing and nothing to do with the historic chance of winning a point on serve. The odds will move to reflect the number of paths left to finish the match. If you model it, you can plot all available paths and work to anticipate them.

I suggest the best way to test this is to model just one game of tennis then backtest to check. Tennis is specifically related to the underlying score and any change in momentum or physical condition of the player.
So wouldn’t it make sense to have a backtesting system where you can test your model of calculating or predicting a server’s chance of winning a point?

Then I could see how accurate I had been over 1000 matches. Or tweak my model to fit 500 matches and test 500 out of sample matches?
Post Reply

Return to “Betfair Data”