Backtesting irregularities

[ruthlessimon]

At the moment, the only thing I can think of that could explain is that there's a marked increase in volatility on average post-feb - so therefore "false breakouts" and stop losses getting triggered more often - though I'm saying this with very little confidence

Is there a way that you can test this? i.e. modify the strategy dependant on month - Jan/Feb with a widened stop etc, changing to a tighter stop into Mar/Apr etc equity curve vs an unchanged strategy through Jan/Feb/Mar.

I make great efforts to build aspects like that in my spreadsheets. It might not be entirely accurate from an automation standpoint, but will highlight biases/knowledge if the equity curves vary (at least that's what I tell myself )

[sa7med]

At the moment, the only thing I can think of that could explain is that there's a marked increase in volatility on average post-feb - so therefore "false breakouts" and stop losses getting triggered more often - though I'm saying this with very little confidence

Is there a way that you can test this? i.e. modify the strategy dependant on month - Jan/Feb with a widened stop etc, changing to a tighter stop into Mar/Apr etc equity curve vs an unchanged strategy through Jan/Feb/Mar.

I make great efforts to build aspects like that in my spreadsheets. It might not be entirely accurate from an automation standpoint, but will highlight biases/knowledge if the equity curves vary (at least that's what I tell myself )

That's a good suggestion. What I have tried is removing any stop loss and just have it greenup at the off - doesn't seem to make a difference but requires further testing. I must say its easy to get mind f$#%#d with all this. I think I need to leap into some stat/data science books for a while. I find all this a bit over my head.

[wearthefoxhat]

I've been testing some strategies as of late with not that much success. I did start to notice however that many strategies that I try do well in Jan and Feb, but then take a dip in March and April. The same thing's been happening to different strategies that I'm testing. Im struggling to figure out why this is the case. I can understand this occurring for one or two similar strategies, but why would this be affecting different strategies that are tested under differing circumstances? I've tried to see if this is due to different race types but that doesnt seem to be the case. To add to the confusion, a bot that I have been running throughout all those months appears to be doing roughly the same. Has anyone experienced similar or can shed some light on this? Would love to hear your thoughts.

All weather racing markets seem to a different animal for trading purposes. Jan/Feb is where they are scheduled to help cover the abandoned jump races meetings that normally occur. Maybe your strategy performs well at those tracks. (I reckon you may be surprised)

March/April sees the bigger and better class racing festivals, ie Cheltenham/Aintree, and the start of the Turf Flat.

[sa7med]

I've been testing some strategies as of late with not that much success. I did start to notice however that many strategies that I try do well in Jan and Feb, but then take a dip in March and April. The same thing's been happening to different strategies that I'm testing. Im struggling to figure out why this is the case. I can understand this occurring for one or two similar strategies, but why would this be affecting different strategies that are tested under differing circumstances? I've tried to see if this is due to different race types but that doesnt seem to be the case. To add to the confusion, a bot that I have been running throughout all those months appears to be doing roughly the same. Has anyone experienced similar or can shed some light on this? Would love to hear your thoughts.

All weather racing markets seem to a different animal for trading purposes. Jan/Feb is where they are scheduled to help cover the abandoned jump races meetings that normally occur. Maybe your strategy performs well at those tracks. (I reckon you may be surprised)

March/April sees the bigger and better class racing festivals, ie Cheltenham/Aintree, and the start of the Turf Flat.

Thanks, will definitely have a look at that. How do I identify these types of races?

[wearthefoxhat]

I've been testing some strategies as of late with not that much success. I did start to notice however that many strategies that I try do well in Jan and Feb, but then take a dip in March and April. The same thing's been happening to different strategies that I'm testing. Im struggling to figure out why this is the case. I can understand this occurring for one or two similar strategies, but why would this be affecting different strategies that are tested under differing circumstances? I've tried to see if this is due to different race types but that doesnt seem to be the case. To add to the confusion, a bot that I have been running throughout all those months appears to be doing roughly the same. Has anyone experienced similar or can shed some light on this? Would love to hear your thoughts.

All weather racing markets seem to a different animal for trading purposes. Jan/Feb is where they are scheduled to help cover the abandoned jump races meetings that normally occur. Maybe your strategy performs well at those tracks. (I reckon you may be surprised)

March/April sees the bigger and better class racing festivals, ie Cheltenham/Aintree, and the start of the Turf Flat.

Thanks, will definitely have a look at that. How do I identify these types of races?

Seperate by track;

Lingfield
Southwell
Kempton
Chelmsford
Newcastle
Wolverhampton
Dundalk (IRE)

Have a look at the types of quirks each course has, below will give you a start. Check your own records to see if your results are better at these courses compared to Turf-Flat or National Hunt Jumps.

https://www.allaboutbetting.co.uk/horse ... er-courses

[sa7med]

Seperate by track;

Lingfield
Southwell
Kempton
Chelmsford
Newcastle
Wolverhampton
Dundalk (IRE)

Have a look at the types of quirks each course has, below will give you a start. Check your own records to see if your results are better at these courses compared to Turf-Flat or National Hunt Jumps.

https://www.allaboutbetting.co.uk/horse ... er-courses

Okay, will have a look. Thanks!

[ruthlessimon]

I think I need to leap into some stat/data science books

Here's an idea I'm actively working on atm

I think we need to disconnect the variable (course in this case), after generating the figures - because there's a guarantee of finding a best/worst course for any strategy. But the key is whether it's predictably the best/worst. Let's assume you lost -300ticks at Lingfield over a 3mth period. If we then randomly shuffle that p&l (tick) data (say 1000 times), then plot the results - we can then calculate the chance that the -300 was the result of randomness. i.e. If the majority of the "random data" produce losses of -300, your original -300 doesn't look so special (i.e. random data shouldn't match your data, if the variable is vital)

i.e. a result that would suggest Lingfield is a vital variable

Oringial Lingfield : -300
Lingfield random 1 : 34
Lingfield random 2 : -23
Lingfield random 3 : 12
..
....

Now I'm not a statistician, & this might not be the best way to do it. Hopefully, Peter will come in a correct/improve on the above idea

[ShaunWhite]

Bang, top question after top question Si.

These are stats questions that go beyond sport. I sometimes wonder if we'd get more answers on maths forums, but I probably wouldn't understand them.

Peter, when you've finished answering Simon ....did I once hear you say that at times you've employed statisticians to prove/disprove your work? Where from and at what cost/value? If you needed their help then I certainly do!

[ShaunWhite]

I'm often reminded here of a character on the Fast Show called Patrick Nice, one sketch in particular ...

" ...and I looked at the formula again, and noticed that I'd put a decimal point in the wrong place. So I corrected it and discovered that I really had found a cure for cancer... which was nice."

[ruthlessimon]

r.e. my previous post

So.. I plugged in a basic "trend" strategy, which would provide a back/lay signal, then trade it, exit @ post.

Windsor came back as +42ticks on the real data. Now I've only done a couple of sorts, but the +42 sticks out as something that might be significant. (assuming I've set this experiment up right!)

1. REAL: +42ticks
2. RAND1: -15ticks
3. RAND2: +15ticks
4. RAND3: -9ticks

Now I struggle with the "understanding why" bit. Is this inherent to Windsor (i.e. Windsor just has good trending drifts)? Or is this being caused by an extraneous variable? Did those drifts have high volume? Was MMWAP much higher than my entry (therefore it wasn't a trend)? Etc

[Korattt]

guys, is this excercise an effort to find an angle via automation, manual or both?

[ShaunWhite]

guys, is this excercise an effort to find an angle via automation, manual or both?

Any angle Korattt, we're not fussy.

Simon I've a theory about Windsor but I'm in no position to test it. It's my hunch/theory based purely on human behavior that momentum is more likely to occur where the proportion of affluent but largely ill-informed attendees is higher. There might be an element of 'I'll just do what you're doing', I certain used to hear and see it when I used to go to Windsor, Kempton and Sandown but less so at low-rent Lingfield. Does the phenomenon you've seen also occur on other London courses, when it's nice weather, perhaps on a Friday or just after payday, especially when there's better quality elsewhere to keep the smart money busy?

Problem is that most of those factors aren't described in the usual data, but I 'feel' I've seen it while grinding through my 10k.

Markets are afterall almost as much about human behavior as they are anything else. I'd be suprised if regional variations and demographics weren't a factor at some level. There may even be a correlation with net disposable income, but the data doesn't go back far enough for people to have had any!

[ShaunWhite]

.... We've said it before the problem with data is that it's a very narrow set of information and completely lacking in context. It only takes 4 or 5 favourites at one course who run around with an out of control jockey while going down to skew the figures. 2nd favs larking about elsewhere would have the opposite effect.

[ruthlessimon]

momentum is more likely to occur where the proportion of affluent but largely ill-informed attendees is higher.

Which reminded me of this video : https://www.youtube.com/watch?v=e6R4HCwP6nc

.. buuutttt how do we combat this big bad bias (https://en.wikipedia.org/wiki/Look-elsewhere_effect) - i.e. confirmation bias

Affluent ill-informed, yeah Surrey would be a good candidate, but there sits Lingfield - which has a gigantic extraneous variable attached. How would we weight those two aspects into predicting momentum?

That last bit is aimed more at Peter

[sa7med]

Seperate by track;

Lingfield
Southwell
Kempton
Chelmsford
Newcastle
Wolverhampton
Dundalk (IRE)

Have a look at the types of quirks each course has, below will give you a start. Check your own records to see if your results are better at these courses compared to Turf-Flat or National Hunt Jumps.

https://www.allaboutbetting.co.uk/horse ... er-courses

Okay, will have a look. Thanks!

Haven't been very in depth and have only checked one strategy but wanted to update. It does seem that it performs significantly better at these AW tracks, especially Kempton. It was right there in front of my but I guess I got too lost in the numbers and forgot to look at some letters as well I don't have a full picture of why this is the case yet, but I feel better that I at least have some sort of grip on it, tenuous though it may be. Thanks to wearthefoxhat and others for pointing me in the right direction ( i think ). Though its a check on only one season, the findings are based on about 7500 bets... not sure how significant that is. I've lost all confidence in big sample sizes since RS posted the million coinflip PNL Ill check on other strategies when I have time and report back.