How do you know you have a profitable system?

[Sovereign]

marksmeets302 - I double checked my average pnl for that original set, and it was actually 0.31 - I dunno how I made that error, apologies (EDIT: I actually determined it was 0.28 up to that set of 210 races, but the full pnl up until the end of that day was 0.31), It certainly seemed quite a jump, so my reckoning is that I just don't have enough data to confirm either way, the fluctuations on a day to day basis are too large. It's now at 0.35 as today progresses.

xitian - I think what you're saying is right - I definitely don't have enough data, so I'm going to wait out the month and see where I end up, taking snapshots of the average pnl figures, my stdev etc as the month progresses to see if the fluctuations start to diminish. A picture, perhaps, paints a thousand words:




As for percentage of races meeting the criteria, roughly 75%.

[marksmeets302]

It's a bit surprising the t-test says you have a profitable strategy; from the graph alone I would also say you need more data. But hey, congrats! I'd be a bit conservative at the moment and not take out a second mortgage to leverage your system, but things are looking good. Get some more data, get acquainted with the system and try to improve - you could be on to something!

[foxwood]

For instance, if your average pnl is 2, and the standard deviation 10, then after 9000 races with 95% certainty you would have made between
(2 pounds)*9000 - 2*sqrt(9000)*(10 pounds)
and
(2 pounds)*9000 + 2*sqrt(9000)*(10 pounds).

Been looking closely at this to evaluate a promising but volatile strategy and it struck me that this would only work for a "normal distribution" of the pnl.

For a profitable strategy however, I think the pnl distribution will always (?) be skewed positively. This would apply to trading and even more so for level stakes betting ie there would be a large number of small losses with smaller numbers of varying profitable amounts.

So, I wonder if it is valid to use the standard deviation / average eg this article talks about using logs or regression to eliminate skewness - kinda fries my brain trying to work that lot out ! https://www.ma.utexas.edu/users/mks/sta ... tions.html

PeterLe's monte carlo analysis looks interesting - might try and replicate that and see if it gives anything like a similar answer which might help.

[marksmeets302]

foxwood, I usually remove the outliers which are a result of problems with betfair, internet connection etc. After that, I'm often left with something that is normally distributed. Of course it will have a positive or negative mean depending on the profitability of the strategy, but a skew is rare in my opinion. Some years ago I spend some time on how to check if a distribution is normal. It seems you can use something called the Jarque-Bera test for this. If you google for it you'll find tools that perform this test for you.

[gazuty]

Doh, this excellent thread is racing along while I'm on holidays.

[foxwood]

foxwood, I usually remove the outliers which are a result of problems with betfair, internet connection etc. After that, I'm often left with something that is normally distributed. Of course it will have a positive or negative mean depending on the profitability of the strategy, but a skew is rare in my opinion. Some years ago I spend some time on how to check if a distribution is normal. It seems you can use something called the Jarque-Bera test for this. If you google for it you'll find tools that perform this test for you.

Many thanks for that - like most of these formulae it looks horrendous to implement but seems there are some Excel and package solutions that will take the grunt out of it.

Turns out the strategy I am looking at is bimodal (ie two humps) - the losses and profits each form roughly normal distributions either side of the zero line.

Could be because my graph Y axis is abs(total pnl) at X axis points of 'PL £ per race'. Might try graphing the numbers in different ways to see if can view it as a normal distribution before the headaches set in !

[brunty122]

Hi,

I have been reading this post and applying some of the methods to my results. I am struggling however when it comes to the Monte Carlo simulation. I have data over 243 races which I know is quite small at the minute but wanted to give it a go anyway.

I know my average profit over the 243 races and the standard deviation.
Ive also worked out the +/- standard deviation to 3500 races.

Could anybody point me in the right direction as to how I would do the Monte Carlo simulation in excel?

Thanks
brunty122

[brunty122]

Anyone?

I have data for 506 races now but I am still unsure as to how I would go about doing the Monte Carlo simulation

Thanks
Brunty122

[marksmeets302]

This: https://excel.tv/monte-carlo-simulation ... -download/ seems to have a nice explanation

[brunty122]

This: https://excel.tv/monte-carlo-simulation ... -download/ seems to have a nice explanation

Cheers! Easy video to follow

[brunty122]

Hi

The Below graph is the P/L using past data for 452 days for one of my strategies. These are the results from paper betting not actually using real money in the market, however I dont think the results would differ to much as it is more of a value betting strategy using flat stakes which I dont feel will have a problem being matched.

P L Chart.png

It showed an average profit of £17.93 and a Std Dev of 75.71

The bit that is slightly concerning and what I would like some feedback on is the results of the + - Std Dev results.
It showed that over 1000 days the results would be between the number shown below to a 95.4% certanty

Std Dev - =£-24,991.6
Std Dev+ =£60,851.6

The bit that looks scary is obviously the -£24,991 as I could go the best part of 2 years and be negative by quite abit. I was wondering if this is normal for a value betting type strategy. I have had a mess around with the number of days I have been looking at and changed the 1000 days to 10,000 to see what impact it would have on the results. That changes the figures to

Std Dev - = 43,569.9
Std Dev + = 315,030.1


As shown by the graph it is showing an up trend but that -£24,991 suprised me quite abit and I just wanted some feedback. I may have even done the calculations wrong so feel free to tell me!


I have also run a Monte Carlo simulation and its showing that over 50 months on average 90% of months will be in profit

Thanks
Brunty122

[Atho55]

I am also testing strategies taken from 1 years worth of Betfair data and have found that even in practice mode the results are not always representative of the paper results and despite both my Back and Lay being place x @ SP y I have occurances that they do not get matched or get matched at different odds.

I`m not sure if this will help you in your decision making but I have found that by summing the returns by day then plotting this accumulatively you can project your past returns forward as a guide to measure the present ones against. The idea being to see if the areas of losses could be avoided. Only a couple of weeks in but you can get the gist from these. The Blue line is from 2016-2017 and represents actual paper returns projected forward. The actual results from the rules applied in BA are below it.

Hope that helps

Glyn

[marksmeets302]

brunty122, the negative number of -24K is not that surprising at 1000 data points. In that case it would mean you had been very unlucky on an extraordinary number of days. The pnl of a day is the result of several races. If you take a single race result as one data point instead of a full day you will get different means and standard deviations, but 1000 days will now be much more data points, perhaps 30000 or so. You will see that you need much less days to rule out a loss with 95% probability.

The best way to go about this is to actually ask for the individual race profit and losses, betfair can provide those numbers for you. There is a shortcut though: there's a way to scale the results. If you do let's say 30 races per day then you can take your original mean, divide it by 30, and your original standard deviation and divide it by the square root of 30. You probably will not have a fixed number of races per day, and I think the more they fluctuate the less useful this method will be.

There's also a third method: just look at the graph. That's a beautiful curve! I wouldn't worry too much.
Congratulations, very well done.

[brunty122]

Thanks Glyn, will have a look at that next time I have time

If you take a single race result as one data point instead of a full day you will get different means and standard deviations, but 1000 days will now be much more data points, perhaps 30000 or so. You will see that you need much less days to rule out a loss with 95% probability.

I did think about this marksmeets302 but I just had the daily results on hand so used them (Maybe being lazy haha) I can see where you are coming from though and can see my mistake. As a rough estimate I did what you said but for 20 races a day and it produced better results. Cheers for the help!!!