I started experimenting with the Betfair streaming API, writing some simple NodeJs software to spot certain things in the market.
How would you calculate the Last Traded Amount (LTA)?
My initial algorithm calculates it for a runner using his total traded volume (tv):
(tv Current Message - tv Prev Message) / 2
I set conflation to 20ms, heartbeat to 500ms, subscribe to far less than 50 markets at a time.
However my formula returns some weirdly big amounts, it feels as though Betfair server doesn’t really send me a message every time there’s a change for the runner.
So, how do you guys calculate LTA?
Streaming API - Last Traded Amount
Set conflate to None/0, Betfair will send you data as quick as you can pull it off the socket or they will start conflating it for you regardless. Conflate will be True on the response if this has occurred.
What do you consider to be weird? You have the site to compare by checking out the graphs for a rough idea.
What do you consider to be weird? You have the site to compare by checking out the graphs for a rough idea.
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
I'm a bit confused by conflation. I thought I means to combine data from various sources but in the context of the streaming api, does it mean some sort of buffering or combining/summerising of multiple marketMessages into a single 'catch up' message?
Outside of computing it does mean to put more than one thing together - but the outcome is erroneous or ambiguousShaunWhite wrote: ↑Thu Dec 06, 2018 10:01 pmI'm a bit confused by conflation. I thought I means to combine data from various sources but in the context of the streaming api, does it mean some sort of buffering or combining/summerising of multiple marketMessages into a single 'catch up' message?
- northbound
- Posts: 737
- Joined: Mon Mar 20, 2017 11:22 pm
Thanks Linus, I’ll try it.
Basically this bit of software is the very first one I write that reads a stream. It’s a simple algo that keeps track of the current tv of a runner and also the previous one. If the difference is above a certain threshold, it generates a log entry.
Weird stuff:
- Observing my logs and BetAngel in real time: there were preplay horse racing instances of a £3k LTA on BetAngel (streaming 500ms) while on my software it was £10k plus (streaming 20ms).
- Observing Betfair charts on a greyhound race’s very early trades, I could see a lay spike of £500+ on the Betfair chart. My software (streaming 20ms), instructed to create a log entry when a spike of £200+ occurred, never did it.
There’s an 80% the fault is mine somehow.
Going back to my original question: would you say my way of calculating LTA is correct?
Basically this bit of software is the very first one I write that reads a stream. It’s a simple algo that keeps track of the current tv of a runner and also the previous one. If the difference is above a certain threshold, it generates a log entry.
Weird stuff:
- Observing my logs and BetAngel in real time: there were preplay horse racing instances of a £3k LTA on BetAngel (streaming 500ms) while on my software it was £10k plus (streaming 20ms).
- Observing Betfair charts on a greyhound race’s very early trades, I could see a lay spike of £500+ on the Betfair chart. My software (streaming 20ms), instructed to create a log entry when a spike of £200+ occurred, never did it.
There’s an 80% the fault is mine somehow.
Going back to my original question: would you say my way of calculating LTA is correct?
- northbound
- Posts: 737
- Joined: Mon Mar 20, 2017 11:22 pm
My basic understanding is as follows.ShaunWhite wrote: ↑Thu Dec 06, 2018 10:01 pmI'm a bit confused by conflation. I thought I means to combine data from various sources but in the context of the streaming api, does it mean some sort of buffering or combining/summerising of multiple marketMessages into a single 'catch up' message?
Imagine a runner whose traded volume increases by exactly £100 every one second.
If you set conflate to 1000ms, you will get one message per second, with the vol figure being £100 bigger than the previous message’s.
If you set conflate to 3000ms, you will get one message every 3secs, with the vol figure being £300 bigger than the previous message’s.
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
Thx northbound,
Pretty much what I thought, always worth checking though.
Pretty much what I thought, always worth checking though.
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
On the 2nd observation, the spike on the bf chart could well have been over a longer time frame, you might even have had 4 x £125's 20ms apart.northbound wrote: ↑Thu Dec 06, 2018 10:19 pmThanks Linus, I’ll try it.
Basically this bit of software is the very first one I write that reads a stream. It’s a simple algo that keeps track of the current tv of a runner and also the previous one. If the difference is above a certain threshold, it generates a log entry.
Weird stuff:
- Observing my logs and BetAngel in real time: there were preplay horse racing instances of a £3k LTA on BetAngel (streaming 500ms) while on my software it was £10k plus (streaming 20ms).
- Observing Betfair charts on a greyhound race’s very early trades, I could see a lay spike of £500+ on the Betfair chart. My software (streaming 20ms), instructed to create a log entry when a spike of £200+ occurred, never did it.
There’s an 80% the fault is mine somehow.
Going back to my original question: would you say my way of calculating LTA is correct?
Not sure about the first, if you were able to say exactly where/when it was I might be able to look in my saved stream and shed some light. I'm a bit of a beginner though so 'might' is the operative word.
The logic seems sound to me but have you thought about comparing the tv with trd, as a cross check.
- northbound
- Posts: 737
- Joined: Mon Mar 20, 2017 11:22 pm
Yep, it’s on my list of things to try.ShaunWhite wrote: ↑Fri Dec 07, 2018 1:52 amhave you thought about comparing the tv with trd, as a cross check.
On greyhound races, I’ll probably also try and log every message received, spot a spike on Betfair chart, stop the logger and look into recent messages to see what did Betfair stream actually send.
betfair charts doesnt log every volume spike, they conflate them in chunks. I wouldnt be very practical to draw graphs with 100s of thousands little spikes. Check if your stream volumes updates match betangles or web volumes, if they do, you are good, ignore betfair graphs
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
That's not far off another idea I had....to just try to read the stream as fast as pos and then calculate the change over time with code. Basically creating you own little rolling 20ms baskets and doing your own conflation... somehow. I'm not calculating LTA myself but that's how I'm working out other changes over time and the results seem to be sensible. I don't use the ConflateMs to control the batching, I do it at my end.northbound wrote: ↑Fri Dec 07, 2018 8:01 amOn greyhound races, I’ll probably also try and log every message received, spot a spike on Betfair chart, stop the logger and look into recent messages to see what did Betfair stream actually send.
- northbound
- Posts: 737
- Joined: Mon Mar 20, 2017 11:22 pm
That wouldn’t work for the kind of tool I’m trying to build, which is to get a Telegram alert as soon as a big chunk gets matched in specific markets.ShaunWhite wrote: ↑Fri Dec 07, 2018 2:18 pmread the stream as fast as pos and then calculate the change over time with code.
It also depends which technology you use to read the stream. My choice for now is NodeJs as it’s non-blocking and can send the Telegram alert “in background” without halting the stream reading process.
If you use a normal “blocking” language, most probably you’re right: have one process that reads and saves the stream, while another process makes sense of the data.