any idea how to scrape betfair's website?

Post Reply
cyxstudio
Posts: 297
Joined: Sun Mar 23, 2014 11:18 pm

I want to scrap betfair odds without using the API because i am not doing high freq trading so i dont mind 5 sec delay and dont wanna the hassle of using an API key to log into betfair.

I just want to integrate them into my excel so it will highlight value.

However i take a close look at the betfair code and i realise they use angular to dynamically generated codes for markets. Hence the conventional XML request method (GET) doesnt work as you wont get the dynamically generated content.

Anyone have any idea or can point me into the right direction?
Last edited by cyxstudio on Tue Apr 30, 2019 1:02 am, edited 2 times in total.
LinusP
Posts: 1871
Joined: Mon Jul 02, 2012 10:45 pm

Open up chrome and developers tab and you can see the network requests being made, its very similar to the API-NG they make publicly available but uses a fixed appKey and session token from a different login endpoint. Tbh getting access is worth £299 of your time.
cyxstudio
Posts: 297
Joined: Sun Mar 23, 2014 11:18 pm

thx

after some grinding , trial and error and searching through wall of codes i finally got it done.

First send this xml request to get the marketid of all major football events

Code: Select all

strLogin = "https://www.betfair.com/www/sports/navigation/facet/v1/search?_ak=nzIFcwyWhrlwYMrh&alt=json"

Set xmlhttp = CreateObject("MSXML2.ServerXMLHTTP.6.0")
xmlhttp.Open "POST", strLogin
xmlhttp.setRequestHeader "Content-Type", "application/json"
xmlhttp.setRequestHeader "Referer", "https://www.betfair.com/exchange/plus/football"
   body = " { ""from"": ""Epic"",""select"": []}"
formdata = "  {""filter"":{""marketBettingTypes"":[""ASIAN_HANDICAP_SINGLE_LINE"",""ASIAN_HANDICAP_DOUBLE_LINE"",""ODDS""],""productTypes"":[""EXCHANGE""],""marketTypeCodes"":[""MATCH_ODDS""],""selectBy"":""RANK"",""contentGroup"":{""language"":""en"",""regionCode"":""UK""},""turnInPlayEnabled"":true,""maxResults"":0,""eventTypeIds"":[1]},""facets"":[{""type"":""EVENT_TYPE"",""skipValues"":0,""maxValues"":10,""next"":{""type"":""COMPETITION"",""skipValues"":0,""maxValues"":5,""next"":{""type"":""EVENT"",""skipValues"":0,""maxValues"":10,""next"":{""type"":""MARKET"",""maxValues"":1}}}}],""currencyCode"":""GBP"",""locale"":""en_GB""}  "
xmlhttp.send (formdata)

'Save the response to a string
strReturn = xmlhttp.responseText

Dim doc As MSHTML.HTMLDocument

Set doc = New MSHTML.HTMLDocument
Set TableHTML = New MSHTML.HTMLDocument
doc.body.innerHTML = strReturn



Then you can json parse (i simply use string manipulation technique) and pick out all the market ID and send it usng a XMLrequest with it. It returns a list of football events with match odds.

Code: Select all

    
strLogin = "https://www.betfair.com/www/sports/exchange/readonly/v1/bymarket?_ak=nzIFcwyWhrlwYMrh&alt=json&currencyCode=GBP&locale=en_GB&marketIds=1.157733812&rollupLimit=10&rollupModel=STAKE&types=EVENT,RUNNER_DESCRIPTION, RUNNER_EXCHANGE_PRICES_BEST"
Set xmlhttp = CreateObject("MSXML2.ServerXMLHTTP.6.0")
xmlhttp.Open "GET", strLogin
xmlhttp.setRequestHeader "Content-Type", "application/json"
xmlhttp.setRequestHeader "Content-Length", "1242"
xmlhttp.setRequestHeader "Accept", "application/json"
xmlhttp.setRequestHeader "Host", "www.betfair.com"
'xmlhttp.send ("_ak=nzIFcwyWhrlwYMrh&alt=json&currencyCode=GBP&locale=en_GB&marketIds=1.157733639,1.157733812,1.157770859,1.157771032,1.158139961,1.157916210,1.158241164,1.158241748,1.158238537,1.158239157,1.158241289,1.158238785,1.158239744,1.158241431,1.158220932,1.158240949,1.158238041,1.158238165&rollupLimit=10&rollupModel=STAKE&types=MARKET_STATE,MARKET_RATES,MARKET_DESCRIPTION,EVENT,RUNNER_DESCRIPTION,RUNNER_STATE,RUNNER_EXCHANGE_PRICES_BEST,RUNNER_METADATA,MARKET_LICENCE")
xmlhttp.send

'Save the response to a string
strReturn = xmlhttp.responseText
runnerjp
Posts: 21
Joined: Tue Jan 28, 2014 11:26 pm

Did you ever get this working in an excel file that you could share?

I want to pull the football markets in the am and then filter down on specific odds ranges.

Example: Get all Home teams where odds are under evens.
User avatar
Dallas
Posts: 22673
Joined: Sun Aug 09, 2015 10:57 pm
Location: Working From Home

runnerjp wrote:
Wed Jul 17, 2019 3:01 pm

Example: Get all Home teams where odds are under evens.
You could do that with a Filter in Bet Angel and have them all in Guardian in a matter of seconds
viewtopic.php?f=61&t=18794

Then export to excel if you want them there
Post Reply

Return to “Technology”