Botty Challenges

Advanced automation available in Guardian - Chat with others and share files here.
Post Reply
User avatar
Dallas
Posts: 22673
Joined: Sun Aug 09, 2015 10:57 pm
Location: Working From Home

mcgoo wrote:
Tue Nov 13, 2018 10:54 am
Removing the automation file/rules from the race immediately shows results in terms of the clock resumes normal countdown..re-applying the rules file causes the issue again. I suppose this would make sense in terms of a complex bot causing performance issues except that the VPS doesn't appear to be struggling for resources :? . I have 149 markets added to Guardian with the bot applied to all of them but restrict refresh is on as I said. I don't recall this being an issue with previous versions of BA but could be wrong there.
Depending if you need historical info for your rules you could shorten the number of historical data points captured by BA, by default it stores the last 7200 data points for each selection of each market, this isn't in the main options but if you PM me I'll let you know how to change it
poklius
Posts: 105
Joined: Sun May 17, 2009 11:58 am

I run my own horse racing python bot on 1CPU, 1GB RAM, 18 strategies, quite a lot of complex calculations, moving averages, sorting, searching, loging, saving raw json data to files etc. And even at peak times bot is at 20-30% of resources. 4CPU's for a simple bot is ridiculous..
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

mcgoo wrote:
Tue Nov 13, 2018 11:16 am
jimibt wrote:
Tue Nov 13, 2018 11:07 am
mcgoo - one thing that i pulled my hair out for a while over was the fact that if you have a lot of logging going on, after a while BA slows up considerably!! i think that every time a new log entry is written, it takes the entire previous chunk back up into memory. thus, if you have a large logfile, over many days, then it grinds to a halt. two ways to mitigate this:

1. switch off logging (on signals and stored values) once in production
2. clear the log entry (from the main Guardian screen)

these two steps sorted out what sounds like a similar issue.

Thanks Jimibt. I'll try those.Cheers-I did use a lot of logging to troubleshoot.I'll let you know
PS: VPS is doing updates.. :) ..might be tomorrow but thanks again
That seemed to solve it.Thanks again Jimibt :ugeek:
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

Dallas wrote:
Tue Nov 13, 2018 12:03 pm
mcgoo wrote:
Tue Nov 13, 2018 10:54 am
Removing the automation file/rules from the race immediately shows results in terms of the clock resumes normal countdown..re-applying the rules file causes the issue again. I suppose this would make sense in terms of a complex bot causing performance issues except that the VPS doesn't appear to be struggling for resources :? . I have 149 markets added to Guardian with the bot applied to all of them but restrict refresh is on as I said. I don't recall this being an issue with previous versions of BA but could be wrong there.
Depending if you need historical info for your rules you could shorten the number of historical data points captured by BA, by default it stores the last 7200 data points for each selection of each market, this isn't in the main options but if you PM me I'll let you know how to change it
thanks Dallas. I seem to be ok for now but good to know.Cheers
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

poklius wrote:
Tue Nov 13, 2018 12:27 pm
I run my own horse racing python bot on 1CPU, 1GB RAM, 18 strategies, quite a lot of complex calculations, moving averages, sorting, searching, loging, saving raw json data to files etc. And even at peak times bot is at 20-30% of resources. 4CPU's for a simple bot is ridiculous..
Wish I could code like that :D
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

mcgoo wrote:
Wed Nov 14, 2018 8:41 am
mcgoo wrote:
Tue Nov 13, 2018 11:16 am
jimibt wrote:
Tue Nov 13, 2018 11:07 am
mcgoo - one thing that i pulled my hair out for a while over was the fact that if you have a lot of logging going on, after a while BA slows up considerably!! i think that every time a new log entry is written, it takes the entire previous chunk back up into memory. thus, if you have a large logfile, over many days, then it grinds to a halt. two ways to mitigate this:

1. switch off logging (on signals and stored values) once in production
2. clear the log entry (from the main Guardian screen)

these two steps sorted out what sounds like a similar issue.

Thanks Jimibt. I'll try those.Cheers-I did use a lot of logging to troubleshoot.I'll let you know
PS: VPS is doing updates.. :) ..might be tomorrow but thanks again
That seemed to solve it.Thanks again Jimibt :ugeek:
you cannot imagine my relief when i discovered that little *tip*. i thought it was my machine, i thought it was too many rules etc,etc... simple things
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

jimibt wrote:
Wed Nov 14, 2018 8:50 am
mcgoo wrote:
Wed Nov 14, 2018 8:41 am
mcgoo wrote:
Tue Nov 13, 2018 11:16 am



Thanks Jimibt. I'll try those.Cheers-I did use a lot of logging to troubleshoot.I'll let you know
PS: VPS is doing updates.. :) ..might be tomorrow but thanks again
That seemed to solve it.Thanks again Jimibt :ugeek:
you cannot imagine my relief when i discovered that little *tip*. i thought it was my machine, i thought it was too many rules etc,etc... simple things
Indeed-was making my brain fry :D Thanks mate-relief is the word :ugeek: :mrgreen:
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

mcgoo wrote:
Wed Nov 14, 2018 8:54 am
jimibt wrote:
Wed Nov 14, 2018 8:50 am
mcgoo wrote:
Wed Nov 14, 2018 8:41 am


That seemed to solve it.Thanks again Jimibt :ugeek:
you cannot imagine my relief when i discovered that little *tip*. i thought it was my machine, i thought it was too many rules etc,etc... simple things
Indeed-was making my brain fry :D Thanks mate-relief is the word :ugeek: :mrgreen:
right, will let you get back to your VB ;)
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

jimibt wrote:
Wed Nov 14, 2018 9:42 am
mcgoo wrote:
Wed Nov 14, 2018 8:54 am
jimibt wrote:
Wed Nov 14, 2018 8:50 am


you cannot imagine my relief when i discovered that little *tip*. i thought it was my machine, i thought it was too many rules etc,etc... simple things
Indeed-was making my brain fry :D Thanks mate-relief is the word :ugeek: :mrgreen:
right, will let you get back to your VB ;)
:twisted: :twisted: :twisted: :lol: :lol: :lol:
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

So, I am still muddling along. My horse/dogs bot (it works surprisingly well on both) runs well at the moment-profit :mrgreen: -for now.
I started to notice performance problems again though and this evening noticed how the clock was sticking and bets are being delayed. There is not much in the way of logging going on, so its not that. Processor is at 10% and memory 12% usage. I tried only adding dog markets..and performance improved. I have noticed this before. This perhaps has something to do with synchronous queuing with BF and API calls to 2 different kinds of markets,not sure, but thought I would mention it as it definitely i a problem that perhaps the BA team can look at if someone else notices it too.Different hosting providers and different machine specs-same issue. The low processor and memory usage is interesting, Thats as far as my investigation has gone.Just mentioning it for interest
User avatar
jimibt
Posts: 3641
Joined: Mon Nov 30, 2015 6:42 pm
Location: Narnia

mcgoo wrote:
Fri Dec 21, 2018 10:47 am
So, I am still muddling along. My horse/dogs bot (it works surprisingly well on both) runs well at the moment-profit :mrgreen: -for now.
I started to notice performance problems again though and this evening noticed how the clock was sticking and bets are being delayed. There is not much in the way of logging going on, so its not that. Processor is at 10% and memory 12% usage. I tried only adding dog markets..and performance improved. I have noticed this before. This perhaps has something to do with synchronous queuing with BF and API calls to 2 different kinds of markets,not sure, but thought I would mention it as it definitely i a problem that perhaps the BA team can look at if someone else notices it too.Different hosting providers and different machine specs-same issue. The low processor and memory usage is interesting, Thats as far as my investigation has gone.Just mentioning it for interest
is this in practice mode or live?? if practice mode, hopefully you've been adding a swarm of live bets to keep BF happy. it sounds very similar to an issue i had when my account was suspended. but then again, normally that is accompanied by an email. you can tell if this is the case mind you by checking for erratic price changes IP - this is a sure sign (in my experience).

fingers xx'd it's just a comms glitch. have a great xmas!
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

jimibt wrote:
Fri Dec 21, 2018 10:55 am
mcgoo wrote:
Fri Dec 21, 2018 10:47 am
So, I am still muddling along. My horse/dogs bot (it works surprisingly well on both) runs well at the moment-profit :mrgreen: -for now.
I started to notice performance problems again though and this evening noticed how the clock was sticking and bets are being delayed. There is not much in the way of logging going on, so its not that. Processor is at 10% and memory 12% usage. I tried only adding dog markets..and performance improved. I have noticed this before. This perhaps has something to do with synchronous queuing with BF and API calls to 2 different kinds of markets,not sure, but thought I would mention it as it definitely i a problem that perhaps the BA team can look at if someone else notices it too.Different hosting providers and different machine specs-same issue. The low processor and memory usage is interesting, Thats as far as my investigation has gone.Just mentioning it for interest
is this in practice mode or live?? if practice mode, hopefully you've been adding a swarm of live bets to keep BF happy. it sounds very similar to an issue i had when my account was suspended. but then again, normally that is accompanied by an email. you can tell if this is the case mind you by checking for erratic price changes IP - this is a sure sign (in my experience).

fingers xx'd it's just a comms glitch. have a great xmas!
You too thanks mate.
I only trade live now..thousands of markets over months. I have definitely seen a pattern with both kinds of markets loaded. The low processor and mem % perhaps indicates comms but the ping to the intertron is <1ms. I work with devs who have suggested that it could be queuing related but who knows. Just mentioning the observation and I can work around it with another VPS if necessary.
foxwood
Posts: 390
Joined: Mon Jul 23, 2012 2:54 pm

Problem sounds like memory leaks and/or large lists. Some intensive apps I use foul up deep in .NET memory management - if I restart them every few days everything works fine - leave it running non-stop and eventually machine reboots for no discernible reason (win10 !)

Do you restart BA every day ? Or at least start another instance if the previous one is still running markets from yesterday. Another way might be to split the 149 markets into (say) 3 or 4 groups and run a separate BA instance for each group.
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

foxwood wrote:
Fri Dec 21, 2018 11:27 am
Problem sounds like memory leaks and/or large lists. Some intensive apps I use foul up deep in .NET memory management - if I restart them every few days everything works fine - leave it running non-stop and eventually machine reboots for no discernible reason (win10 !)

Do you restart BA every day ? Or at least start another instance if the previous one is still running markets from yesterday. Another way might be to split the 149 markets into (say) 3 or 4 groups and run a separate BA instance for each group.
Thanks for the reply. Restart BA everyday and even if I restart the machine itself the problem reoccurs.It seems to happen when different market types( dogs + horses) are concurrently loaded. I have run the bot for the last 6 hours on dogs without an issue.This is ok to do but wondered if anyone else has seen this.cheers
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

Still muddling along :) . Question that I think has been asked before (via a suggestion from what I recall) but thought I would check if there is a better way than I am using or an update coming which will handle it:
I am trying to find the best way to record the price an (my) opening trade got matched at. Currently, I use a SV to store the last traded price when my bet is placed-which usually means matched in the context I use it. The problem comes when the market is moving fast and the actual price matched can be 5-10 ticks out (different from the last traded price at the same instant-by another trader/bot)or when BF match you at a better price and that can be 10+ ticks (saw one today 13 ticks off the SV/price) out. Is there a smarter way to do this or will the next update cover it? To my mind bots are far less accurate/efficient without this capability. Have I missed something?

Thanks in advance :ugeek:
Post Reply

Return to “Bet Angel - Automation”