Hello, During peak hours, 100 seconds would be ideal. However, at 2am (EST) when the load is much less, the wait time can be as little as 10 seconds. Having to wait 90 seconds extra per request is quite an excessive amount of time to wait between requests when 10 is sufficient in offhours. I'm currently processing a batch of 40,000 requests, which at 100 seconds per request would take about 47 days. Since opening this ticket, I've gotten a little familiar with what are the peak hours to the Google API, and it seems that 20 seconds during offpeak and 40 seconds during onpeak (although this has crept up to 50 now) is sufficient to run the requests uninterrupted.
The only issue I have with the exponential backoff policy is that it doesn't account for peak and off-peak hours. In my example, it would have increased wait times to 100 seconds during onpeak hours but would never come down during offpeak hours. While it would run, the runtime would be 47 days whereas with my guesstimated wait times, I can accomplish the same results with less than half the runtime. Ideally, getting a next wait time on each request would be the most efficient way of cutting runtime while not making too many requests. I'm not sure if the above is an acceptable user case, but it does seem logical to cutting the runtime of large requests. Thank you. On Tuesday, October 4, 2016 at 4:35:52 AM UTC-4, Thanet Knack Praneenararat (AdWords API Team) wrote: > Hello Charles, > > Could you please clarify why 100 seconds are too excessive? > I would like to use this information for discussion with engineering team > as well. > > Currently, the exponential backoff policy is encouraged to be used in many > places of AdWords API, so 100 seconds is what we can expect in many > situations. > But if you have any use cases where that waiting time is too long, then I > can pass that information to the engineering team. > > Thanks in advance. > > Best, > Thanet, AdWords API Team > > On Tuesday, October 4, 2016 at 4:07:27 AM UTC+9, Charles Michael wrote: >> >> Hi, >> >> I remember when you first suggested the pseudocode for this but I never >> implemented it because I couldn't come up with a way to come back down in >> value over time. As I recall, you would double the wait time on every Retry >> request until the requests go through. If I ran this during peak hours, I >> would get a sleep time of around 100 seconds, but this is a very excessive >> wait time when it's not peak hours. I suppose you could cut the values in >> half until a retry comes back up, but this isn't really a reliable method >> in my opinion. It's a temporary workaround but we'd have excessive wait >> times in my opinion so I opted to just use a higher constant value >> (originally 40, I've raised it to 50 to compensate further). >> >> Thanks >> >> >> >> On Monday, October 3, 2016 at 1:09:00 PM UTC-4, Thanet Knack >> Praneenararat (AdWords API Team) wrote: >>> >>> Hello Charles, >>> >>> Have you considered to use the exponential backoff policy as I mentioned >>> in the previous reply? >>> The exponential backoff policy shouldn't be difficult to implement, as >>> illustrated in the AddCompleteCampaignUsingBatchJob >>> <https://github.com/googleads/googleads-java-lib/blob/master/examples/adwords_axis/src/main/java/adwords/axis/v201607/campaignmanagement/AddCompleteCampaignsUsingBatchJob.java#L166> >>> >>> example. >>> I'm confirming with engineering but we haven't reached the conclusion >>> yet. >>> >>> Best, >>> Thanet, AdWords API Team >>> >>> On Tuesday, October 4, 2016 at 1:42:41 AM UTC+9, Charles Michael wrote: >>>> >>>> Sorry for bumping this again, but the issue has gotten progressively >>>> worse. Starting about an hour ago (noon EST), with wait times of 40 >>>> seconds, I'm failing with the same retry after 30 second errors. Was there >>>> any feedback from the engineering team as its very difficult to code >>>> something when the response time (30 seconds) is not valid. >>>> >>>> Thank you. >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Monday, September 26, 2016 at 5:55:52 AM UTC-4, Thanet Knack >>>> Praneenararat (AdWords API Team) wrote: >>>>> >>>>> Hello Charles, >>>>> >>>>> Thanks for your valuable feedback. >>>>> I'll pass this to engineering team as well. >>>>> >>>>> I'll keep this post updated when I have any more news. >>>>> >>>>> Cheers, >>>>> Thanet, AdWords API Team >>>>> >>>>> On Saturday, September 24, 2016 at 3:40:19 AM UTC+9, Charles Michael >>>>> wrote: >>>>>> >>>>>> The only issue with this strategy though is you could potentially be >>>>>> pausing for excessively long periods of time. During peak hours I would >>>>>> be >>>>>> pausing upwards of 60 seconds, but there isn't an efficient way for the >>>>>> pause time to come back down, especially when its no longer peak hours. >>>>>> >>>>>> If I may make a suggestion, it would be very beneficial to include a >>>>>> PauseTime so that developers know what the minimum pause time would be. >>>>>> Currently we're pretty much just guessing and if we hit the >>>>>> RATE_EXCEEDED >>>>>> response, we have to wait an unknown amount of time before making the >>>>>> same >>>>>> request over (since 30 seconds isn't guaranteed to work). If there's an >>>>>> accurate response number in the RATE_EXCEEDED response, that's great, >>>>>> but I >>>>>> think it would even be better to include in successful responses the >>>>>> minimum pause time for that particular account so that we can >>>>>> programmatically pause that amount of time and simply never hit the >>>>>> RATE_EXCEEDED page. >>>>>> >>>>>> For now I've implemented a flat 40 second pause time which seems to >>>>>> be sufficient, although to run my current keyword database, it will take >>>>>> about 3-4 weeks. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Thursday, September 22, 2016 at 10:40:15 PM UTC-4, Thanet Knack >>>>>> Praneenararat (AdWords API Team) wrote: >>>>>>> >>>>>>> Hello Charles, >>>>>>> >>>>>>> I'm still waiting for the final answer of this. >>>>>>> A solution for now is to use exponential backoff policy, for >>>>>>> example, if you first pause 5 seconds before the first retry, you could >>>>>>> pause 10 seconds after the second and 20 seconds after the third retry, >>>>>>> instead of following the *retryAfterSeconds* value. >>>>>>> We are discussing about the use of *retryAfterSeconds* but don't >>>>>>> reach the conclusion yet. >>>>>>> >>>>>>> I'll update this thread when getting more information. >>>>>>> >>>>>>> Best, >>>>>>> Thanet, AdWords API Team >>>>>>> >>>>>>> >>>>>>> On Friday, September 23, 2016 at 6:09:06 AM UTC+9, Charles Michael >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi, was there any update to this? Since originally posting this, >>>>>>>> I've narrowed the issue to only occurring between 12pm - 5pm EST, >>>>>>>> outside >>>>>>>> of this range there are no issues with taking more than 30 seconds >>>>>>>> between >>>>>>>> requests. >>>>>>>> >>>>>>> -- -- =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~ Also find us on our blog and Google+: https://googleadsdeveloper.blogspot.com/ https://plus.google.com/+GoogleAdsDevelopers/posts =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~ You received this message because you are subscribed to the Google Groups "AdWords API Forum" group. To post to this group, send email to adwords-api@googlegroups.com To unsubscribe from this group, send email to adwords-api+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/adwords-api?hl=en --- You received this message because you are subscribed to the Google Groups "AdWords API Forum" group. To unsubscribe from this group and stop receiving emails from it, send an email to adwords-api+unsubscr...@googlegroups.com. Visit this group at https://groups.google.com/group/adwords-api. To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/e80ec0d2-e377-4bbf-907b-a6b4dcee6e06%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
