Thanks for the update, I'm currently waiting between 40-50 seconds which is pretty much as you suggested, a multiplier between 1-2X of 30 seconds.
I looked into BatchJobService, and while it seems promising being able to retry requests automatically, it doesn't seem to have support for TargetingIdeaSelector. Looks like I'll need to simply wait it out and let it run the full duration. Thanks for your help though! On Wednesday, October 5, 2016 at 1:50:59 AM UTC-4, Thanet Knack Praneenararat (AdWords API Team) wrote: > > Hi Charles, > > Thanks for more insight. > > I've discussed with engineering team and unfortunately, due to technical > issues, we can't provide the exact waiting time via *retryAfterSeconds*. > So, we suggest that you treat the *retryAfterSeconds* as the lower bound > of your waiting time, > and apply a random multiplier, which is a float number ranging between 1 > and 2 to that value. > The resulting value can be used as your waiting time. > > By the way, if you submit 40,000 requests, I wonder if you use > BatchJobService > <https://developers.google.com/adwords/api/docs/guides/batch-jobs> to do > that? > With BatchJobService, most of the time, you probably don't need to worry > about the waiting time. > > Best, > Thanet, AdWords API Team > > On Tuesday, October 4, 2016 at 10:18:37 PM UTC+9, Charles Michael wrote: >> >> Hello, >> >> During peak hours, 100 seconds would be ideal. However, at 2am (EST) when >> the load is much less, the wait time can be as little as 10 seconds. Having >> to wait 90 seconds extra per request is quite an excessive amount of time >> to wait between requests when 10 is sufficient in offhours. I'm currently >> processing a batch of 40,000 requests, which at 100 seconds per request >> would take about 47 days. Since opening this ticket, I've gotten a little >> familiar with what are the peak hours to the Google API, and it seems that >> 20 seconds during offpeak and 40 seconds during onpeak (although this has >> crept up to 50 now) is sufficient to run the requests uninterrupted. >> >> The only issue I have with the exponential backoff policy is that it >> doesn't account for peak and off-peak hours. In my example, it would have >> increased wait times to 100 seconds during onpeak hours but would never >> come down during offpeak hours. While it would run, the runtime would be 47 >> days whereas with my guesstimated wait times, I can accomplish the same >> results with less than half the runtime. Ideally, getting a next wait time >> on each request would be the most efficient way of cutting runtime while >> not making too many requests. >> >> I'm not sure if the above is an acceptable user case, but it does seem >> logical to cutting the runtime of large requests. >> >> Thank you. >> >> >> On Tuesday, October 4, 2016 at 4:35:52 AM UTC-4, Thanet Knack >> Praneenararat (AdWords API Team) wrote: >> >>> Hello Charles, >>> >>> Could you please clarify why 100 seconds are too excessive? >>> I would like to use this information for discussion with engineering >>> team as well. >>> >>> Currently, the exponential backoff policy is encouraged to be used in >>> many places of AdWords API, so 100 seconds is what we can expect in many >>> situations. >>> But if you have any use cases where that waiting time is too long, then >>> I can pass that information to the engineering team. >>> >>> Thanks in advance. >>> >>> Best, >>> Thanet, AdWords API Team >>> >>> On Tuesday, October 4, 2016 at 4:07:27 AM UTC+9, Charles Michael wrote: >>>> >>>> Hi, >>>> >>>> I remember when you first suggested the pseudocode for this but I never >>>> implemented it because I couldn't come up with a way to come back down in >>>> value over time. As I recall, you would double the wait time on every >>>> Retry >>>> request until the requests go through. If I ran this during peak hours, I >>>> would get a sleep time of around 100 seconds, but this is a very excessive >>>> wait time when it's not peak hours. I suppose you could cut the values in >>>> half until a retry comes back up, but this isn't really a reliable method >>>> in my opinion. It's a temporary workaround but we'd have excessive wait >>>> times in my opinion so I opted to just use a higher constant value >>>> (originally 40, I've raised it to 50 to compensate further). >>>> >>>> Thanks >>>> >>>> >>>> >>>> On Monday, October 3, 2016 at 1:09:00 PM UTC-4, Thanet Knack >>>> Praneenararat (AdWords API Team) wrote: >>>>> >>>>> Hello Charles, >>>>> >>>>> Have you considered to use the exponential backoff policy as I >>>>> mentioned in the previous reply? >>>>> The exponential backoff policy shouldn't be difficult to implement, as >>>>> illustrated in the AddCompleteCampaignUsingBatchJob >>>>> <https://github.com/googleads/googleads-java-lib/blob/master/examples/adwords_axis/src/main/java/adwords/axis/v201607/campaignmanagement/AddCompleteCampaignsUsingBatchJob.java#L166> >>>>> >>>>> example. >>>>> I'm confirming with engineering but we haven't reached the conclusion >>>>> yet. >>>>> >>>>> Best, >>>>> Thanet, AdWords API Team >>>>> >>>>> On Tuesday, October 4, 2016 at 1:42:41 AM UTC+9, Charles Michael wrote: >>>>>> >>>>>> Sorry for bumping this again, but the issue has gotten progressively >>>>>> worse. Starting about an hour ago (noon EST), with wait times of 40 >>>>>> seconds, I'm failing with the same retry after 30 second errors. Was >>>>>> there >>>>>> any feedback from the engineering team as its very difficult to code >>>>>> something when the response time (30 seconds) is not valid. >>>>>> >>>>>> Thank you. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Monday, September 26, 2016 at 5:55:52 AM UTC-4, Thanet Knack >>>>>> Praneenararat (AdWords API Team) wrote: >>>>>>> >>>>>>> Hello Charles, >>>>>>> >>>>>>> Thanks for your valuable feedback. >>>>>>> I'll pass this to engineering team as well. >>>>>>> >>>>>>> I'll keep this post updated when I have any more news. >>>>>>> >>>>>>> Cheers, >>>>>>> Thanet, AdWords API Team >>>>>>> >>>>>>> On Saturday, September 24, 2016 at 3:40:19 AM UTC+9, Charles Michael >>>>>>> wrote: >>>>>>>> >>>>>>>> The only issue with this strategy though is you could potentially >>>>>>>> be pausing for excessively long periods of time. During peak hours I >>>>>>>> would >>>>>>>> be pausing upwards of 60 seconds, but there isn't an efficient way for >>>>>>>> the >>>>>>>> pause time to come back down, especially when its no longer peak >>>>>>>> hours. >>>>>>>> >>>>>>>> If I may make a suggestion, it would be very beneficial to include >>>>>>>> a PauseTime so that developers know what the minimum pause time would >>>>>>>> be. >>>>>>>> Currently we're pretty much just guessing and if we hit the >>>>>>>> RATE_EXCEEDED >>>>>>>> response, we have to wait an unknown amount of time before making the >>>>>>>> same >>>>>>>> request over (since 30 seconds isn't guaranteed to work). If there's >>>>>>>> an >>>>>>>> accurate response number in the RATE_EXCEEDED response, that's great, >>>>>>>> but I >>>>>>>> think it would even be better to include in successful responses the >>>>>>>> minimum pause time for that particular account so that we can >>>>>>>> programmatically pause that amount of time and simply never hit the >>>>>>>> RATE_EXCEEDED page. >>>>>>>> >>>>>>>> For now I've implemented a flat 40 second pause time which seems to >>>>>>>> be sufficient, although to run my current keyword database, it will >>>>>>>> take >>>>>>>> about 3-4 weeks. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thursday, September 22, 2016 at 10:40:15 PM UTC-4, Thanet Knack >>>>>>>> Praneenararat (AdWords API Team) wrote: >>>>>>>>> >>>>>>>>> Hello Charles, >>>>>>>>> >>>>>>>>> I'm still waiting for the final answer of this. >>>>>>>>> A solution for now is to use exponential backoff policy, for >>>>>>>>> example, if you first pause 5 seconds before the first retry, you >>>>>>>>> could >>>>>>>>> pause 10 seconds after the second and 20 seconds after the third >>>>>>>>> retry, >>>>>>>>> instead of following the *retryAfterSeconds* value. >>>>>>>>> We are discussing about the use of *retryAfterSeconds* but don't >>>>>>>>> reach the conclusion yet. >>>>>>>>> >>>>>>>>> I'll update this thread when getting more information. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Thanet, AdWords API Team >>>>>>>>> >>>>>>>>> >>>>>>>>> On Friday, September 23, 2016 at 6:09:06 AM UTC+9, Charles Michael >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi, was there any update to this? Since originally posting this, >>>>>>>>>> I've narrowed the issue to only occurring between 12pm - 5pm EST, >>>>>>>>>> outside >>>>>>>>>> of this range there are no issues with taking more than 30 seconds >>>>>>>>>> between >>>>>>>>>> requests. >>>>>>>>>> >>>>>>>>> -- -- =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~ Also find us on our blog and Google+: https://googleadsdeveloper.blogspot.com/ https://plus.google.com/+GoogleAdsDevelopers/posts =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~ You received this message because you are subscribed to the Google Groups "AdWords API Forum" group. To post to this group, send email to adwords-api@googlegroups.com To unsubscribe from this group, send email to adwords-api+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/adwords-api?hl=en --- You received this message because you are subscribed to the Google Groups "AdWords API Forum" group. To unsubscribe from this group and stop receiving emails from it, send an email to adwords-api+unsubscr...@googlegroups.com. Visit this group at https://groups.google.com/group/adwords-api. To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/a36795c5-88eb-4ba2-b8bf-a1aa3f264622%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
