Re: [google-appengine] Re: Cast some light on Task Queue execution. I am confused

['Alex Martelli' via Google App Engine]

On Sun, Dec 27, 2015 at 9:32 PM, Naresh Pokuri  wrote:

> Thanks for the reply. When will be bucket refilled? what parameters impact
> bucket filling?
>

One token is added to the bucket every 1/rate -- so if your rate is 60/h,
one token will be added every minute. However, no tokens are added if the
bucket is already full (i.e it contains bucket-size tokens; default
bucket-size is 5 unless explicitly specified).

No other parameter beyond rate and bucket-size constrain bucket filling.

A queued task will grab a token and execute IF there's at least a token
available in the bucket AND this doesn't push the number of tasks
concurrently executing on that queue beyond the max-concurrent-requests
parameter (default for the latter, 1000, unless explicitly specified).
However, max-concurrent-requests constrains task execution, NOT bucket
filling.

I think wikipedia, at https://en.wikipedia.org/wiki/Token_bucket , does a
good job of presenting the general idea of token-bucket algorithms, beyond
the specific use of it in app engine task queues in push mode.


Alex



>
> I have the same configuration in my project for a queue and I have a cron
> job which adds tasks per every five minutes to that queue. On google
> console I see one task getting executed in a minute. If I click on 'Run
> Now' on console all are getting executed in 5 sec. What makes my queue
> execute a task per minute. Am I missing something here?
>
> On Friday, 25 December 2015 02:22:04 UTC+5:30, Zeehad (Cloud Platform
> Support) wrote:
>>
>> The 5 tasks in the ‘push-task’ queue will be completed after 5 seconds
>> given that they each have 1 sec latency.
>>
>> 1. 120 means the bucket will have 120 tokens
>> initially.
>>
>> 2.  60/h will mean a maximum of 60 tasks will be processed
>> per hour.
>>
>> For example, consider the following:
>>
>>
>>-
>>
>>Your above mentioned configurations
>>-
>>
>>100 tasks in the queue
>>-
>>
>>1 second latency each
>>
>>
>> Due to max-concurrent-request being 1 and rate only allowing 60 tasks per
>> hour, 60 tasks will complete in the first 60 seconds. Then no more tasks
>> will be processed for the next 59 minutes of that hour. After the hour is
>> over, the remaining 40 tasks will be processed in 40 seconds.
>>
>> Hope that helps. Cheers!
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/7d587334-72d9-42af-9fa7-616868838ac9%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAE46Be9Up_4No08BQO7ikS%3Dzeb43_SOBmJ2FzmXd8CRX1pYSag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Cast some light on Task Queue execution. I am confused

[Naresh Pokuri]

Thanks for the reply. When will be bucket refilled? what parameters impact 
bucket filling?

I have the same configuration in my project for a queue and I have a cron 
job which adds tasks per every five minutes to that queue. On google 
console I see one task getting executed in a minute. If I click on 'Run 
Now' on console all are getting executed in 5 sec. What makes my queue 
execute a task per minute. Am I missing something here? 

On Friday, 25 December 2015 02:22:04 UTC+5:30, Zeehad (Cloud Platform 
Support) wrote:
>
> The 5 tasks in the ‘push-task’ queue will be completed after 5 seconds 
> given that they each have 1 sec latency.
>
> 1. 120 means the bucket will have 120 tokens 
> initially.
>
> 2.  60/h will mean a maximum of 60 tasks will be processed 
> per hour. 
>
> For example, consider the following:
>
>
>- 
>
>Your above mentioned configurations
>- 
>
>100 tasks in the queue
>- 
>
>1 second latency each
>
>
> Due to max-concurrent-request being 1 and rate only allowing 60 tasks per 
> hour, 60 tasks will complete in the first 60 seconds. Then no more tasks 
> will be processed for the next 59 minutes of that hour. After the hour is 
> over, the remaining 40 tasks will be processed in 40 seconds.
>
> Hope that helps. Cheers!
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7d587334-72d9-42af-9fa7-616868838ac9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Cast some light on Task Queue execution. I am confused

[Zeehad (Cloud Platform Support)]

The 5 tasks in the ‘push-task’ queue will be completed after 5 seconds 
given that they each have 1 sec latency.

1. 120 means the bucket will have 120 tokens 
initially.

2.  60/h will mean a maximum of 60 tasks will be processed per 
hour. 

For example, consider the following:


   - 
   
   Your above mentioned configurations
   - 
   
   100 tasks in the queue
   - 
   
   1 second latency each
   

Due to max-concurrent-request being 1 and rate only allowing 60 tasks per 
hour, 60 tasks will complete in the first 60 seconds. Then no more tasks 
will be processed for the next 59 minutes of that hour. After the hour is 
over, the remaining 40 tasks will be processed in 40 seconds.

Hope that helps. Cheers!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4ac3156b-a5e6-4328-9c80-770050c1da5b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.