Hey Brandon, No offense, but there's almost nothing correct in this. I'm also not sure where some of the terms come from -- I couldn't find references to them in the Python or Java docs.
Tasks will basically be run as fast as your settings allow. There are three main settings: rate, bucket_size, and max_concurrent_requests. The bucket size controls the "token bucket" behavior. Rate controls the frequency at which tasks may be selected from the queue. The concurrent request limit gives you more fine grained control of a queue (handy with the new pricing). The only limit to how many tasks you can have, that I'm aware of, is the stored task count/bytes quotas, and possibly the api calls quota. Tasks shouldn't ever be lost. If an app is actually losing tasks that is a bug in either the app's code or app engine's. I'm not sure what timeout you're referring too. You can specify a delay before a task will be run (eta or countdown params), and for pull queues a lease durration. Robert On Fri, Feb 10, 2012 at 15:52, Brandon Wirtz <drak...@digerat.com> wrote: >> The bucket and rates are for controlling execution. They don't limit > adding >> tasks to a queue. > > Doesn't the Token bucket? > > Also there is a max QueSize in MB you can specify in the YAML. I don't know > what the limit is. > > > My rough understanding was that > > Token Rate was how many task/s you could add. > > Process rate was how fast tasks would be processed. > > Queue Size was number of tasks that could be pending. > > Time Out was the time to expire > > > If your token rate is 5 per S > And your process is 2 per > And your Queue size is 5000 > And your time out is 75 minutes > > If 18 people wanted to make 10 new tasks in 3 seconds. The Token bucket > would empty and only 15 tasks would be created. > > If 500 people an hour wanted to create 20 tasks each (10000 tasks, 3600 > seconds) yields 2.78 Tasks/s the Token bucket would not go dry. The Process > bucket would spin up instances enough to process the tasks, and it would try > to space them out such that it would take 5000 seconds > But you would lose some of those task because the Task Queue would not have > only processed all of the tasks before it hit the 5000 task limit (7200-ish > would complete) > And because you are limiting tasks to 75 minutes, but it would take 1 hour > 23 minutes to process the task a portion would get dropped. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
