Looks like google exponentially backs off a task when it fails, so
here is a version that uses eta to schedule the next task 10s in the
future.
### in queue.yaml
queue:
- name: default
rate: 20/m
bucket_size: 1
### in app.yaml
handlers:
- url: /_ah/queue/default
script: queue.py
login: admin
### in queue.py
import os, sys, datetime
from wsgiref.handlers import CGIHandler
from google.appengine.api.labs import taskqueue
delta = datetime.timedelta(seconds=10)
def wsgi_app(env, res):
taskqueue.add(eta=datetime.datetime.now() + delta)
res('200 OK',[('Content-Type','text/plain')])
return ['']
def main():
CGIHandler().run(wsgi_app)
if __name__ == '__main__':
main()
#### how to launch
1) visit "/_ah/queue/default" in your browser
2) login as an admin which inserts the first tasks
3) view the logs
By the way this is on the production server, the dev server does not
run the tasks automatically.
### view the logs
10-29 10:28PM 11.010 /_ah/queue/default 200 17ms 7cpu_ms
10-29 10:28PM 01.018 /_ah/queue/default 200 28ms 9cpu_ms
10-29 10:27PM 51.007 /_ah/queue/default 200 16ms 5cpu_ms
10-29 10:27PM 41.011 /_ah/queue/default 200 18ms 5cpu_ms
10-29 10:27PM 31.007 /_ah/queue/default 200 38ms 14cpu_ms
10-29 10:27PM 21.009 /_ah/queue/default 200 23ms 7cpu_ms
10-29 10:27PM 11.009 /_ah/queue/default 200 20ms 7cpu_ms
10-29 10:27PM 01.009 /_ah/queue/default 200 20ms 10cpu_ms
10-29 10:26PM 51.007 /_ah/queue/default 200 44ms 35cpu_ms
10-29 10:26PM 41.017 /_ah/queue/default 200 24ms 12cpu_ms
Literally like clock work, a task hits your app once every 10 seconds.
Robin
On Oct 29, 11:47 pm, Robin B <robi...@gmail.com> wrote:
> I looked at the docs and came up with this code:
>
> ### somehow run this code once to insert atleast one task in the queue
> from google.appengine.api.labs import taskqueue
> taskqueue.add(url='/_ah/queue/default') # will return 404
>
> ### add this to queue.yaml
> queue:
> - name: default
> rate: 10/m
> bucket_size: 1
>
> ### look at the logs and see the task hitting your app about every 10
> sec or so, enough to keep a handler hot
>
> 10-29 09:42PM 46.443 /_ah/queue/default 404 7ms 4cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:41PM 55.219 /_ah/queue/default 404 10ms 6cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:41PM 29.602 /_ah/queue/default 404 6ms 3cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:41PM 16.774 /_ah/queue/default 404 5ms 2cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:40PM 54.808 /_ah/queue/default 404 5ms 2cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:40PM 48.794 /_ah/queue/default 404 7ms 3cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:40PM 42.776 /_ah/queue/default 404 10ms 4cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:40PM 36.772 /_ah/queue/default 404 63ms 74cpu_ms 0kb
> AppEngine-Google; (+http://code.google.com/appengine)
> 10-29 09:40PM 13.176 /_ah/queue/default 404 6ms 2cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:40PM 07.161 /_ah/queue/default 404 7ms 4cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
> 10-29 09:40PM 07.161 /_ah/queue/default 404 7ms 4cpu_ms 0kb AppEngine-
> Google; (+http://code.google.com/appengine)
>
> Robin
>
> On Oct 29, 10:32 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > do you have an example of code?
>
> > On Oct 29, 10:16 pm, Robin B <robi...@gmail.com> wrote:
>
> > > > neat - now I get it. That's a nice hack!
> > > > Did you find that out from the Google App Engine group, or thought of
> > > > it yourself?
>
> > > Thought of it myself.
>
> > > Robin
>
> > > On Oct 29, 12:09 am, Richard <richar...@gmail.com> wrote:
>
> > > > neat - now I get it. That's a nice hack!
> > > > Did you find that out from the Google App Engine group, or thought of
> > > > it yourself?
>
> > > > Richard
>
> > > > On Oct 29, 10:29 am, Robin B <robi...@gmail.com> wrote:
>
> > > > > > The docs say a task will run when resources are free. In your
> > > > > > experience do the tasks generally run on time?
>
> > > > > Yes, they are trying to say that task queue webhooks are subject to
> > > > > the same quota restrictions as your app.
>
> > > > > > Also how would you make a task run periodically?
>
> > > > > This is answered in a previous message in this thread:
>
> > > > > "To keep an app hot, use the task
> > > > > queue to hit a url that simply returns a non-200 HTTP status code, so
> > > > > the task will retry indefinitely at up to 10 Hz (10 calls/second), you
> > > > > can specify how often a task is called."
>
> > > > > Tasks are considered 'completed' and removed if they result in a 200
> > > > > HTTP status code.
>
> > > > > > My understanding is
> > > > > > they are for one off events and cron is for periodic.
>
> > > > > You are right, always returning a non-200 status code is obviously not
> > > > > the intended purpose of the task queue, but in this situation we are
> > > > > hacking the task queue to accomplish a specific goal of calling the
> > > > > task periodically since tasks can run much more frequently than cron
> > > > > which is necessary to keep a handler 'hot'.
>
> > > > > Robin
>
> > > > > > Richard
>
> > > > > > On Oct 28, 11:23 pm, Robin B <robi...@gmail.com> wrote:
>
> > > > > > > Oops typo: I said cron is 1/sec it actually its 1/minute, task
> > > > > > > queue
> > > > > > > is 10/second, so cron is not fast enough but task queue is plenty
> > > > > > > fast.
>
> > > > > > > Robin
>
> > > > > > > On Oct 28, 4:18 am, Richard <richar...@gmail.com> wrote:
>
> > > > > > > > that's really useful - thanks Robin!
>
> > > > > > > > if Python handlers last ~15 seconds then wouldn't cron be fast
> > > > > > > > enough
> > > > > > > > (at 1/sec)?
> > > > > > > > Task queue is labelled experimental so I am wary to use it at
> > > > > > > > the
> > > > > > > > moment.
>
> > > > > > > > Richard
>
> > > > > > > > On Oct 28, 4:13 pm, Robin B <robi...@gmail.com> wrote:
>
> > > > > > > > > > How long does it stay cached?
>
> > > > > > > > > Inactive Python handlers used to last ~1 minute, recently it
> > > > > > > > > is closer
> > > > > > > > > to 15 seconds.
>
> > > > > > > > > Java servlets last longer, over a minute, maybe becuase they
> > > > > > > > > are so
> > > > > > > > > slow to boot (6 seconds +).
>
> > > > > > > > > One of the coming releases is supposed to speed up the cold
> > > > > > > > > boot
> > > > > > > > > times.
>
> > > > > > > > > Cron has a maximum frequency of 1 Hz (1 call/second), so it
> > > > > > > > > is no
> > > > > > > > > longer fast enough to do the job. To keep an app hot, use
> > > > > > > > > the task
> > > > > > > > > queue to hit a url that simply returns a non-200 HTTP status
> > > > > > > > > code, so
> > > > > > > > > the task will retry indefinitely at up to 10 Hz (10
> > > > > > > > > calls/second), you
> > > > > > > > > can specify how often a task is called.
>
> > > > > > > > > Robin
>
> > > > > > > > > On Oct 27, 9:53 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > > > > > How long does it stay cached?
>
> > > > > > > > > > On Oct 27, 9:26 pm, Thadeus Burgess <thade...@thadeusb.com>
> > > > > > > > > > wrote:
>
> > > > > > > > > > > run a cron on your local computer to urllib the page ? :)
> > > > > > > > > > > All I can come up
> > > > > > > > > > > with. It is just one of the ways GAE works, when a page
> > > > > > > > > > > is no longer needed,
> > > > > > > > > > > it kills the processes until their needed again. Had this
> > > > > > > > > > > problem with
> > > > > > > > > > > dreamhost before they supported Passenger
>
> > > > > > > > > > > -Thadeus
>
> > > > > > > > > > > On Tue, Oct 27, 2009 at 6:57 PM, mdipierro
> > > > > > > > > > > <mdipie...@cs.depaul.edu> wrote:
>
> > > > > > > > > > > > I do not know.
>
> > > > > > > > > > > > On Oct 27, 6:51 pm, Richard <richar...@gmail.com> wrote:
> > > > > > > > > > > > > is it possible to get web2py on GAE as responsive as
> > > > > > > > > > > > > Slicehost? (See
> > > > > > > > > > > > > Thadeus' app: surrenderthebooty.thadeusb.com)
>
> > > > > > > > > > > > > On Oct 26, 4:42 pm, Richard <richar...@gmail.com>
> > > > > > > > > > > > > wrote:
>
> > > > > > > > > > > > > > I noticed my GAE app is slow for the first webpage
> > > > > > > > > > > > > > I load, but snappy
> > > > > > > > > > > > > > for subsequent webpages.
> > > > > > > > > > > > > > So it seems there is a caching issue, which has
> > > > > > > > > > > > > > been discussed a few
> > > > > > > > > > > > > > times (courtesy of Google site search):
> > > > > > > > > > > >http://groups.google.com/group/web2py/browse_thread/thread/329388bec9....
> > > > > > > > > > > > ..
>
> > > > > > > > > > > > > > Is there a way to keep the web2py structures
> > > > > > > > > > > > > > cached? Perhaps a CRON
> > > > > > > > > > > > > > job to load a page every so often?
>
> > > > > > > > > > > > > > And are there other ways to tune apps on GAE, such
> > > > > > > > > > > > > > as setting
> > > > > > > > > > > > > > "migrate=False"?
>
> > > > > > > > > > > > > > thanks,
> > > > > > > > > > > > > > Richard
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To post to this group, send email to web2py@googlegroups.com
To unsubscribe from this group, send email to
web2py+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en
-~----------~----~----~----~------~----~------~--~---
