On Tuesday, November 12, 2013 11:23:25 PM UTC+1, LightOfMooN wrote: > > I don't want worker without timeout. I just want scheduler task not to be > stopped because of timeout from one of thousands records. Why scheduler > doesn't execute tasks with status=='TIMEOUT'? >
because any task that times out is not a task that anyone should retry automatically. > > I have some function like: > > thing = db.executesql(db(db.things.id>0)._select(..., > orderby=db.things.up_date))[0] > try: > get url, parse, ... > db.executesql(db(db.things.id==thing[0])._update(availability=True, > up_date=request.now)) > except: > db.executesql(db(db.things.id==thing[0])._update(availability=False, > up_date=request.now)) > > And one wrong URL or 404 break all update system because of timeout, which > stops all task. > > it should, because you're wanting all of those to happen in a single transaction. If your app is consistent with a single "fetch", just do foreach thing in things: .....fetch() ....insert() .....commit() ....mark thing as fetched and if you want the "fetch" to have a timeout so you can actually control your task and not having it run away, set a timeout on the fetch, and a proportional timeout on the task. Even better, make a single task that fetches the single url, so your timeoutted tasks will be only the ones that actually timed out. @limedrop : me doesn't, given that is the most bad design choice a task processor can implement. If you want to shoot yourself in the foot, a single task that requeues timeoutted tasks is easy to accomplish, as @lightofmoon found out. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.