[web2py] Re: scheduler workers get deleted constantly and one task is assigned to many different runs

2015-05-04 Thread Pengfei Yu
This need to be verified with AWS's tech support I guess. The instances in AWS region cn-north-1 are weird in many aspects from my recent experiences... On Sunday, May 3, 2015 at 8:18:59 AM UTC-4, Niphlod wrote: seems that the same thing that happened a long time ago on travis-ci: glad you

[web2py] Re: scheduler workers get deleted constantly and one task is assigned to many different runs

2015-05-03 Thread Niphlod
seems that the same thing that happened a long time ago on travis-ci: glad you worked it out, but I'm really unsure why different regions should carry different versions of the same operating system. On Thursday, April 30, 2015 at 6:57:08 PM UTC+2, Pengfei Yu wrote: Hi Niphlod, I think I

[web2py] Re: scheduler workers get deleted constantly and one task is assigned to many different runs

2015-04-30 Thread Niphlod
On Wednesday, April 29, 2015 at 11:53:56 PM UTC+2, Pengfei Yu wrote: Hi Niphlod and Dave, I think the worker is deleted because the the last heartbeat doesn't change for a while. When I comment out the line #dead_workers.delete() in gluon/schdeuler.py, the task is stuck as QUEUED without

[web2py] Re: scheduler workers get deleted constantly and one task is assigned to many different runs

2015-04-30 Thread Pengfei Yu
Hi Niphlod, I think I solve the problem. Followed by my last post, the issure is in shared memory for the multiprocessing module of python. According to the solution on this link (http://stackoverflow.com/questions/2009278/python-multiprocessing-permission-denied ), I let none being mounted

[web2py] Re: scheduler workers get deleted constantly and one task is assigned to many different runs

2015-04-30 Thread Pengfei Yu
Hi Niphlod, Thanks for the suggestions! When I DEBUG it, I get this error for my task function named run: DEBUG:web2py.scheduler.ip-172-31-2-63#102815: work to do 8 DEBUG:web2py.scheduler.ip-172-31-2-63#102815:new scheduler_run record INFO:web2py.scheduler.ip-172-31-2-63#102815:new task 8

[web2py] Re: scheduler workers get deleted constantly and one task is assigned to many different runs

2015-04-29 Thread Niphlod
whoa. Thank god I put some statistics in to help with debugging.. It seems that the worker processes are handling one task and dieing in the middle of the process. Also, it seems that your task is never-ending. workers that haven't a task to be processed are running fine (empty_runs on

[web2py] Re: scheduler workers get deleted constantly and one task is assigned to many different runs

2015-04-29 Thread Dave S
On Wednesday, April 29, 2015 at 10:17:18 AM UTC-7, Pengfei Yu wrote: [...] I don't understand why the worker is deleted immediately once it has been assigned to a run for a task. Are the workers being deleted, or are they aborting because of an internal error? If the latter, there

[web2py] Re: scheduler workers get deleted constantly and one task is assigned to many different runs

2015-04-29 Thread Pengfei Yu
Hi Niphlod and Dave, I think the worker is deleted because the the last heartbeat doesn't change for a while. When I comment out the line #dead_workers.delete() in gluon/schdeuler.py, the task is stuck as QUEUED without new run record created. And the worker supposed to be deleted with the