[web2py] Re: Monitoring scheduler with supervisord
Any luck with this Simone? On Friday, September 14, 2012 12:39:19 PM UTC-4, Niphlod wrote: ok, give me a few hours, I need to get home first :P --
[web2py] Re: Monitoring scheduler with supervisord
no free time at home to test on every platform, but the code part is done. in a few days. On Thursday, September 20, 2012 6:09:59 PM UTC+2, Yarin wrote: Any luck with this Simone? On Friday, September 14, 2012 12:39:19 PM UTC-4, Niphlod wrote: ok, give me a few hours, I need to get home first :P --
Re: [web2py] Re: Monitoring scheduler with supervisord
You the man On Thu, Sep 20, 2012 at 12:13 PM, Niphlod niph...@gmail.com wrote: no free time at home to test on every platform, but the code part is done. in a few days. On Thursday, September 20, 2012 6:09:59 PM UTC+2, Yarin wrote: Any luck with this Simone? On Friday, September 14, 2012 12:39:19 PM UTC-4, Niphlod wrote: ok, give me a few hours, I need to get home first :P -- --
[web2py] Re: Monitoring scheduler with supervisord
@Niphlod- I was hoping this embedded mode patch was going to become part
of the 2.0 implementation. I see you added a -X flag to allow for launching
the scheduler alongside an app, but we can't use this in our situation
where web2py is running as wsgi under Apache while our scheduler processes
run under Supervisord. Thoughts?
On Wednesday, August 15, 2012 11:44:18 AM UTC-4, Niphlod wrote:
Because web2py lets you launch multiple schedulers with web2py.py -K
myapp,myapp or web2py.py -K myapp1,myapp2 the actual scheduler process is a
subprocess of the main one you launch with web2py.py.
If you launch with the external method, as python gluon/scheduler.py -t
tasks.py -f etc no subprocess is spawned and then the PID reported on
the scheduler_worker table is the same one you have on console.
Obviously a scheduler spawns a process for every task processed (that's
the whole point of the scheduler) but there's no master spawning the
scheduler itself.
BTW, I'm working on having the embedded mode to terminate all workers
processes spawned when the master is killed. It's on the feature list.
PS: for the moment you can make a script to work as embedded mode and
with no subprocesses spawned. Place it in the same folder where web2py.py
lives. This will run a single scheduler process with no subprocesses
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import sys
if '__file__' in globals():
path = os.path.dirname(os.path.abspath(__file__))
os.chdir(path)
else:
path = os.getcwd()
sys.path = [path]+[p for p in sys.path if not p==path]
from gluon.shell import run
from gluon import main
import logging
logging.getLogger().setLevel(logging.INFO)
if __name__ == '__main__':
code = from gluon import current; current._scheduler.loop()
app = 'myapp'
run(app,True,True,None,False,code)
On Wednesday, August 15, 2012 5:48:54 AM UTC+2, Yarin wrote:
I'm trying to use supervisord to monitor scheduler workers.
My supervisord.conf file:
[program:my_scheduler]
command={my_site_path}/web2py.py -K my_app
When I launch supervisord, a worker shows up in the scheduler. However,
when I compare the listed pid in the supervisord console, it doesn't match
up with the pid of the worker as found in the worker name. Sure enough,
when I stop or kill the process in supervisord, it has no effect on the
worker, which keeps on ticking. Meanwhile, the 'killed' worker in the
supervisord console now shows a pid = 0.
This confuses me because when you launch a worker directly in a terminal
window, it doesn't daemonize- so I don't understand why supervisord loses
control of it, and why there are 2 different PIDs. Are there subprocesses
being launched im not aware of, and if so does anyone have an idea of how
supervisord can track them?
--
[web2py] Re: Monitoring scheduler with supervisord
Scheduler is not useful if managed by a webserver, so forget about -X: just like softcron, it is meant to be used when the internal webserver is used, not when web2py is managed by apache co. Having a master starting the actual workers is needed to have multiple workers started with one command through web2py. If you want to manage the workers through supervisord with the standard config you have to use this script. On Friday, September 14, 2012 5:41:47 PM UTC+2, Yarin wrote: @Niphlod- I was hoping this embedded mode patch was going to become part of the 2.0 implementation. I see you added a -X flag to allow for launching the scheduler alongside an app, but we can't use this in our situation where web2py is running as wsgi under Apache while our scheduler processes run under Supervisord. Thoughts? --
[web2py] Re: Monitoring scheduler with supervisord
Right, I was just voting for having the script integrated into web2py outright, so that calling it with maybe a lowercase -k would execute a single scheduler process without spawning subprocesses. On Friday, September 14, 2012 11:57:49 AM UTC-4, Niphlod wrote: Scheduler is not useful if managed by a webserver, so forget about -X: just like softcron, it is meant to be used when the internal webserver is used, not when web2py is managed by apache co. Having a master starting the actual workers is needed to have multiple workers started with one command through web2py. If you want to manage the workers through supervisord with the standard config you have to use this script. On Friday, September 14, 2012 5:41:47 PM UTC+2, Yarin wrote: @Niphlod- I was hoping this embedded mode patch was going to become part of the 2.0 implementation. I see you added a -X flag to allow for launching the scheduler alongside an app, but we can't use this in our situation where web2py is running as wsgi under Apache while our scheduler processes run under Supervisord. Thoughts? --
[web2py] Re: Monitoring scheduler with supervisord
well, maybe we can support not spawning if -K is followed by one app only. Thoughts ? On Friday, September 14, 2012 6:16:08 PM UTC+2, Yarin wrote: Right, I was just voting for having the script integrated into web2py outright, so that calling it with maybe a lowercase -k would execute a single scheduler process without spawning subprocesses. On Friday, September 14, 2012 11:57:49 AM UTC-4, Niphlod wrote: Scheduler is not useful if managed by a webserver, so forget about -X: just like softcron, it is meant to be used when the internal webserver is used, not when web2py is managed by apache co. Having a master starting the actual workers is needed to have multiple workers started with one command through web2py. If you want to manage the workers through supervisord with the standard config you have to use this script. On Friday, September 14, 2012 5:41:47 PM UTC+2, Yarin wrote: @Niphlod- I was hoping this embedded mode patch was going to become part of the 2.0 implementation. I see you added a -X flag to allow for launching the scheduler alongside an app, but we can't use this in our situation where web2py is running as wsgi under Apache while our scheduler processes run under Supervisord. Thoughts? --
[web2py] Re: Monitoring scheduler with supervisord
Sure that works for me On Friday, September 14, 2012 12:25:02 PM UTC-4, Niphlod wrote: well, maybe we can support not spawning if -K is followed by one app only. Thoughts ? On Friday, September 14, 2012 6:16:08 PM UTC+2, Yarin wrote: Right, I was just voting for having the script integrated into web2py outright, so that calling it with maybe a lowercase -k would execute a single scheduler process without spawning subprocesses. On Friday, September 14, 2012 11:57:49 AM UTC-4, Niphlod wrote: Scheduler is not useful if managed by a webserver, so forget about -X: just like softcron, it is meant to be used when the internal webserver is used, not when web2py is managed by apache co. Having a master starting the actual workers is needed to have multiple workers started with one command through web2py. If you want to manage the workers through supervisord with the standard config you have to use this script. On Friday, September 14, 2012 5:41:47 PM UTC+2, Yarin wrote: @Niphlod- I was hoping this embedded mode patch was going to become part of the 2.0 implementation. I see you added a -X flag to allow for launching the scheduler alongside an app, but we can't use this in our situation where web2py is running as wsgi under Apache while our scheduler processes run under Supervisord. Thoughts? --
[web2py] Re: Monitoring scheduler with supervisord
ok, give me a few hours, I need to get home first :P --
[web2py] Re: Monitoring scheduler with supervisord
Because web2py let you launch multiple schedulers with web2py.py -K
myapp,myapp or web2py.py -K myapp1,myapp2 the actual scheduler process is a
subprocess of the main one you launch with web2py.py.
If you launch with the external method, as python gluon/scheduler.py -t
tasks.py -f etc no subprocess is spawned and then the PID reported on
the scheduler_worker table is the same one you have on console.
Obviously a scheduler spawns a process for every task processed (that's the
whole point of the scheduler) but there's no master spawning the
scheduler itself.
BTW, I'm working on having the embedded mode to terminate all workers
processes spawned when the master is killed. It's on the feature list.
PS: for the moment you can make a script to work as embedded mode and
with no subprocesses spawned. Place it in the same folder where web2py.py
lives. This will run a single scheduler process with no subprocesses
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from gluon.shell import run
import logging
logging.getLogger().setLevel(logging.INFO)
if __name__ == '__main__':
code = from gluon import current; current._scheduler.loop()
app = 'myapp'
run(app,True,True,None,False,code)
On Wednesday, August 15, 2012 5:48:54 AM UTC+2, Yarin wrote:
I'm trying to use supervisord to monitor scheduler workers.
My supervisord.conf file:
[program:my_scheduler]
command={my_site_path}/web2py.py -K my_app
When I launch supervisord, a worker shows up in the scheduler. However,
when I compare the listed pid in the supervisord console, it doesn't match
up with the pid of the worker as found in the worker name. Sure enough,
when I stop or kill the process in supervisord, it has no effect on the
worker, which keeps on ticking. Meanwhile, the 'killed' worker in the
supervisord console now shows a pid = 0.
This confuses me because when you launch a worker directly in a terminal
window, it doesn't daemonize- so I don't understand why supervisord loses
control of it, and why there are 2 different PIDs. Are there subprocesses
being launched im not aware of, and if so does anyone have an idea of how
supervisord can track them?
--
[web2py] Re: Monitoring scheduler with supervisord
The embed-mode script works great and allows us to use Supervisord- Thanks--
On Wednesday, August 15, 2012 11:44:18 AM UTC-4, Niphlod wrote:
Because web2py lets you launch multiple schedulers with web2py.py -K
myapp,myapp or web2py.py -K myapp1,myapp2 the actual scheduler process is a
subprocess of the main one you launch with web2py.py.
If you launch with the external method, as python gluon/scheduler.py -t
tasks.py -f etc no subprocess is spawned and then the PID reported on
the scheduler_worker table is the same one you have on console.
Obviously a scheduler spawns a process for every task processed (that's
the whole point of the scheduler) but there's no master spawning the
scheduler itself.
BTW, I'm working on having the embedded mode to terminate all workers
processes spawned when the master is killed. It's on the feature list.
PS: for the moment you can make a script to work as embedded mode and
with no subprocesses spawned. Place it in the same folder where web2py.py
lives. This will run a single scheduler process with no subprocesses
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from gluon.shell import run
from gluon import main
import logging
logging.getLogger().setLevel(logging.INFO)
if __name__ == '__main__':
code = from gluon import current; current._scheduler.loop()
app = 'myapp'
run(app,True,True,None,False,code)
On Wednesday, August 15, 2012 5:48:54 AM UTC+2, Yarin wrote:
I'm trying to use supervisord to monitor scheduler workers.
My supervisord.conf file:
[program:my_scheduler]
command={my_site_path}/web2py.py -K my_app
When I launch supervisord, a worker shows up in the scheduler. However,
when I compare the listed pid in the supervisord console, it doesn't match
up with the pid of the worker as found in the worker name. Sure enough,
when I stop or kill the process in supervisord, it has no effect on the
worker, which keeps on ticking. Meanwhile, the 'killed' worker in the
supervisord console now shows a pid = 0.
This confuses me because when you launch a worker directly in a terminal
window, it doesn't daemonize- so I don't understand why supervisord loses
control of it, and why there are 2 different PIDs. Are there subprocesses
being launched im not aware of, and if so does anyone have an idea of how
supervisord can track them?
--
