Re: [web2py] Re: scheduler question
On Tuesday, February 6, 2018 at 12:10:18 PM UTC-8, Andrea Fae' wrote:
>
> I created like in the book. It's a model. I'm scheduling from the database
> administrative interface. 2 days ago worked perfectly! I don't know!
>
>
I've never tried that. The production system I run the scheduler on
doesn't have a GUI to run a browser from, and I don't often set up SSL port
forwarding to run appadmin from my client system. I schedule either
through a web page (exposed controller function) or through using -S on the
command line (non-exposed controller function.
See
https://github.com/web2py/web2py-book/blob/master/sources/29-web2py-english/13.markmin#L942>
(The online copy of the book hasn't been synched with github for a while,
it seems, so I can't point you at a pretty copy.)
/dps
> Il 06 feb 2018 9:05 PM, "Dave S" > ha
> scritto:
>
>>
>>
>> On Tuesday, February 6, 2018 at 7:58:15 AM UTC-8, Andrea Fae' wrote:
>>>
>>> Hello, days ago I started to use scheduler and all was working very
>>> well. I could see db.scheduler_run.task_id lines in the db with all the
>>> results and console.
>>>
>>> Now the scheduler works very well but I don't see nothing in the
>>> table db.scheduler_run.task_id. Why? Talk completed without problems...
>>>
>>
>> I don't know. My scheduler_run rows all show a task_id. I have more
>> than 2500 runs since switching to 2.15.4 (minimum of 3 runs a day). I had
>> 1900+ runs under 2.14.6.
>>
>>
>>> This is my routine (from the moment I tried to use AppConfig - now is
>>> commented - I noted no lines in the table above...)
>>>
>>>
>> Where does this code live? In a model file, a module, or ??
>>
>> How do you schedule it?
>>
>>
>>
>>> def chiusura_automatica_ticket():
>>> esito = False
>>> # questa è la data odierna
>>> todate = datetime.date.today()
>>> # print 'todate = ', todate
>>>
>>> # carico da appconfig.ini il numero di giorni lavorativi in cui lo
>>> scehdulature chiude in automatico le richieste di intervento
>>> # mycustom_conf = AppConfig()
>>> # questo è il delta in giorni lavorativi indietro
>>> # giorni_da_apertura =
>>> mycustom_conf.get('custom.automatic_ticket_closing')
>>> giorni_da_apertura = 30
>>> print giorni_da_apertura
>>>
>>> # questa è la data di partenza
>>> fromdate = date_by_subtract_business_days(todate,giorni_da_apertura)
>>> # print 'fromdate = ',fromdate
>>>
>>> # seleziono i ticket prima della data "fromdate" che non siano
>>> ancora chiusi
>>> query = (db.ticket.created_on < fromdate) & (db.ticket.chiuso ==
>>> False)
>>> db(query).update(chiuso = True, risoluzione = 'CHIUSO
>>> AUTOMATICAMENTE')
>>> # commit necessaria
>>> esito = db.commit()
>>> return esito
>>>
>>
>>
>> /dps
>>
>>
>> --
>> Resources:
>> - http://web2py.com
>> - http://web2py.com/book (Documentation)
>> - http://github.com/web2py/web2py (Source code)
>> - https://code.google.com/p/web2py/issues/list (Report Issues)
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "web2py-users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/web2py/Xjtf3rRBZyU/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> web2py+un...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [web2py] Re: scheduler question
I created like in the book. It's a model. I'm scheduling from the database
administrative interface. 2 days ago worked perfectly! I don't know!
Il 06 feb 2018 9:05 PM, "Dave S" ha scritto:
>
>
> On Tuesday, February 6, 2018 at 7:58:15 AM UTC-8, Andrea Fae' wrote:
>>
>> Hello, days ago I started to use scheduler and all was working very well.
>> I could see db.scheduler_run.task_id lines in the db with all the results
>> and console.
>>
>> Now the scheduler works very well but I don't see nothing in the
>> table db.scheduler_run.task_id. Why? Talk completed without problems...
>>
>
> I don't know. My scheduler_run rows all show a task_id. I have more than
> 2500 runs since switching to 2.15.4 (minimum of 3 runs a day). I had 1900+
> runs under 2.14.6.
>
>
>> This is my routine (from the moment I tried to use AppConfig - now is
>> commented - I noted no lines in the table above...)
>>
>>
> Where does this code live? In a model file, a module, or ??
>
> How do you schedule it?
>
>
>
>> def chiusura_automatica_ticket():
>> esito = False
>> # questa è la data odierna
>> todate = datetime.date.today()
>> # print 'todate = ', todate
>>
>> # carico da appconfig.ini il numero di giorni lavorativi in cui lo
>> scehdulature chiude in automatico le richieste di intervento
>> # mycustom_conf = AppConfig()
>> # questo è il delta in giorni lavorativi indietro
>> # giorni_da_apertura = mycustom_conf.get('custom.auto
>> matic_ticket_closing')
>> giorni_da_apertura = 30
>> print giorni_da_apertura
>>
>> # questa è la data di partenza
>> fromdate = date_by_subtract_business_days(todate,giorni_da_apertura)
>> # print 'fromdate = ',fromdate
>>
>> # seleziono i ticket prima della data "fromdate" che non siano ancora
>> chiusi
>> query = (db.ticket.created_on < fromdate) & (db.ticket.chiuso ==
>> False)
>> db(query).update(chiuso = True, risoluzione = 'CHIUSO
>> AUTOMATICAMENTE')
>> # commit necessaria
>> esito = db.commit()
>> return esito
>>
>
>
> /dps
>
>
> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/web2py/Xjtf3rRBZyU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[web2py] Re: scheduler question
On Tuesday, February 6, 2018 at 7:58:15 AM UTC-8, Andrea Fae' wrote:
>
> Hello, days ago I started to use scheduler and all was working very well.
> I could see db.scheduler_run.task_id lines in the db with all the results
> and console.
>
> Now the scheduler works very well but I don't see nothing in the
> table db.scheduler_run.task_id. Why? Talk completed without problems...
>
I don't know. My scheduler_run rows all show a task_id. I have more than
2500 runs since switching to 2.15.4 (minimum of 3 runs a day). I had 1900+
runs under 2.14.6.
> This is my routine (from the moment I tried to use AppConfig - now is
> commented - I noted no lines in the table above...)
>
>
Where does this code live? In a model file, a module, or ??
How do you schedule it?
> def chiusura_automatica_ticket():
> esito = False
> # questa è la data odierna
> todate = datetime.date.today()
> # print 'todate = ', todate
>
> # carico da appconfig.ini il numero di giorni lavorativi in cui lo
> scehdulature chiude in automatico le richieste di intervento
> # mycustom_conf = AppConfig()
> # questo è il delta in giorni lavorativi indietro
> # giorni_da_apertura =
> mycustom_conf.get('custom.automatic_ticket_closing')
> giorni_da_apertura = 30
> print giorni_da_apertura
>
> # questa è la data di partenza
> fromdate = date_by_subtract_business_days(todate,giorni_da_apertura)
> # print 'fromdate = ',fromdate
>
> # seleziono i ticket prima della data "fromdate" che non siano ancora
> chiusi
> query = (db.ticket.created_on < fromdate) & (db.ticket.chiuso == False)
> db(query).update(chiuso = True, risoluzione = 'CHIUSO AUTOMATICAMENTE')
> # commit necessaria
> esito = db.commit()
> return esito
>
/dps
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Scheduler Question
uhm. sqlite lockings then ? On Monday, June 15, 2015 at 6:13:04 PM UTC+2, Michael Gheith wrote: > > I am querying a remote PostgreSQL database, and then inserting certain > values into my local SQLite database, that's all... > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Scheduler Question
I am querying a remote PostgreSQL database, and then inserting certain values into my local SQLite database, that's all... -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Scheduler Question
seems that the function that goes into timeout leaves zombie processes
behind. What are you doing in that function ?
On Friday, June 12, 2015 at 11:56:09 PM UTC+2, Michael Gheith wrote:
>
> Hmmm. Looking in the database, I have 21 TIMEOUTS which corresponds to
> the 21 workers with all the same process ID of 10978... a red herring?
>
>
> $ pstree -p 10978
>
> python(10978)─┬─python(1719)
>
> ├─python(2063)
>
> ├─python(2977)
>
> ├─python(5383)
>
> ├─python(8176)
>
> ├─python(11013)
>
> ├─python(11069)
>
> ├─python(11521)
>
> ├─python(14466)
>
> ├─python(14490)
>
> ├─python(15190)
>
> ├─python(16210)
>
> ├─python(17019)
>
> ├─python(20546)
>
> ├─python(27816)
>
> ├─python(27882)
>
> ├─python(28702)
>
> ├─python(29459)
>
> ├─python(30343)
>
> ├─python(32193)
>
> └─{python}(10993)
>
>
> On Friday, June 12, 2015 at 3:49:12 PM UTC-5, Niphlod wrote:
>>
>> inspect the log to see why another worker gets started. Each worker
>> started can result AT MOST as two processes, the worker itself and the
>> process that actually executes the task.
>>
>> On Friday, June 12, 2015 at 10:36:35 PM UTC+2, Michael Gheith wrote:
>>>
>>> I have an application that uses the scheduler. It runs really good,
>>> however I noticed odd behavior which I don't think is normal.
>>>
>>> When I launch a worker thread (just one) with the following everything
>>> works great:
>>> python web2py.py -K myapp
>>>
>>> A few days later I will come back to the node that the application is
>>> running on, and will run the following to see my web2py processes:
>>> pgrep -fl web2py
>>>
>>> and I will get something like the following:
>>>
>>> 11013 python web2py.py -K myapp
>>>
>>> 11069 python web2py.py -K myapp
>>>
>>> 11521 python web2py.py -K myapp
>>>
>>> 14466 python web2py.py -K myapp
>>>
>>> 14490 python web2py.py -K myapp
>>>
>>> 15190 python web2py.py -K myapp
>>>
>>> 16210 python web2py.py -K myapp
>>>
>>> 17019 python web2py.py -K myapp
>>>
>>>
>>> Why am I getting more than one of these processes? I should just have
>>> one, right? Is this a bug, or normal behavior? Please advise, thanks!!
>>>
>>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Scheduler Question
Hmmm. Looking in the database, I have 21 TIMEOUTS which corresponds to the
21 workers with all the same process ID of 10978... a red herring?
$ pstree -p 10978
python(10978)─┬─python(1719)
├─python(2063)
├─python(2977)
├─python(5383)
├─python(8176)
├─python(11013)
├─python(11069)
├─python(11521)
├─python(14466)
├─python(14490)
├─python(15190)
├─python(16210)
├─python(17019)
├─python(20546)
├─python(27816)
├─python(27882)
├─python(28702)
├─python(29459)
├─python(30343)
├─python(32193)
└─{python}(10993)
On Friday, June 12, 2015 at 3:49:12 PM UTC-5, Niphlod wrote:
>
> inspect the log to see why another worker gets started. Each worker
> started can result AT MOST as two processes, the worker itself and the
> process that actually executes the task.
>
> On Friday, June 12, 2015 at 10:36:35 PM UTC+2, Michael Gheith wrote:
>>
>> I have an application that uses the scheduler. It runs really good,
>> however I noticed odd behavior which I don't think is normal.
>>
>> When I launch a worker thread (just one) with the following everything
>> works great:
>> python web2py.py -K myapp
>>
>> A few days later I will come back to the node that the application is
>> running on, and will run the following to see my web2py processes:
>> pgrep -fl web2py
>>
>> and I will get something like the following:
>>
>> 11013 python web2py.py -K myapp
>>
>> 11069 python web2py.py -K myapp
>>
>> 11521 python web2py.py -K myapp
>>
>> 14466 python web2py.py -K myapp
>>
>> 14490 python web2py.py -K myapp
>>
>> 15190 python web2py.py -K myapp
>>
>> 16210 python web2py.py -K myapp
>>
>> 17019 python web2py.py -K myapp
>>
>>
>> Why am I getting more than one of these processes? I should just have
>> one, right? Is this a bug, or normal behavior? Please advise, thanks!!
>>
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Scheduler Question
inspect the log to see why another worker gets started. Each worker started can result AT MOST as two processes, the worker itself and the process that actually executes the task. On Friday, June 12, 2015 at 10:36:35 PM UTC+2, Michael Gheith wrote: > > I have an application that uses the scheduler. It runs really good, > however I noticed odd behavior which I don't think is normal. > > When I launch a worker thread (just one) with the following everything > works great: > python web2py.py -K myapp > > A few days later I will come back to the node that the application is > running on, and will run the following to see my web2py processes: > pgrep -fl web2py > > and I will get something like the following: > > 11013 python web2py.py -K myapp > > 11069 python web2py.py -K myapp > > 11521 python web2py.py -K myapp > > 14466 python web2py.py -K myapp > > 14490 python web2py.py -K myapp > > 15190 python web2py.py -K myapp > > 16210 python web2py.py -K myapp > > 17019 python web2py.py -K myapp > > > Why am I getting more than one of these processes? I should just have > one, right? Is this a bug, or normal behavior? Please advise, thanks!! > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: scheduler question
On Thursday, January 16, 2014 5:13:59 AM UTC+1, Jayadevan M wrote: > > A couple of questions about web2py scheduler - > 1) I have configured web2py using nginx znd uwsgi on CentOS. These > services will automatically restart if the server reboots. How can I ensure > that web2py scheduler also restarts? > Use whatever CentOS offers as a service monitor. Don't know CentOS enough to tell what's the best option out there. Alternatively, since its in your toolbelt yet, checkout http://uwsgi-docs.readthedocs.org/en/latest/AttachingDaemons.html > 2) Is this the right way of starting the scheduler from command line? > nohup python web2py.py -K myapp & > > The right way is just web2py.py -K yourapp depending on what process/service manager you'll use to watch that process, you might need to add "sugar" around it, such as the line you posted. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Scheduler question
Dear All,
I want help to send sms with attachment, But i didn't succeed.Please help
me.
*Pankaj Pathak*
*Software Developer *
*Shrideva Technomech Pvt. Ltd.*
On Fri, Oct 4, 2013 at 9:24 PM, Niphlod wrote:
>
>
> Il giorno venerdì 4 ottobre 2013 11:31:33 UTC+2, Marin Pranjić ha scritto:
>
>>
>>
>>
>> On Wed, Oct 2, 2013 at 4:13 PM, Niphlod wrote:
>>
>>>
>>>
>>> Il giorno mercoledì 2 ottobre 2013 15:15:16 UTC+2, Marin Pranjić ha
>>> scritto:
>>>
Hi,
I have a task queue that runs in the background. I want to switch to
scheduler because I need more workers.
Queued tasks are long running (video transcoding, 10-20 minutes each).
1. If I terminate (Ctrl+C) scheduler process, scheduler_run record is
STOPPED and scheduler_task record is back to QUEUED status. That's cool.
What happens if there is a system failure or power supply loss? Is the
record stuck at RUNNING or is there a mechanism that 'fixes' invalid state
once scheduler is up again?
This is likely to happen multiple times.
>>>
>>> How do you handle it is up to you. schedule a task that inspects what
>>> has been done by your task, eventually prune temporary files, reschedule
>>> the task, do whatever you need. There's no limits in configuration because
>>> anyone needs different type of tasks ("retry blindly" vs "stop and warn me"
>>> approaches are very different and scheduler doesn't pose a limit on those).
>>> However in that particular case (assuming your using a TIMEOUT, which you
>>> definitely **should**) tasks in RUNNING will be marked as TIMEOUT.
>>>
>>>
>>
>> What do you mean "using a TIMEOUT" ?
>> I just tried: killall -s SIGKILL while my task was running. Once I
>> started scheduler again it was set to ASSIGNED. This is what I want, but I
>> don't understand timeout thing, can you please explain?
>>
>
> queue_task() uses the default timeout of 60 seconds. this means that a
> task can get at most 60 seconds to run. if you queue a task with a relly
> long timeout, than the scheduler won't potentially "return" from a runaway
> task and will most surely "hiccup" not processing any other task.
>
> every once in while the schedulers watch over the schedulers that are not
> sending the heartbeat (most likely killed) and get every running task
> associated with those workers, updating those tasks to queued.
>
>
>>
>>
>>>
2. Is there a way of stopping a task without terminating a worker?
>>> Nope, but it's not that hard to schedule the os to restart a worker if
>>> it's killed, and it's a task better handled separately (attached processes,
>>> zombie resources, etc).
>>> If you follow the deployment config with upstart as soon as it's killed
>>> it restarts automatically.
>>>
>>>
>>
>> I'm not sure if I understand. I have 4 workers running, and I need to
>> kill only one of them and restart it. I don't want to restart other 3
>> workers. How do I do that?
>>
>>
>
> set it as TERMINATE or KILL and then do web2py.py -K appname (or configure
> 4 upstart services with web2py.py -K appname and let upstart restart them)
>
>
>
> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Scheduler question
Il giorno venerdì 4 ottobre 2013 11:31:33 UTC+2, Marin Pranjić ha scritto:
>
>
>
>
> On Wed, Oct 2, 2013 at 4:13 PM, Niphlod >wrote:
>
>>
>>
>> Il giorno mercoledì 2 ottobre 2013 15:15:16 UTC+2, Marin Pranjić ha
>> scritto:
>>
>>> Hi,
>>>
>>> I have a task queue that runs in the background. I want to switch to
>>> scheduler because I need more workers.
>>> Queued tasks are long running (video transcoding, 10-20 minutes each).
>>>
>>> 1. If I terminate (Ctrl+C) scheduler process, scheduler_run record is
>>> STOPPED and scheduler_task record is back to QUEUED status. That's cool.
>>> What happens if there is a system failure or power supply loss? Is the
>>> record stuck at RUNNING or is there a mechanism that 'fixes' invalid state
>>> once scheduler is up again?
>>> This is likely to happen multiple times.
>>>
>>
>> How do you handle it is up to you. schedule a task that inspects what has
>> been done by your task, eventually prune temporary files, reschedule the
>> task, do whatever you need. There's no limits in configuration because
>> anyone needs different type of tasks ("retry blindly" vs "stop and warn me"
>> approaches are very different and scheduler doesn't pose a limit on those).
>> However in that particular case (assuming your using a TIMEOUT, which you
>> definitely **should**) tasks in RUNNING will be marked as TIMEOUT.
>>
>>
>
> What do you mean "using a TIMEOUT" ?
> I just tried: killall -s SIGKILL while my task was running. Once I started
> scheduler again it was set to ASSIGNED. This is what I want, but I don't
> understand timeout thing, can you please explain?
>
queue_task() uses the default timeout of 60 seconds. this means that a task
can get at most 60 seconds to run. if you queue a task with a relly long
timeout, than the scheduler won't potentially "return" from a runaway task
and will most surely "hiccup" not processing any other task.
every once in while the schedulers watch over the schedulers that are not
sending the heartbeat (most likely killed) and get every running task
associated with those workers, updating those tasks to queued.
>
>
>>
>>> 2. Is there a way of stopping a task without terminating a worker?
>>>
>>>
>> Nope, but it's not that hard to schedule the os to restart a worker if
>> it's killed, and it's a task better handled separately (attached processes,
>> zombie resources, etc).
>> If you follow the deployment config with upstart as soon as it's killed
>> it restarts automatically.
>>
>>
>
> I'm not sure if I understand. I have 4 workers running, and I need to kill
> only one of them and restart it. I don't want to restart other 3 workers.
> How do I do that?
>
>
set it as TERMINATE or KILL and then do web2py.py -K appname (or configure
4 upstart services with web2py.py -K appname and let upstart restart them)
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [web2py] Re: Scheduler question
On Wed, Oct 2, 2013 at 4:13 PM, Niphlod wrote:
>
>
> Il giorno mercoledì 2 ottobre 2013 15:15:16 UTC+2, Marin Pranjić ha
> scritto:
>
>> Hi,
>>
>> I have a task queue that runs in the background. I want to switch to
>> scheduler because I need more workers.
>> Queued tasks are long running (video transcoding, 10-20 minutes each).
>>
>> 1. If I terminate (Ctrl+C) scheduler process, scheduler_run record is
>> STOPPED and scheduler_task record is back to QUEUED status. That's cool.
>> What happens if there is a system failure or power supply loss? Is the
>> record stuck at RUNNING or is there a mechanism that 'fixes' invalid state
>> once scheduler is up again?
>> This is likely to happen multiple times.
>>
>
> How do you handle it is up to you. schedule a task that inspects what has
> been done by your task, eventually prune temporary files, reschedule the
> task, do whatever you need. There's no limits in configuration because
> anyone needs different type of tasks ("retry blindly" vs "stop and warn me"
> approaches are very different and scheduler doesn't pose a limit on those).
> However in that particular case (assuming your using a TIMEOUT, which you
> definitely **should**) tasks in RUNNING will be marked as TIMEOUT.
>
>
What do you mean "using a TIMEOUT" ?
I just tried: killall -s SIGKILL while my task was running. Once I started
scheduler again it was set to ASSIGNED. This is what I want, but I don't
understand timeout thing, can you please explain?
>
>> 2. Is there a way of stopping a task without terminating a worker?
>>
>>
> Nope, but it's not that hard to schedule the os to restart a worker if
> it's killed, and it's a task better handled separately (attached processes,
> zombie resources, etc).
> If you follow the deployment config with upstart as soon as it's killed it
> restarts automatically.
>
>
I'm not sure if I understand. I have 4 workers running, and I need to kill
only one of them and restart it. I don't want to restart other 3 workers.
How do I do that?
> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to the Google Groups
> "web2py-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to web2py+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler question
Il giorno giovedì 3 ottobre 2013 08:19:48 UTC+2, Andrew W ha scritto:
>
> Hi Niphlod,
> If I could add to the questions (I'm having some success with the
> scheduler but there's a few gaps in my understanding):
> What process removes the rows from the scheduler_worker table ?
> Does it make any difference to Kill a worker by updating it's status or by
> just Ctl-C (Or closing the window in Windows) to stop the task ?
>
>
ctrl+c means killing from outside. setting to KILL means killing (still
abruptly) but within inside the process. YMMV but I assume it's always best
to avoid ctrl+c in production :D
> I have one worker row with a status of KILL. The scheduler worker process
> has stopped correctly. Does there need to be at least one active worker
> to clean up this table ? Will this row be removed when I restart a worker
> instance?
>
>
yes. all workers do a cleanup at the start and every once in a while
> What happens when I have several Assigned Tasks, but the worker to which
> they are assigned is Killed ? Will they be reassigned automatically ?
>
yes.
>
> Does KILLing a worker terminate the task that the worker is running as
> well as the specific worker instance ? I expect so but to be clear there
> are two things running - the worker instance, and the specific task that
> the worker has initiated. Some of the wording talks about the worker
> task, but is that referring to both or just one of these two processes ?
>
when you KILL a worker, the task gets terminated abruptly and the worker
exits as soon as the task process has been shut down
>
> Probably just a general web2py question, but I have a menu item that calls
> a controller which runs " scheduler.queue_task(" . Unfortunately I get
> two rows inserted into scheduler_task and i can't work out why. The
> controller appears to be called twice.
>
log a line before calling queue_task. then it's just a matter to see where
the request comes from :D
>
> Thanks
>
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler question
Hi Niphlod,
If I could add to the questions (I'm having some success with the scheduler
but there's a few gaps in my understanding):
What process removes the rows from the scheduler_worker table ?
Does it make any difference to Kill a worker by updating it's status or by
just Ctl-C (Or closing the window in Windows) to stop the task ?
I have one worker row with a status of KILL. The scheduler worker process
has stopped correctly. Does there need to be at least one active worker
to clean up this table ? Will this row be removed when I restart a worker
instance?
What happens when I have several Assigned Tasks, but the worker to which
they are assigned is Killed ? Will they be reassigned automatically ?
Does KILLing a worker terminate the task that the worker is running as well
as the specific worker instance ? I expect so but to be clear there are
two things running - the worker instance, and the specific task that the
worker has initiated. Some of the wording talks about the worker task,
but is that referring to both or just one of these two processes ?
Probably just a general web2py question, but I have a menu item that calls
a controller which runs " scheduler.queue_task(" . Unfortunately I get
two rows inserted into scheduler_task and i can't work out why. The
controller appears to be called twice.
Thanks
On Thursday, October 3, 2013 12:13:50 AM UTC+10, Niphlod wrote:
>
>
> Il giorno mercoledì 2 ottobre 2013 15:15:16 UTC+2, Marin Pranjić ha
> scritto:
>>
>> Hi,
>>
>> I have a task queue that runs in the background. I want to switch to
>> scheduler because I need more workers.
>> Queued tasks are long running (video transcoding, 10-20 minutes each).
>>
>> 1. If I terminate (Ctrl+C) scheduler process, scheduler_run record is
>> STOPPED and scheduler_task record is back to QUEUED status. That's cool.
>> What happens if there is a system failure or power supply loss? Is the
>> record stuck at RUNNING or is there a mechanism that 'fixes' invalid state
>> once scheduler is up again?
>> This is likely to happen multiple times.
>>
>
> How do you handle it is up to you. schedule a task that inspects what has
> been done by your task, eventually prune temporary files, reschedule the
> task, do whatever you need. There's no limits in configuration because
> anyone needs different type of tasks ("retry blindly" vs "stop and warn me"
> approaches are very different and scheduler doesn't pose a limit on those).
> However in that particular case (assuming your using a TIMEOUT, which you
> definitely **should**) tasks in RUNNING will be marked as TIMEOUT.
>
>
>>
>> 2. Is there a way of stopping a task without terminating a worker?
>>
>>
> Nope, but it's not that hard to schedule the os to restart a worker if
> it's killed, and it's a task better handled separately (attached processes,
> zombie resources, etc).
> If you follow the deployment config with upstart as soon as it's killed it
> restarts automatically.
>
>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Scheduler question
Il giorno mercoledì 2 ottobre 2013 15:15:16 UTC+2, Marin Pranjić ha scritto:
>
> Hi,
>
> I have a task queue that runs in the background. I want to switch to
> scheduler because I need more workers.
> Queued tasks are long running (video transcoding, 10-20 minutes each).
>
> 1. If I terminate (Ctrl+C) scheduler process, scheduler_run record is
> STOPPED and scheduler_task record is back to QUEUED status. That's cool.
> What happens if there is a system failure or power supply loss? Is the
> record stuck at RUNNING or is there a mechanism that 'fixes' invalid state
> once scheduler is up again?
> This is likely to happen multiple times.
>
How do you handle it is up to you. schedule a task that inspects what has
been done by your task, eventually prune temporary files, reschedule the
task, do whatever you need. There's no limits in configuration because
anyone needs different type of tasks ("retry blindly" vs "stop and warn me"
approaches are very different and scheduler doesn't pose a limit on those).
However in that particular case (assuming your using a TIMEOUT, which you
definitely **should**) tasks in RUNNING will be marked as TIMEOUT.
>
> 2. Is there a way of stopping a task without terminating a worker?
>
>
Nope, but it's not that hard to schedule the os to restart a worker if it's
killed, and it's a task better handled separately (attached processes,
zombie resources, etc).
If you follow the deployment config with upstart as soon as it's killed it
restarts automatically.
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: scheduler question, how to use queue_task?
Right, my vars dicts seemed a bit odd when I looked at them, now I know why ;) Same thing bit me, apparently. Thanks for reporting back! Regards, Ales On Thursday, March 7, 2013 12:15:16 PM UTC+1, Tim Richardson wrote: > > > > On Thursday, 7 March 2013 21:49:12 UTC+11, Niphlod wrote: >> >> seems perfectly legit in the snippet, maybe there's something else in >> your "complete" code. Can you inspect the errors returned by queue_task >> (i.e. print result_ID and see what does it contain) ? >> >>> >>> >>> >>> > My bad. Under the old way (insert into table) it was ok to pass things as > strings, and while I caught the function name, I did not notice that my > pvars was a dict hiding in a string. Using a real dict solved the problem. > -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: scheduler question, how to use queue_task?
pargs and pvars are "automatically" json-encoded, but needs to be real python objects. If you want to pass strings already converted use args='...' and vars='...' but the whole point of queue_task is to avoid repetition (e.g. json.dumps(this), json.dumps(that)) On Thursday, March 7, 2013 12:15:16 PM UTC+1, Tim Richardson wrote: > > > > On Thursday, 7 March 2013 21:49:12 UTC+11, Niphlod wrote: >> >> seems perfectly legit in the snippet, maybe there's something else in >> your "complete" code. Can you inspect the errors returned by queue_task >> (i.e. print result_ID and see what does it contain) ? >> >>> >>> >>> >>> > My bad. Under the old way (insert into table) it was ok to pass things as > strings, and while I caught the function name, I did not notice that my > pvars was a dict hiding in a string. Using a real dict solved the problem. > -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: scheduler question, how to use queue_task?
On Thursday, 7 March 2013 21:49:12 UTC+11, Niphlod wrote: > > seems perfectly legit in the snippet, maybe there's something else in your > "complete" code. Can you inspect the errors returned by queue_task (i.e. > print result_ID and see what does it contain) ? > >> >> >> >> My bad. Under the old way (insert into table) it was ok to pass things as strings, and while I caught the function name, I did not notice that my pvars was a dict hiding in a string. Using a real dict solved the problem. -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: scheduler question, how to use queue_task?
I believe I just stumbled on the same problem last night. I was upgrading an app with an old scheduler (1.99.2) to current (2.4.2). I temporarily solved it by continuing to insert the db record manually, as I needed to finish this quickly and had no time to troubleshoot... I'll be able to do further tests tomorrow. Regards, Ales On Thursday, March 7, 2013 11:34:23 AM UTC+1, Tim Richardson wrote: > > I have an app using the scheduler the old-fashioned way. I have a model > which sets up the scheduler. > myscheduler = Scheduler(db,dict(import_task_add=import_task_add)) > > The model also defines the task which the scheduler runs. > > However, the scheduling of a job is done by a controller, after the user > provides some job parameters. > Under the old approach, I start the job by inserting a record in table > db.scheduler_task > > Now, I want to use immediate=True which is not a column in the table; I > have to use scheduler_instance.queue_task > > and then in the controller I try > > result_ID=myscheduler.queue_task(import_task_add, >pvars='...'}, >immediate=True, timeout = 600) > > but it's silently failing. I don't get an entry in the scheduler tables. > Doing it in the controller is the least change starting from my current > code, but is this the wrong approach? > > > > -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: scheduler question, how to use queue_task?
seems perfectly legit in the snippet, maybe there's something else in your "complete" code. Can you inspect the errors returned by queue_task (i.e. print result_ID and see what does it contain) ? On Thursday, March 7, 2013 11:34:23 AM UTC+1, Tim Richardson wrote: > > I have an app using the scheduler the old-fashioned way. I have a model > which sets up the scheduler. > myscheduler = Scheduler(db,dict(import_task_add=import_task_add)) > > The model also defines the task which the scheduler runs. > > However, the scheduling of a job is done by a controller, after the user > provides some job parameters. > Under the old approach, I start the job by inserting a record in table > db.scheduler_task > > Now, I want to use immediate=True which is not a column in the table; I > have to use scheduler_instance.queue_task > > and then in the controller I try > > result_ID=myscheduler.queue_task(import_task_add, >pvars='...'}, >immediate=True, timeout = 600) > > but it's silently failing. I don't get an entry in the scheduler tables. > Doing it in the controller is the least change starting from my current > code, but is this the wrong approach? > > > > -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
