[web2py] Re: Fun with Scheduler
I'll see if I can abstract a simple example.
The concept is... I load a big file into the system and call module1.task-A
to process it. This is an unzip task and it bursts the large file into 50
smaller ones that each need to be scheduled with module2.task-B for
processing. So you go from the model scope to calling something in module1
(I got that part working) when then wants to schedule tasks from module2.
That part didn't work because module2 is not in the context of the
scheduler. I even tried putting a "from module2 import task-B" in the
model file where the scheduler was created, but that had no effect.
I think Paolo's idea might help. I could create a "stub" for each of the
module functions in the model's context and avoid the issue.
On Wednesday, June 5, 2013 9:21:18 AM UTC-7, Niphlod wrote:
>
> uhm. inserting things like this can end up with undesired side-effects
> (i.e. inserting args "printed as a list"). First things first, you should
> leverage the queue_task API.
> As for "let's try to figure out what function name will be valid", please
> test the functionality before "by hand" and then adjust the code
> accordingly.
> If you schedule a function named "abcd" and that's not on globals, it
> won't be executed, cause basically the scheduler does
>
> function = getfromglobals(name)
> result = function(args, vars)
>
> If you can package a MINIMAL application to show what you need maybe we
> can figure out a general decorator that will be included in the next
> scheduler release :P
>
> Il giorno mercoledì 5 giugno 2013 12:57:40 UTC+2, Joe Barnhart ha scritto:
>>
>> I think the web2py scheduler is the most amazing thing ever added to the
>> platform. It is truly a wonder in that it works so well and has such a
>> clean design.
>>
>> I was trying to improve it in the most minor way -- I wanted to create a
>> decorator that causes a function to be submitted to the scheduler instead
>> of being executed directly. That way I could easily switch from background
>> to foreground processing by commenting out the decorator, or add new
>> functions to the scheduler by adding the decorator. Here's what I came up
>> with:
>>
>>
>> def scheduled(name="Scheduled",timeout=60):
>> def wrap(f):
>> def new_f(*a):
>> current.db.scheduler_task.insert(
>> task_name=name,
>> function_name=f.__name__,
>> args='"%s"'%str(list(a)),
>> timeout=timeout)
>> return new_f
>> return wrap
>>
>>
>> In this decorator, I only passed two variables to the scheduler --
>> task_name and timeout. The heck of it is, I think the decorator is working
>> except I'm running into a conceptual problem with the scheduler in
>> general. Using it I can create a new scheduled task:
>>
>>
>> @scheduled(name='import_times')
>> def import_cl2times(fid):
>> from ssobjects import SsMessage
>> from ssimporter import SsImportTimes
>> if fid:
>> fname=current.db.fileobject(fid).get('filename')
>> SsImportTimes.cl2read(fid)
>> msg=SsMessage(subject="Meet results",message="File import of
>> meet results %s is complete."%fname)
>> msg.send()
>> return
>>
>>
>> This code lives in a module and is called by the scheduler. Or maybe
>> isn't. I seem to be having module/controller/model difficulty at the
>> moment.
>>
>> I have much of my functionality in modules, to permit the speed of
>> compiled code in my rather large and growing website. I would like to call
>> the decorator from either the context of the models, or from tasks already
>> running in the background, in a module. Yes, I have background tasks which
>> schedule other background tasks. Have I mentioned how much I love the
>> scheduler??!?
>>
>> I can't seem to get the various imports right to have the scheduler
>> actually be able to run a bit of code in a module. i had it working at one
>> point when the scheduler and all of its background tasks were in the model
>> file. But I couldn't schedule tasks from a module since the module can't
>> see the model's context. But now that everything is in modules, the
>> scheduler can't seem to find the code to run it.
>>
>> Any help is appreciated.
>>
>>
>>
--
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Fun with Scheduler
uhm. inserting things like this can end up with undesired side-effects
(i.e. inserting args "printed as a list"). First things first, you should
leverage the queue_task API.
As for "let's try to figure out what function name will be valid", please
test the functionality before "by hand" and then adjust the code
accordingly.
If you schedule a function named "abcd" and that's not on globals, it won't
be executed, cause basically the scheduler does
function = getfromglobals(name)
result = function(args, vars)
If you can package a MINIMAL application to show what you need maybe we can
figure out a general decorator that will be included in the next scheduler
release :P
Il giorno mercoledì 5 giugno 2013 12:57:40 UTC+2, Joe Barnhart ha scritto:
>
> I think the web2py scheduler is the most amazing thing ever added to the
> platform. It is truly a wonder in that it works so well and has such a
> clean design.
>
> I was trying to improve it in the most minor way -- I wanted to create a
> decorator that causes a function to be submitted to the scheduler instead
> of being executed directly. That way I could easily switch from background
> to foreground processing by commenting out the decorator, or add new
> functions to the scheduler by adding the decorator. Here's what I came up
> with:
>
>
> def scheduled(name="Scheduled",timeout=60):
> def wrap(f):
> def new_f(*a):
> current.db.scheduler_task.insert(
> task_name=name,
> function_name=f.__name__,
> args='"%s"'%str(list(a)),
> timeout=timeout)
> return new_f
> return wrap
>
>
> In this decorator, I only passed two variables to the scheduler --
> task_name and timeout. The heck of it is, I think the decorator is working
> except I'm running into a conceptual problem with the scheduler in
> general. Using it I can create a new scheduled task:
>
>
> @scheduled(name='import_times')
> def import_cl2times(fid):
> from ssobjects import SsMessage
> from ssimporter import SsImportTimes
> if fid:
> fname=current.db.fileobject(fid).get('filename')
> SsImportTimes.cl2read(fid)
> msg=SsMessage(subject="Meet results",message="File import of meet
> results %s is complete."%fname)
> msg.send()
> return
>
>
> This code lives in a module and is called by the scheduler. Or maybe
> isn't. I seem to be having module/controller/model difficulty at the
> moment.
>
> I have much of my functionality in modules, to permit the speed of
> compiled code in my rather large and growing website. I would like to call
> the decorator from either the context of the models, or from tasks already
> running in the background, in a module. Yes, I have background tasks which
> schedule other background tasks. Have I mentioned how much I love the
> scheduler??!?
>
> I can't seem to get the various imports right to have the scheduler
> actually be able to run a bit of code in a module. i had it working at one
> point when the scheduler and all of its background tasks were in the model
> file. But I couldn't schedule tasks from a module since the module can't
> see the model's context. But now that everything is in modules, the
> scheduler can't seem to find the code to run it.
>
> Any help is appreciated.
>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
[web2py] Re: Fun with Scheduler
I agree Scheduler is great. I use this technique: in modules/lib.py: def a_func(db): # some code # ... return in models/scheduler.py def a_func(): import lib return lib.a_func(db) from gluon.scheduler import Scheduler scheduler = Scheduler(db) in controllers/a_controller.py def a_func: import lib return lib.a_func(db) so the code is centralized in modules/lib.py and i can call the same code from controller or scheduled task. PB -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
