On 30 Nov 2012, at 9:38 AM, Magnitus <eric_vallee2...@yahoo.ca> wrote:
> What you are suggesting addresses the concurrency access problem (which could
> be resolved with the DB since it locks or with your suggestion).
>
> However, if you stick the logic in your model, you still have the problem of
> the logic appearing on every request.
>
> While you do speed up the checking with the "Test & set" instead of DB
> access, eliminating the disk read, you do burden your model with additional
> code bloat.
>
> I suppose that could be mitigated by inserting your script logic into a
> module and passing the db as an argument to it.
The overhead to check global_settings is negligibly more than an ordinary dict
access, and except for the first request, that's all you'll incur. I'm pretty
sure that's quite a bit less than an import of a module that's already
imported (since it still has to check whether the module is already there).
It's true that the db code will still be there, but it won't be executed.
>
> On Thursday, 29 November 2012 14:00:05 UTC-5, Jonathan Lundell wrote:
> On 29 Nov 2012, at 10:22 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>> >There are a couple of issues. One is: what do you mean by startup? It's
>> >fairly well defined if you're running web2py with Rocket as the server, but
>> >in a wsgi environment, especially one that might have multiple processes,
>> >it's a lot less clear.
>>
>> Yes and no.
>>
>> Obviously, a task that needs to run at startup would have to be robust,
>> because you might start and stop your server multiple times (for example if
>> it creates groups, it would have to check if the groups already exist before
>> creating them).
>>
>> In that context, it isn't too much of a problem if it is executed multiple
>> times if wsgi spawns multiple processes given that it is an overhead only
>> when the server starts.
>>
>> However, if you absolutely need to make sure that the task itself runs only
>> once even with multiple processes spawning, it seems like such
>> synchronization could be handled in the DB (either as a feature of the
>> framework or by the scripter himself, it's not too big of an overhead) or
>> just by using another interprocess synchronisation mechanism that modern
>> OSes provide.
>>
>> > So: what are you really trying to do? In what way is it a problem to run
>> > on the first request after startup (whatever startup is) rather than at
>> > startup itself? If you're initializing the database, can you check that
>> > it's already initialized? (Yes, overhead, but would it work logically?)
>>
>> I'm sure it's not THAT big of a performance overhead, but if possible, I
>> prefer that tasks that only need to be executed at startup be only executed
>> at startup.
>>
>> Not only does it run faster, but intuitively, I consider it to be better
>> design as it modularize your code more since it doesn't mix up tasks that
>> need to be executed at startup only with tasks that need to be executed with
>> every request.
>
> How about something along these lines: set a flag in
> gluon.settings.global_settings, using an app-unique key (because
> global_settings should but doesn't have a per-application namespace you want
> something like 'com.yourdomain.yourapp' or the like). Test & set the flag
> early in your model, and do your db init conditional on the model not being
> set, using db semantics to do it once, since you could have updates from
> multiple processes. A straightforward mechanism (I think?) would be to just
> do an insert with some appropriate field marked unique, and ignore
> insert-collision exceptions.
>
>>
>> On Thursday, 29 November 2012 10:44:17 UTC-5, Jonathan Lundell wrote:
>> On 29 Nov 2012, at 6:41 AM, Magnitus <eric_va...@yahoo.ca> wrote:
>>> I'd be curious to know if the lack of feedback on this thread so far is due
>>> to the devs being incredibly busy or web2py lacking a genuine working
>>> mechanism to execute scripts that have access to the models at startup only.
>>
>> I've tried to do something like this myself (clear memcached on startup, in
>> my case) and couldn't figure out a good way.
>>
>> There are a couple of issues. One is: what do you mean by startup? It's
>> fairly well defined if you're running web2py with Rocket as the server, but
>> in a wsgi environment, especially one that might have multiple processes,
>> it's a lot less clear.
>>
>> Another is that the built-in cron mechanism was badly broken. ISTR seeing
>> some fixes from Massimo go by recently, but I didn't look closely at what
>> was going on.
>>
>> So: what are you really trying to do? In what way is it a problem to run on
>> the first request after startup (whatever startup is) rather than at startup
>> itself? If you're initializing the database, can you check that it's already
>> initialized? (Yes, overhead, but would it work logically?)
>>
>>>
>>> I've tried a lot of permutations for setting a script at the start with
>>> cron and I'm now convinced that the functionality is either broken or so
>>> obscure that most non-devs users that rely on the documentation alone won't
>>> be able to make it work (at least, not without a major head ache).
>>>
>>> Besides, it seems from the documentation that cron is being phased out
>>> anyways, so I'm thinking it would be nice to provide some sort of hook for
>>> users to insert a script at startup with access to the models instead.
>>>
>>> Inserting "from gluon import *" in the module won't work either since the
>>> db handles are not defined in gluon, but in the models.
>>>
>>> I'm guessing that maybe the entire database definition could be moved to a
>>> module instead (I'd have to reflect on all the implications of this though
>>> it would definitely increase the performance at the cost of having to
>>> reboot when you want to modify the db's structure), but that's not the
>>> default out of the box architecture.
>>>
>>> Either way, inserting the script into a module that is included in a
>>> controller is not a 100% satisfying solution since the script is actually
>>> executed on the first page request and not really at startup.
>>>
>>> On Wednesday, 28 November 2012 12:55:00 UTC-5, Magnitus wrote:
>>> After some reflection, I suppose I could write a wrapper script that first
>>> runs python web2py.py --import_models --shell=<MyController>/default
>>> --run="<My Script>" and then starts web2py normally when using web2py
>>> standalone.
>>>
>>> That is one workaround.
>>>
>>> On Wednesday, 28 November 2012 11:19:01 UTC-5, Magnitus wrote:
>>> Hi gents,
>>>
>>> I have been populating my databases with groups (for access control)
>>> manually for a while and then, switched to running a script that does it,
>>> though I still run the script manually in the web2py context.
>>>
>>> Now, I'd like to programmatically implement this in a sensible way (in a
>>> way that runs by itself and doesn't need human intervention at all).
>>>
>>> Obviously, you could just put the logic that does this in a model, but I
>>> don't like the idea of needlessly increasing the size of my models with
>>> this or running the logic to check the existance of and generate groups for
>>> each request.
>>>
>>> Another alternative would be to put the logic in a module that would be
>>> imported (with the idea that it would run on the first import which is done
>>> only once), but then imported modules don't have access to the web2py
>>> objects (notably, my database handle) unless you pass them on as parameters
>>> which you cannot do in a module's global scope when it is first included.
>>>
>>> This leaves me with cron and running it as a cron job at boot time.
>>> However, it isn't working.
>>>
>>> I tried running my script using the web2py context (ie, python web2py.py
>>> --import_models --shell=<MyController>/default --run="<My Script>") and it
>>> ran fine and generated the db entries I wanted.
>>>
>>> Then, I started my web2py execution with cron activated (ie, python
>>> web2py.py -Y).
>>>
>>> I tried the two varations portayed in the example for the crontab ("@reboot
>>> * * * * root *applications/<My App>/cron/Generate_groups.py" and the
>>> shorter "@reboot root *applications/<My App>/cron/Generate_groups.py).
>>>
>>> And yes, my db transactions end with a db.commit().
>>>
>>> I've been banging my head against this for two hours and while I'm sure
>>> whatever is wrong with this will come to me over time, this is definitely a
>>> case of sooner would be better than later.
>>>
>>> Insights would be appreciated.
>>>
>>> --
>>>
>>>
>>>
>>
>>
>>
>> --
>>
>>
>>
>
>
>
> --
>
>
>
--
