John, I did not mean fight about principles or dismiss your suggestion. I perfectly understand what you need and see that rationale in it. It's just that if I'm doing an overhaul, I'd like to do it right - leaving to the users to 'behave' and not shoot themselves in the foot is something I'd like to avoid. Exec is not as easy as it sounds as cron operates really really low, in a separate thread on the level just above WSGI/webserver. I have access to very little of web2py's features 'down there' so things get tricky real fast (especially if you want to include models in the story) - that's why calling a new web2py process to sort or controller/model issues was a simple way of doing it in a foolproof manner.
On Feb 9, 3:14 am, John Heenan <johnmhee...@gmail.com> wrote: > Since AchipA is working on other changes to cron.py and cron.py is > his, I think I should wait to see if AchipA includes additons with > Massimo's preferred format below before being so bold as to submit a > patch myself. > > John Heenan > > On Feb 9, 10:54 am, mdipierro <mdipie...@cs.depaul.edu> wrote: > > > I have no objection to adding this feature. I do not like the !*" > > prefix. > > > I think we can do: > > > @hourly root file -> exec file in Popen > > @hourly root file.py -> exec file in Popen with python > > @hourly root *file.py -> exec file in Popen with python -S <app> -M -R > > <file> > > @hourly root **file.py -> exec file in Popen with python -S <app> -R > > <file> > > @hourly root *file -> exec file in Popen with python -S <file> -M > > @hourly root **file -> exec file in Popen with python -S <file> > > @hourly root *"data" -> exec "data" without Popen with environment as - > > M > > @hourly root **"data" -> exec "data" without Popen without environment > > as -M > > @hourly root *"filename" -> execfile filename without Popen with > > environment as -M > > @hourly root **"filename" -> execfile filename without Popen without > > environment as -M > > (we can distinguish data from filename checking if "..." starts with > > './' or '/'.) > > > I would take a patch in this direction. You only add the last 4 > > options at the end of the cron file. There is nothing else that needs > > to be done I think. > > > Massimo > > > All is needed it a > > > On Feb 8, 6:30 pm, John Heenan <johnmhee...@gmail.com> wrote: > > > > Hi > > > > I stated "My point is can we have an option to:" > > > > Maybe I should code up a patch for cron.py and submit it for > > > examination. Cron.py is impressive, I am not criticising it or > > > attacking its approach. > > > > The option I am requesting are ADDITIONS to cron.py that are TRIVIAL > > > to code in as additions to cron.py. Web2py already has a prepared > > > environment ready to be passed for exec use. > > > > To reiterate, the option I am requesting is an additional option (not > > > a replacement) to do an exec (or execfile) on a file that ends > > > with .py (or .pyc) AND an option to do an exec on a string (such as > > > "import a; a.mycronaction()") > > > > For example, suppose ! is chosen to indicate not to use the default of > > > the Popen class wrapper but to use exec instead, then the following is > > > an example of use: > > > > */10 * * * * root ! > > > **expire_sessions.py #use the web2py environment with execfile on > > > the .py file but do not execute models > > > or > > > */10 * * * * root !*"import a; > > > a.mycronaction()" #use the web2py environment with exec on the string > > > and include execution of models > > > > I do not want to become involved in a grand philosophical or > > > ideological debate about the merits or otherwise of using the Popen > > > wrapper class verses the internal Python exec (or execfile) function. > > > For trivial tasks (such as deleting stale session files) the Popen > > > class wrapper is overkill. Using Popen involves unnecessary extra RAM > > > use and process invocation. Invoking a Python process is very > > > expensive in CPU and RAM terms. In fact Python is impractical in a web > > > environment unless the Python process remains persistent, This is the > > > reason why hosting is relatively expensive for python web solutions > > > compared to PHP and Perl. > > > > While such concerns might seem trivial for tasks that are not repeated > > > often, it is not trivial if hosting is in an environment where there > > > are penalties for exceeding allocated RAM, such as I believe occurs at > > > Webfaction, popular with those trialling web2py. > > > > For trivial housekeeping tasks, just a minimal (non models) > > > environment associated with the application is adequate, such as is > > > currently passed in for use with Popen. > > > > If tasks are so involved that processes are at a non negligible risk > > > of choking then users will hardly be concerned about using execfile > > > instead of Popen and will instead code up an independent supervisory > > > process outside web2py. > > > > Deadlock or race conditions are not any more of an issue using exec > > > than with normal code (provided the code passed in for exec does > > > nothing out of the ordinary). A new thread is not invoked. > > > > As is, cron.py waits for a Popen invoked process to end without a > > > supervisor which can result in a lock up of web2py. > > > > It can be argued that there is a lack of hard quantified information > > > about the resources used by Python based web frameworks. Those who > > > develop websites and GUIs have frequently little care about boring > > > issues of resource usage. I bothered to take a look and was horrified > > > by the memory usage of Python. We need to show respect for practical > > > issues associated with using Python. Otherwise web2py and other > > > frameworks risk becoming relegated to academic toys that are from > > > 'enterprise' class. Java based frameworks are fighting back. Sun is > > > pushing GlassFish heavily into the enterprise environment. > > > > Python based frameworks cannot expect to be competitive with PHP at > > > what PHP is best at (low resource usage) if Python frameworks don't > > > address issues in a practical way that are holding back their adoption > > > (such as resource usage leading to expensive hosting). > > > > John Heenan > > > > On Feb 8, 11:40 pm, AchipA <attila.cs...@gmail.com> wrote: > > > > > Fiddling with this now, but have a few concerns, so I'd like Massimo > > > > to chime is as the exec expert. The main reason for going POpen is to > > > > have a clean, thread safe environment, and with exec I'm not sure we > > > > won't hit race conditions or deadlock possibilities, especially if we > > > > touch on models. Some help how to exec stuff with the proper > > > > environment would be welcome :) > > > > > Another thing to note is that when execing, we no longer have control > > > > over what we execute - i.e. while we can kill/clean zombie POpened > > > > processes, if we get stuck in just another thread (=with exec) it's > > > > stuck there until the whole web2py/webserver is restarted. > > > > > On Jan 31, 7:27 am, John Heenan <johnmhee...@gmail.com> wrote: > > > > > > I have not get my point across. > > > > > > By 'Python exec' I mean using the 'exec' internal Python command. I > > > > > don't mean 'Python exec' as a shorthand for 'execute a new Unix, > > > > > Windows or Mac OS process under the ownership of the cron code written > > > > > in Python'. > > > > > > I have just examined the most recent trunk version of cron.py > > > > > inhttp://code.google.com/p/web2py/source/browse/gluon/contrib/cron.py > > > > > > For .py files the Popen command is still executed using the > > > > > cronlauncher class that uses Popen > > > > > > For those of us trying ot keep RAM usage down, who don't want to use a > > > > > regular crontab and who only want to run Python files, using Popen is > > > > > not necessasry. > > > > > > My point is can we have an option to: > > > > > > 1) Just do an Python exec on an opened file contents (with ot without > > > > > the models envrionment), instead of doing a Popen > > > > > 2) Have an option to do an Python exec on a string, such as on "import > > > > > a; a.mycronaction()" > > > > > > Thanks > > > > > > John Heenan > > > > > > On Jan 31, 2:17 pm, mdipierro <mdipie...@cs.depaul.edu> wrote: > > > > > > > Unless we misunderstand we have that already. The file just needs to > > > > > > end in .py as in > > > > > > > * * * * * root myscript.py > > > > > > > You can also call actions > > > > > > > * * * * * root *myscript.py > > > > > > > so that models are imported. > > > > > > > On Jan 30, 8:36 pm, John Heenan <johnmhee...@gmail.com> wrote: > > > > > > > > Hi AchipA > > > > > > > > Another feature request, if the cron file is a python file, how > > > > > > > about > > > > > > > a cron option to open the file and do an exec on the file > > > > > > > contents? > > > > > > > > Thanks > > > > > > > > John Heenan > > > > > > > > On Jan 31, 12:33 pm, John Heenan <johnmhee...@gmail.com> wrote: > > > > > > > > > Hi AchipA > > > > > > > > > Your cron is nice work and neatly deals with the messy side of > > > > > > > > launching an independent OS process and waiting for it to > > > > > > > > finish. > > > > > > > > > For those of us who do not wish to launch an independent > > > > > > > > process (to > > > > > > > > keep down memory use) and are happy to stick with Python for > > > > > > > > cron > > > > > > > > jobs, following is a request to add in the following feature to > > > > > > > > your > > > > > > > > cron fpr web2py. > > > > > > > > > Provide a mechanism to just perform a Python exec on a string > > > > > > > > statment. > > > > > > > > > For example a string statement might be > > > > > > > > "import a; a.mycronaction()" > > > > > > > > > Thanks > > > > > > > > > John Heenan > > > > > > > > > On Jan 31, 6:52 am, AchipA <attila.cs...@gmail.com> wrote: > > > > > > > > > > Just to chime in, I'm still alive and preparing an update to > > > > > > > > > cron as > > > > > > > > > quite a few things changed since I last touched it. As for the > > > > > > > > > questions - all cron modes support the * and ** syntax, and > > > > > > > > > yes, cron > > > > > > > > > files can be modified on the fly. > > > > > > > > > > On Jan 26, 4:00 pm, mdipierro <mdipie...@cs.depaul.edu> wrote: > > > > > > > > > > > I did not write cron personally and I am still trying to > > > > > > > > > > understand > > > > > > > > > > some of it, Attila did. > > > > > > > > > > I am in the looking at cron these days to treamline it a > > > > > > > > > > bit and I am > > > > > > > > > > trying to understand better as it works. > > > > > > > > > > > Since windows does not have cron, you should use web2py > > > > > > > > > > hardcron. It > > > > > > > > > > should work on windows 7 but I have not tried it. The > > > > > > > > > > "root" is the > > > > > > > > > > username that should run the cron task. It is only used by > > > > > > > > > > extcron and > > > > > > > > > > ignored otherwise (but must be there, for compatibility of > > > > > > > > > > syntax). > > > > > > > > > > > I am not sure if tasks staring with * or ** run under > > > > > > > > > > extcron so I > > > > > > > > > > suggest not using pycron. Given hardcron you do not need > > > > > > > > > > pycron > > > > > > > > > > anyway. > > ... > > read more » -- You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to web...@googlegroups.com. To unsubscribe from this group, send email to web2py+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/web2py?hl=en.
