subject:"\[web2py\] Cron\: what happens if a process isn't finished and the next time to start it comes again\?"

[web2py] Cron: what happens if a process isn't finished and the next time to start it comes again?

2010-12-28 Thread Lorin Rivers

I am working on moving some of the computation-intensive processes to a
cron-launched scheme. I have some that run every minute, every 2 minutes,
every 5 minutes and every 15 minutes.

In production, it's possible that the volume of incoming data would result
in the task not finishing before its next cron slot fires again. In
development , however, I can flood the system quite easily by importing data
in larger chunks.

At the moment, my laptop (fairly decent machine) is essentially unusable
because there are so many python processes consuming RAM.

Here's a slightly cleaned-up sample of ps:

 6716 ttys0000:01.70 ../Python web2py.py -a 1234 -i 127.0.0.1 -p 8000
 6722 ttys0000:36.46 ../Python ../web2py.py -J -M -S Debug/DAQ/setup_DAQ
-a recycle
 6724 ttys0000:00.00 (Python)
 7239 ttys0000:00.00 (Python)
 7543 ttys0000:00.00 (Python)
 7641 ttys0000:00.00 (Python)
 7720 ttys0000:00.00 (Python)
 7819 ttys0000:00.00 (Python)
 8781 ttys0000:00.00 (Python)
 8783 ttys0000:00.00 (Python)
 8784 ttys0000:00.00 (Python)
 8787 ttys0000:00.00 (Python)
 8789 ttys0000:00.00 (Python)
 8790 ttys0000:00.00 (Python)
 8791 ttys0000:00.00 (Python)
 8792 ttys0000:00.00 (Python)
 8867 ttys0000:00.00 (Python)
 8868 ttys0000:00.00 (Python)
 8872 ttys0000:00.00 (Python)
 8946 ttys0000:00.00 (Python)
 8947 ttys0000:00.00 (Python)
 8949 ttys0000:00.00 (Python)
 8950 ttys0000:00.00 (Python)
 8951 ttys0000:00.00 (Python)
 8948 ttys0000:00.46 ../Python ../web2py.py -J -M -S
Debug/rounding/array_1minsRollup -a recycle
 7320 ttys0001:04.75 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 7424 ttys0000:56.54 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 7544 ttys0000:50.19 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 7643 ttys0000:42.32 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 7722 ttys0000:36.26 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 7820 ttys0000:30.36 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 7899 ttys0000:25.12 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8013 ttys0000:20.99 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8090 ttys0000:17.28 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8189 ttys0000:14.05 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8271 ttys0000:11.55 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8364 ttys0000:07.50 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8455 ttys0000:07.51 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8557 ttys0000:05.08 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8635 ttys0000:00.73 ../Python ../web2py.py -J -M -S
Debug/rounding/data_tableRollup -a recycle
 8458 ttys0000:18.24 ../Python ../web2py.py -J -M -S
Debug/rounding/panel_1minsRollup -a recycle
 8638 ttys0000:16.68 ../Python ../web2py.py -J -M -S
Debug/rounding/panel_1minsRollup -a recycle
 8786 ttys0000:00.57 ../Python ../web2py.py -J -M -S
Debug/rounding/panel_1minsRollup -a recycle
 7061 ttys0000:43.78 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7140 ttys0000:12.82 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7240 ttys0000:12.35 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7319 ttys0000:12.37 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7423 ttys0000:12.20 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7541 ttys0000:12.10 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7640 ttys0000:12.26 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7721 ttys0000:12.31 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7818 ttys0000:12.59 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 7898 ttys0000:12.53 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 8012 ttys0000:12.46 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 8092 ttys0000:12.46 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 8188 ttys0000:12.26 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 8270 ttys0000:11.96 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 8363 ttys0000:12.05 ../Python ../web2py.py -J -M -S
Debug/rounding/stringdataRollup -a recycle
 8454 ttys0000:12.12 ../Python ../web2py.py -J -M -S

Re: [web2py] Cron: what happens if a process isn't finished and the next time to start it comes again?

2010-12-28 Thread Jonathan Lundell

On Dec 28, 2010, at 1:18 PM, Lorin Rivers wrote:
 I am working on moving some of the computation-intensive processes to a 
 cron-launched scheme. I have some that run every minute, every 2 minutes, 
 every 5 minutes and every 15 minutes.
 
 In production, it's possible that the volume of incoming data would result in 
 the task not finishing before its next cron slot fires again. In development 
 , however, I can flood the system quite easily by importing data in larger 
 chunks.
 
 At the moment, my laptop (fairly decent machine) is essentially unusable 
 because there are so many python processes consuming RAM.

Do you see this log message?

logger.warning('WEB2PY CRON: Stale cron.master detected')

I don't think that there's a good global solution to this kind of problem; cron 
is really oriented toward jobs that can finish in a short time period.

One approach would be to have a daemon process (or a pool of them, maybe one 
per core) to do the work, and in your cron jobs simply queue work for them.

Alternatively (or perhaps in addition) you might want to introduce a little 
back pressure to your clients, so as not to accept more work than your server 
is capable of handling in a reasonable time. 

With the latest version of web2py/Rocket, you can lower the limit on request 
threads (I count 22 below), which would be a brute-force way of getting back 
pressure.

 
 Here's a slightly cleaned-up sample of ps:
 
  6716 ttys0000:01.70 ../Python web2py.py -a 1234 -i 127.0.0.1 -p 8000
  6722 ttys0000:36.46 ../Python ../web2py.py -J -M -S Debug/DAQ/setup_DAQ 
 -a recycle
  6724 ttys0000:00.00 (Python)
  7239 ttys0000:00.00 (Python)
  7543 ttys0000:00.00 (Python)
  7641 ttys0000:00.00 (Python)
  7720 ttys0000:00.00 (Python)
  7819 ttys0000:00.00 (Python)
  8781 ttys0000:00.00 (Python)
  8783 ttys0000:00.00 (Python)
  8784 ttys0000:00.00 (Python)
  8787 ttys0000:00.00 (Python)
  8789 ttys0000:00.00 (Python)
  8790 ttys0000:00.00 (Python)
  8791 ttys0000:00.00 (Python)
  8792 ttys0000:00.00 (Python)
  8867 ttys0000:00.00 (Python)
  8868 ttys0000:00.00 (Python)
  8872 ttys0000:00.00 (Python)
  8946 ttys0000:00.00 (Python)
  8947 ttys0000:00.00 (Python)
  8949 ttys0000:00.00 (Python)
  8950 ttys0000:00.00 (Python)
  8951 ttys0000:00.00 (Python)
  8948 ttys0000:00.46 ../Python ../web2py.py -J -M -S 
 Debug/rounding/array_1minsRollup -a recycle
  7320 ttys0001:04.75 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  7424 ttys0000:56.54 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  7544 ttys0000:50.19 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  7643 ttys0000:42.32 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  7722 ttys0000:36.26 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  7820 ttys0000:30.36 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  7899 ttys0000:25.12 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8013 ttys0000:20.99 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8090 ttys0000:17.28 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8189 ttys0000:14.05 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8271 ttys0000:11.55 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8364 ttys0000:07.50 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8455 ttys0000:07.51 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8557 ttys0000:05.08 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8635 ttys0000:00.73 ../Python ../web2py.py -J -M -S 
 Debug/rounding/data_tableRollup -a recycle
  8458 ttys0000:18.24 ../Python ../web2py.py -J -M -S 
 Debug/rounding/panel_1minsRollup -a recycle
  8638 ttys0000:16.68 ../Python ../web2py.py -J -M -S 
 Debug/rounding/panel_1minsRollup -a recycle
  8786 ttys0000:00.57 ../Python ../web2py.py -J -M -S 
 Debug/rounding/panel_1minsRollup -a recycle
  7061 ttys0000:43.78 ../Python ../web2py.py -J -M -S 
 Debug/rounding/stringdataRollup -a recycle
  7140 ttys0000:12.82 ../Python ../web2py.py -J -M -S 
 Debug/rounding/stringdataRollup -a recycle
  7240 ttys0000:12.35 ../Python ../web2py.py -J -M -S 
 Debug/rounding/stringdataRollup -a recycle
  7319 ttys0000:12.37 ../Python ../web2py.py -J -M -S 
 Debug/rounding/stringdataRollup -a recycle
  7423 ttys0000:12.20 ../Python ../web2py.py -J -M -S 
 Debug/rounding/stringdataRollup -a recycle
  7541 ttys0000:12.10 ../Python ../web2py.py -J -M -S 
 Debug/rounding/stringdataRollup -a recycle
  7640 ttys0000:12.26

[web2py] Cron: what happens if a process isn't finished and the next time to start it comes again?

Re: [web2py] Cron: what happens if a process isn't finished and the next time to start it comes again?

2 matches

Site Navigation

Mail list logo

Footer information