Roy Smith wrote:
> [EMAIL PROTECTED] wrote:
>
>
>>Hello,
>>
>>I write a lot of CGI scripts, in Python of course. Now I need to
>>convert some to long-running processes. I'm having trouble finding
>>resources about the best practices to do that.
>>
>>I've found a lot of email discussions that say something like, "You
>>need to educate yourself about the differences when you have long-
>>running processes" but I've not had a lot of luck with finding things
>>that explain the differences.
>
>
> The biggest differences between run-and-exit vs. long running processes are
> resource management and error recovery. Let's take them one at a time.
>
> Resource management. In a short-lived process, you really don't have to
> worry about this at all. Snarf as much memory as you need, open as many
> files as you want, and when you exit, the operating system cleans it all up
> for you. With a long running process, you have to worry about stuff like
> that.
>
> In Python, you're isolate from the low-level details of memory management,
> but still need to think about it a bit. Imagine you had code that looked
> like this in your main loop:
>
> for request in getNextRequest():
> requestList.append (request)
> processRequest(request)
>
> requestList is going to keep growing without bounds and eventually will eat
> up all available memory in the system and your process will crash.
> Everything you store, you also need to delete when you're done with it.
In particular, it is a good idea to call gc.collect() every now
and then, specially if you are in such a loop. I don't know what is the
gc policy in python, but an application of mine that seemed to eat as much
memory as it was available was reduced to a constant small amount of
memory after I started to call the gc directly.
> The other big thing is error recovery. In a short lived process, if
> something fails, you print an error message and exit. In a long running
> process, you need to somehow recover from the error and keep going as best
> you can. This can be tricky.
You should have your main loop inside a try/except, to catch any
exceptions that were not otherwise caught without exiting the application.
Log the exceptions, of course, but in most long running applications
work in a loop like Roy Smith's code above, so if one of them fails it
won't disrupt the others to come.
--
Bruno Barberi Gnecco <brunobg_at_users.sourceforge.net>
The earth is like a tiny grain of sand, only much, much heavier.
--
http://mail.python.org/mailman/listinfo/python-list
