Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
Hi again, After lots of testing I can say: It helped. I do still not exactly know why forking failed, but it seems to be somehow related to the high number of processes. I also do not understand why this is a *fatal* error, since it could easily be handled by returning -1 (like in the PHP docs). First I wanted to create a queue for tasks; instead of forking directly only a limited number of processes should be run from the queue, when one finishes another should start. But I decided not to do this, because the queue can easily grow to reach the memory limit. At last I decided to simply pause the script just before forking, if a maximum number of processes has been reached, until one has finished. Not optimal, since the parent process has a higher priority task, but at least it's stable now. To me it appears to be impossible to track if it was the maximum number of processes or a lack of any system resource. However, thank you all for your help. - Oz Robin Vickery wrote: On 02/05/06, Oz [EMAIL PROTECTED] wrote: Hi, My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) The 11 is from errno.h and it corresponds to EAGAIN. From the docs for fork() The fork() function shall fail if: [EAGAIN] The system lacked the necessary resources to create another process, or the system-imposed limit on the total number of processes under execution system-wide or by a single user {CHILD_MAX} would be exceeded. Does that help? -robin -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
Oz wrote: First I wanted to create a queue for tasks; instead of forking directly only a limited number of processes should be run from the queue, when one finishes another should start. But I decided not to do this, because the queue can easily grow to reach the memory limit. If the queue of processes can grow to any reasonable memory limit (lets say 100 bytes/entry, 32 Mb memory limit) that means that you've got 320,000 processes to run? If each one takes a minute to run that's 9 months worth of running ... if 1 a second it's almost 4 days... if you launch PROCESS_MAX processes your system will page and you'll be running VERY slowly ... Perhaps it's time to examine the problem and think about a different solution. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
On Thu, May 4, 2006 3:45 pm, Oz wrote: After lots of testing I can say: It helped. I do still not exactly know why forking failed, but it seems to be somehow related to the high number of processes. I'm pretty confident that it is directly related to the high number of processes. Your Operating System has a compiled-in (or possibly configurable in /etc/ somewhere) limit on the number of processes to allow. There are very good reasons for this limit, however, not least of which is catching malicious and/or run-away infinite fork code. (aka fork-bomb) You might be able to tweak the limit (much) higher. You may even be able to remove the limit, in principle, and just suffer the consequences if you fork-bomb yourself, and crash the machine. There may even be probably N-tier fail-safe roll-over applications where this is the right answer, as weird as that seems. I also do not understand why this is a *fatal* error, since it could easily be handled by returning -1 (like in the PHP docs). You can (and probably should) use http://php.net/set_error_handler to catch the E_FATAL and do whatever you think is appropriate. For most cases, a failed fork probably should be E_FATAL, though, so that's unlikely to change. First I wanted to create a queue for tasks; instead of forking directly only a limited number of processes should be run from the queue, when one finishes another should start. But I decided not to do this, because the queue can easily grow to reach the memory limit. At last I decided to simply pause the script just before forking, if a maximum number of processes has been reached, until one has finished. Not optimal, since the parent process has a higher priority task, but at least it's stable now. I'm not seeing how this solves the problem of the queue overflowing... Or perhaps the parent process is getting its tasks from somewhere to build the queue?... You MAY want to consider doing away with this controlling process that queues up tasks, and simply have some kind of id on each source item that is NULL for unprocessed and set the id to the PHP 'pid' when the item is grabbed by a child to be processed. http://php.net/getmypid In other words, instead of the parent getting the data and queueing it up, just have the child get the data and mark it (in a single get/mark operation) as in process It's only one less process, but this arrangement generally makes the application simpler. Depends on your application, of course, so it's just an idea to consider. To me it appears to be impossible to track if it was the maximum number of processes or a lack of any system resource. I think the resource in question is basically your computer rather than a specific resource like open file handlers. You could test this rather quickly by writing an infinite fork process, and seeing at what point it gives you Error 11. If it's around the same number of processes, you know that's it. ps aux | grep yourscriptnamehere -c will (I think) give you a count on how many child processes are running, or you could hack some kind of minimal counter in the fork. -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
On Tue, May 2, 2006 3:57 pm, Jochem Maas wrote: Richard Lynch wrote: I believe that if you installed my perror extension: http://l-i-e.com/perror/ heh cool - I saw your posts to internals regarding trying to write an extension, seems you got a result :-) Yeah. An intermediate result, but a result nonetheless. I was hoping some folks would install it and test it, especially on Windows. Anybody who actually compiles PHP source for Windows that needs a 10-minute break?... My next step, in my copious spare time, is to re-write that as a patch against CVS head, and submit it for PHP 6. Well, okay, at my speed, maybe PHP 7. Definitely by PHP 8. -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
Richard Lynch wrote: On Wed, May 3, 2006 12:51 am, Oz wrote: Richard Lynch wrote: On Tue, May 2, 2006 11:29 am, Oz wrote: My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) I believe that if you installed my perror extension: http://l-i-e.com/perror/ you would be able to easily print out the error message corresponding to 11. I can't it's cursed. I suppose you could be old-school and just use a shell with: perror 11 but that is not nearly as much fun. That puts something out, but where is the realtion? Doesn't help. H. It IS a rather terse message... Allow me to translate, rather freely, into plain English. bash: System error: 11 = Resource deadlock avoided Oh, on my system it's: OS error code 11: Resource temporarily unavailable English: Your script was about to fork, and I, the Operating System, realized that if I allowed that, you would create a deadlock condition in resource management. What that means is that you would have the original process, and the forked child process, BOTH needing access to a single resource. Since it would be impossible to give them both access to that resource, I simply had no choice but to fail to fulfill your request to fork, because it would have created a Resource deadlock. Therefore, I did not fork, and gave you a rather cryptic error message: System error: 11 = Resource deadlock avoided Sorry. --- Hopefully you now understand what is wrong, on some level of understanding. What resource is required by both parent and child process, we don't know, since we have no idea what your script is doing, much less what resources it might be needing to do what it is doing. I could have made a mistake with a file pointer. You could try several things at this point: #1. Attempt to simplify your script to do nothing BUT fork. Then start adding in bits and pieces of your script until you figure out what resource is in deadlock. Then figure out how to avoid the deadlock, by cloning/duplicating that resource. #1A. Rip bits and pieces OUT of your current script, until it forks okay. #2. Put your script up on http://pastebin.com (or similar) and ask experts to look at it and help you. #3. I vaguely recall that you had many many many forks going at once maybe?... Or was that another guy? If you have TOO many forks, don't do that is the answer. Your machine has finite resources. You tried to run too many processes. Don't. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
On Wed, May 3, 2006 12:51 am, Oz wrote: Richard Lynch wrote: On Tue, May 2, 2006 11:29 am, Oz wrote: My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) I believe that if you installed my perror extension: http://l-i-e.com/perror/ you would be able to easily print out the error message corresponding to 11. I can't it's cursed. I suppose you could be old-school and just use a shell with: perror 11 but that is not nearly as much fun. That puts something out, but where is the realtion? Doesn't help. H. It IS a rather terse message... Allow me to translate, rather freely, into plain English. bash: System error: 11 = Resource deadlock avoided English: Your script was about to fork, and I, the Operating System, realized that if I allowed that, you would create a deadlock condition in resource management. What that means is that you would have the original process, and the forked child process, BOTH needing access to a single resource. Since it would be impossible to give them both access to that resource, I simply had no choice but to fail to fulfill your request to fork, because it would have created a Resource deadlock. Therefore, I did not fork, and gave you a rather cryptic error message: System error: 11 = Resource deadlock avoided Sorry. --- Hopefully you now understand what is wrong, on some level of understanding. What resource is required by both parent and child process, we don't know, since we have no idea what your script is doing, much less what resources it might be needing to do what it is doing. You could try several things at this point: #1. Attempt to simplify your script to do nothing BUT fork. Then start adding in bits and pieces of your script until you figure out what resource is in deadlock. Then figure out how to avoid the deadlock, by cloning/duplicating that resource. #1A. Rip bits and pieces OUT of your current script, until it forks okay. #2. Put your script up on http://pastebin.com (or similar) and ask experts to look at it and help you. #3. I vaguely recall that you had many many many forks going at once maybe?... Or was that another guy? If you have TOO many forks, don't do that is the answer. Your machine has finite resources. You tried to run too many processes. Don't. -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
On 02/05/06, Oz [EMAIL PROTECTED] wrote: Hi, My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) The 11 is from errno.h and it corresponds to EAGAIN. From the docs for fork() The fork() function shall fail if: [EAGAIN] The system lacked the necessary resources to create another process, or the system-imposed limit on the total number of processes under execution system-wide or by a single user {CHILD_MAX} would be exceeded. Does that help? -robin -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
On Tue, May 2, 2006 11:29 am, Oz wrote: My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) I believe that if you installed my perror extension: http://l-i-e.com/perror/ you would be able to easily print out the error message corresponding to 11. I suppose you could be old-school and just use a shell with: perror 11 but that is not nearly as much fun. PS My amp goes to 11. -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
Richard Lynch wrote: On Tue, May 2, 2006 11:29 am, Oz wrote: My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) I believe that if you installed my perror extension: http://l-i-e.com/perror/ heh cool - I saw your posts to internals regarding trying to write an extension, seems you got a result :-) you would be able to easily print out the error message corresponding to 11. I suppose you could be old-school and just use a shell with: perror 11 but that is not nearly as much fun. ah but you just did teach *me* something new :-) PS My amp goes to 11. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] What means Fatal error: pcntl_fork(): Error 11?
[snip] My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) [/snip] It means Cut and paste this into Google and you will see entries concerning this error. I could tell you the answer, but then I'd have to kill you. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
Jay Blanchard wrote: [snip] My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) [/snip] It means Cut and paste this into Google and you will see entries concerning this error. No, nice try. I could tell you the answer, but then I'd have to kill you. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] What means Fatal error: pcntl_fork(): Error 11?
Richard Lynch wrote: On Tue, May 2, 2006 11:29 am, Oz wrote: My script dies with the following line (after a few hours running without other errors/warnings/notices): Fatal error: pcntl_fork(): Error 11 in /home/foo/bar.php on line 297 Is there any documentation for pcntl_fork(): Error 11? Does anybody know what causes pcntl_fork(): Error 11? (It's PHP5 and the scripts source is top secret.) I believe that if you installed my perror extension: http://l-i-e.com/perror/ you would be able to easily print out the error message corresponding to 11. I can't it's cursed. I suppose you could be old-school and just use a shell with: perror 11 but that is not nearly as much fun. That puts something out, but where is the realtion? Doesn't help. PS My amp goes to 11. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php