Re: subprocess + python-daemon - bug/problem?
On Sep 8, 5:19 pm, Ben Finney ben+pyt...@benfinney.id.au wrote:
Sewar xsew...@gmail.com writes:
I looked at other daemon libraries and snippets, it's clearly the bug is in
subprocess not python-daemon.
Then I found Python bug #1731717 which discusses it.
I'm running python-2.6.2 which supposedly has the fix for #1731717.
However I still still the problem with subprocess after daemonizing.
I've narrowed it down to just the setting of the SIGCLD signal.
You can reproduce the problem thus (in 2.4.6, 2.5.4 and 2.6.2):
import subprocess, signal
signal.signal(signal.SIGCLD, signal.SIG_IGN)
subprocess.Popen(['echo','foo']).wait()
This patch allows python-daemon to work with subprocess:
--- daemon.py (revision 146)
+++ daemon.py (working copy)
@@ -140,8 +140,6 @@
actually defined in the ``signal`` module will appear in
the
default map:
-* ``signal.SIGCLD``: ``None``
-
* ``signal.SIGTTIN``: ``None``
* ``signal.SIGTTOU``: ``None``
@@ -730,7 +728,6 @@
name_map = {
-'SIGCLD': None,
'SIGTSTP': None,
'SIGTTIN': None,
'SIGTTOU': None,
--
http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
Joel Martin nos...@martintribe.org writes: I'm running python-2.6.2 which supposedly has the fix for #1731717. However I still still the problem with subprocess after daemonizing. I've narrowed it down to just the setting of the SIGCLD signal. You can reproduce the problem thus (in 2.4.6, 2.5.4 and 2.6.2): import subprocess, signal signal.signal(signal.SIGCLD, signal.SIG_IGN) subprocess.Popen(['echo','foo']).wait() This would be good to have in the bug report; please add this test case to URL:http://bugs.python.org/issue1731717 so the responsible parties can reproduce the problem as you've shown. This patch allows python-daemon to work with subprocess: Thank you, this *really* helps narrow down the problem. I don't know if I'll simply be removing the handling as you suggest, but it certainly makes the range of solutions much clearer. -- \ “The process by which banks create money is so simple that the | `\ mind is repelled.” —John Kenneth Galbraith, _Money: Whence It | _o__) Came, Where It Went_, 1975 | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
Ben Finney ben+pyt...@benfinney.id.au writes: I'm not familiar enough with the nuances of the ‘subprocess’ module to know what might be going wrong here. I'd like to know whether it might be a problem in the ‘python-daemon’ library. My test case for this is now:: = #! /usr/bin/python import daemon import os import sys import subprocess fake_console = open(fake_console.txt, w+) daemon.DaemonContext(stdout=fake_console, stderr=fake_console).open() sys.stdout.write(Parent daemon process.\n) os.system(echo Child process via 'os.system'.) subprocess.Popen([echo, Child process via 'subprocess.Popen'.]).wait() sys.stdout.write(Parent daemon process done.\n) = causing (in the ‘fake_console.txt’ output file):: = Child process via os.system. Child process via 'subprocess.Popen'. Parent daemon process. Traceback (most recent call last): File /home/bignose/Projects/python/python-daemon/python-daemon.devel/bin/clegg-example, line 13, in module subprocess.Popen([echo, Child process via 'subprocess.Popen'.]).wait() File /usr/lib/python2.5/subprocess.py, line 1184, in wait pid, sts = self._waitpid_no_intr(self.pid, 0) File /usr/lib/python2.5/subprocess.py, line 1014, in _waitpid_no_intr return os.waitpid(pid, options) OSError: [Errno 10] No child processes = Ben Finney ben+pyt...@benfinney.id.au writes: Joel Martin nos...@martintribe.org writes: I'm running python-2.6.2 which supposedly has the fix for #1731717. However I still still the problem with subprocess after daemonizing. I've narrowed it down to just the setting of the SIGCLD signal. […] Thank you, this *really* helps narrow down the problem. I don't know if I'll simply be removing the handling as you suggest, but it certainly makes the range of solutions much clearer. I'm also glad to see a test case that causes exactly the same error with or without the presence of a ‘daemon.DaemonContext’. Further research shows that handling of ‘SIGCLD’ (or ‘SIGCLD’) is fairly OS-specific, with “ignore it” or “handle it specifically” being correct on different systems. I think Python's default handling of this signal is already good (modulo bug #1731717 to be addressed in ‘subprocess’). So I will apply a change similar to Joel Martin's suggestion, to default to avoid touching the ‘SIGCLD’ signal at all, and with extra notes in the documentation that anyone using child processes needs to be wary of signal handling. This causes the above test case to succeed; the output file contains:: = Child process via os.system. Child process via 'subprocess.Popen'. Parent daemon process. Parent daemon process done. = -- \ “Reality must take precedence over public relations, for nature | `\cannot be fooled.” —Richard P. Feynman | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
I looked at other daemon libraries and snippets, it's clearly the bug is in subprocess not python-daemon. Then I found Python bug #1731717 which discusses it. I wish my project was opensource so I can post more specific test cases. #1731717 http://bugs.python.org/issue1731717 Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
Sewar xsew...@gmail.com writes: I looked at other daemon libraries and snippets, it's clearly the bug is in subprocess not python-daemon. Then I found Python bug #1731717 which discusses it. Thank you very much! I'm glad to see this is a known issue and that some investigation has already been done. (It's a bit depressing that the bug is over two years old, though.) I wish my project was opensource so I can post more specific test cases. Can you make a small, complete test case that shows the problem? You could then post that to the bug report, since it seems they are currently without a good test case for this bug in Python 2.6. (Discussing the details further in this thread would be counter-productive; the discussion should go to the bug report.) Thanks again for finding this! -- \ “I was sleeping the other night, alone, thanks to the | `\ exterminator.” —Emo Philips | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
I got the same bug. Traceback (most recent call last): File ./script1.py, line 30, in module call([python, script2.py, arg1], stdout=sys.stdout, stderr=STDOUT) File /usr/lib/python2.6/subprocess.py, line 444, in call return Popen(*popenargs, **kwargs).wait() File /usr/lib/python2.6/subprocess.py, line 1123, in wait pid, sts = os.waitpid(self.pid, 0) OSError: [Errno 10] No child processes -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
I got the same bug. Traceback (most recent call last): File ./script1.py, line 30, in module call([python, script2.py, arg1], stdout=sys.stdout, stderr=STDOUT) File /usr/lib/python2.6/subprocess.py, line 444, in call return Popen(*popenargs, **kwargs).wait() File /usr/lib/python2.6/subprocess.py, line 1123, in wait pid, sts = os.waitpid(self.pid, 0) OSError: [Errno 10] No child processes -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
Ben Finney ben+pyt...@benfinney.id.au writes: My first thought was perhaps it's related to the fact that the process has no stdout? But setting ‘stdout’ and ‘stderr’ to the same file:: #! /usr/bin/python import daemon import subprocess fake_console = open(fake_console.txt, w+) daemon.DaemonContext(stdout=fake_console, stderr=fake_console).open() subprocess.Popen(['echo', '1']).wait() For newcomers to this thread: the ‘daemon’ module is provided by the URL:http://pypi.python.org/pypi/python-daemon ‘python-daemon’ distribution, of which I am the maintainer. still gives the same error:: 1 Traceback (most recent call last): File /home/bignose/Projects/python/python-daemon/bin/andy-clegg-test, line 8, in module subprocess.Popen(['echo', '1']).wait() File /usr/lib/python2.5/subprocess.py, line 1184, in wait pid, sts = self._waitpid_no_intr(self.pid, 0) File /usr/lib/python2.5/subprocess.py, line 1014, in _waitpid_no_intr return os.waitpid(pid, options) OSError: [Errno 10] No child processes I'm not familiar enough with the nuances of the ‘subprocess’ module to know what might be going wrong here. I'd like to know whether it might be a problem in the ‘python-daemon’ library. I am no closer to a solution on this one. I don't have enough experience or knowledge of the ‘subprocess’ module to know whether it's a bug in ‘subprocess’, a bug in ‘python-daemon’, or a hard-to-track interaction between the two. Any narrowing of the problem would be appreciated. -- \“I bought a dog the other day. I named him Stay. It's fun to | `\ call him. ‘Come here, Stay! Come here, Stay!’ He went insane. | _o__) Now he just ignores me and keeps typing.” —Steven Wright | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
Andy Clegg andy...@gmail.com writes: import daemon import subprocess daemon.DaemonContext(stderr = open(fakeConsole.txt,w+)).open() subprocess.Popen(['echo','1']).wait() However the error remains the same. The error appears in the file specified for the stderr output of the DaemonContext. Here it is without the unwanted extra line-wrapping that seems to plague all Google Mail users (seriously, folks: get a real mail provider that won't mangle your messages):: Traceback (most recent call last): File /home/bignose/Projects/python/python-daemon/bin/andy-clegg-test, line 7, in module subprocess.Popen(['echo', '1']).wait() File /usr/lib/python2.5/subprocess.py, line 1184, in wait pid, sts = self._waitpid_no_intr(self.pid, 0) File /usr/lib/python2.5/subprocess.py, line 1014, in _waitpid_no_intr return os.waitpid(pid, options) OSError: [Errno 10] No child processes I confirm that I'm getting the same error; Python 2.5.4. My first thought was perhaps it's related to the fact that the process has no stdout? But setting ‘stdout’ and ‘stderr’ to the same file:: #! /usr/bin/python import daemon import subprocess fake_console = open(fake_console.txt, w+) daemon.DaemonContext(stdout=fake_console, stderr=fake_console).open() subprocess.Popen(['echo', '1']).wait() still gives the same error:: 1 Traceback (most recent call last): File /home/bignose/Projects/python/python-daemon/bin/andy-clegg-test, line 8, in module subprocess.Popen(['echo', '1']).wait() File /usr/lib/python2.5/subprocess.py, line 1184, in wait pid, sts = self._waitpid_no_intr(self.pid, 0) File /usr/lib/python2.5/subprocess.py, line 1014, in _waitpid_no_intr return os.waitpid(pid, options) OSError: [Errno 10] No child processes I'm not familiar enough with the nuances of the ‘subprocess’ module to know what might be going wrong here. I'd like to know whether it might be a problem in the ‘python-daemon’ library. -- \ “Whatever a man prays for, he prays for a miracle. Every prayer | `\ reduces itself to this: “Great God, grant that twice two be not | _o__) four.”” —Ivan Turgenev | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Email provider recommendations for those tired of mangled messages (was: subprocess + python-daemon - bug/problem?)
Ben Finney ben+pyt...@benfinney.id.au writes: Here it is without the unwanted extra line-wrapping that seems to plague all Google Mail users (seriously, folks: get a real mail provider that won't mangle your messages) I've been asked off-list whether I have any suggestions for those who want a better email provider. Handling email (transport and mailbox services) is certainly a service that's worth getting someone else to handle, provided you can trust they'll do so reliably and well. For me, “well” includes “don't mangle my messages”. My personal solution to this is to manage my own email server machine, but that's an ongoing investment of time that is not something I would recommend to everyone. Nancy McGough maintained (past tense? present tense?) a comprehensive index URL:http://www.ii.com/internet/messaging/imap/isps/#providers of email service providers with IMAP access, hence allowing you to use whatever IMAP mail client you like to communicate with your mailboxes remotely. There are feature lists and price indications for each provider. Several of my colleagues swear by Tuffmail URL:http://tuffmail.com/ as providing excellent uptime, fine control filtering, responsive service, flexible customisation, and good overall value. -- \“Visitors are expected to complain at the office between the | `\ hours of 9 and 11 a.m. daily.” —hotel, Athens | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
subprocess + python-daemon - bug/problem?
Hi all,
I'm trying to daemonize a python program, and occasionally have it run
subprocesses, however I'm running into a nasty error, as follows:
File /users/rsg/ancl/devcocast/devcocast-svn/scripts/DaemonSpawnTes
t.py, line 5, in module
subprocess.Popen('echo 1').wait()
File /usr/lib64/python2.5/subprocess.py, line 594, in __init__
errread, errwrite)
File /usr/lib64/python2.5/subprocess.py, line 1089, in _execute_ch
ild
os.waitpid(self.pid, 0)
OSError: [Errno 10] No child processes
The minimal testcase showing this problem is as follows:
import daemon
import subprocess
daemon.DaemonContext(stderr = open(fakeConsole.txt,w+)).open()
subprocess.Popen('echo 1').wait()
So there is no threading going on (I've found some bugs relating to
subprocess and threading). I'm using Fedora 10, Python 2.5.2, and
python-daemon 1.4.6 from here http://pypi.python.org/pypi/python-daemon/
.
If anyone can shed some light on the situation, I'd be extremely
grateful!
Yours,
Andy
--
http://mail.python.org/mailman/listinfo/python-list
Re: subprocess + python-daemon - bug/problem?
My apologies, the python code should have been:
import daemon
import subprocess
daemon.DaemonContext(stderr = open(fakeConsole.txt,w+)).open()
subprocess.Popen(['echo','1']).wait()
However the error remains the same.
Yours,
Andy
On Jul 9, 10:26 am, Andy Clegg andy...@gmail.com wrote:
Hi all,
I'm trying to daemonize a python program, and occasionally have it run
subprocesses, however I'm running into a nasty error, as follows:
File /users/rsg/ancl/devcocast/devcocast-svn/scripts/DaemonSpawnTes
t.py, line 5, in module
subprocess.Popen('echo 1').wait()
File /usr/lib64/python2.5/subprocess.py, line 594, in __init__
errread, errwrite)
File /usr/lib64/python2.5/subprocess.py, line 1089, in _execute_ch
ild
os.waitpid(self.pid, 0)
OSError: [Errno 10] No child processes
The minimal testcase showing this problem is as follows:
import daemon
import subprocess
daemon.DaemonContext(stderr = open(fakeConsole.txt,w+)).open()
subprocess.Popen('echo 1').wait()
So there is no threading going on (I've found some bugs relating to
subprocess and threading). I'm using Fedora 10, Python 2.5.2, and
python-daemon 1.4.6 from herehttp://pypi.python.org/pypi/python-daemon/
.
If anyone can shed some light on the situation, I'd be extremely
grateful!
Yours,
Andy
--
http://mail.python.org/mailman/listinfo/python-list
