[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry added the comment: emmanuel, The Python interpreter isn't reentrant, so you could only run two interactive sessions connected to the same Python environment if you implemented your own REPL function that unlocked the GIL when waiting for input, then lock it just long enough to interpret the input. Also, the readline module already does what you suggest (sets PyOS_ReadlineFunctionPointer to a GNU libreadline wrapper.) My readline hack (working.c) forces it to behave as it's supposed to. Rather, it *undoes* what PyRun_InteractiveLoop does every iteration, which is pass stdin/stdout for I/O, which libreadline in turn uses. I agree that it would be nice to get a Python developer involved, mostly because I expect things to break when this problem is fixed. Bad things happen when you think you've tested a lot of different cases that actually turn out to be the exact same case. Kevin Barry -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14916 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry added the comment: emmanuel, Regarding your points: All three can be taken care of with a combination of my patch and setting sys.stdin, sys.stdout, and sys.stderr to the pty. (That should really be done internally with another patch, since os.fdopen is OS-specific. Also, sys.stdin, sys.stdout, and sys.stderr should each have distinct underlying file descriptors, which I didn't do in working.c.) Those can safely be replaced since they're just the effective standard files, and sys.__stdin__ et al. refer to the actual C stdin et al. The remaining issue would be that the same descriptor shouldn't be used for both input and output in the interpreter loop, especially if the FILE* passed is only open for reading (since standard input technically doesn't have to be writable.) Kevin Barry -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14916 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry added the comment: emmanuel, Thanks for the suggestion. Your workaround is exactly the same as using dup2 (in C) to replace stdin/stdout/stderr with the pty, however. If you added the following lines to your C code, it would have the same effect as the command-line redirection in the workaround: dup2(fileno(file), STDIN_FILENO); dup2(fileno(file), STDOUT_FILENO); dup2(fileno(file), STDERR_FILENO); In fact, that's exactly what bash does after forking, just before executing exe. In most cases, developers who use PyRun_InteractiveLoop in a pty probably also do exactly that, which is why I'm the only one who's reported this as a bug. For applications like mine, however, where the interactive Python session needs to be an unobtrusive add-on to an otherwise-complete program, this solution won't work. The standard file descriptors aren't disposable in most of the programs I work on. Thanks again! Kevin Barry -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14916 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry added the comment: One additional issue, which my patch doesn't address, is that PyRun_InteractiveLoop should really take *two* FILE* arguments, with the second one being optional. This is because on Linux (and presumably on other *nixes) if a read operation is blocked on a file descriptor then write operations (from other threads) to the same file descriptor will also block. That doesn't happen in the current Python implementations because PyOS_Readline is always called with two FILE* objects, anyway (stdin and stdout.) I would, however, expect such a problem to appear if a user created a Python thread in the interactive session that periodically printed to the terminal, then read input from the terminal. In that case, I would expect to see no output from the thread while the read operations were blocked, but I haven't tested it. (I don't remember if this came up after I applied my patch locally.) I actually considered this when I created the patch; however, I didn't feel like going to all the trouble of adding a member to tok and propagating the change throughout the entire core. I had hoped this bug would get more attention and I'd be able to discuss it with a developer involved in the Python project, but ultimately that didn't happen and I ended up forgetting about it. Kevin Barry -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14916 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry added the comment: I still see the potential cause addressed by my patch in the 2.7, 3.3, and default branches, so I'm assuming that all versions from 2.6 on have this problem. I also see that I can elect to change the Status and Resolution of this report. Does that mean I need to do something besides wait for someone involved in the project to look at my patch? Kevin Barry -- versions: +Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14916 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry ta0k...@gmail.com added the comment:
The patch from before needed a slight modification for when Python actually
defaults to an interactive session on stdin. Since I rebuild this for my
current distro (Slackware64 13.37,) I switched to the Python 2.6.6 source. This
might not be the proper way to handle the default case (e.g. 'Py_Main'), but
it's a start.
The patch (also attached):
--- ./Parser/tokenizer.c.orig 2012-07-23 22:24:56.513992301 -0400
+++ ./Parser/tokenizer.c2012-07-23 22:23:24.329992167 -0400
@@ -805,7 +805,7 @@
return Py_CHARMASK(*tok-cur++);
}
if (tok-prompt != NULL) {
-char *newtok = PyOS_Readline(stdin, stdout, tok-prompt);
+char *newtok = PyOS_Readline(tok-fp? tok-fp : stdin, (tok-fp
tok-fp != stdin)? tok-fp : stdout, tok-prompt);
if (tok-nextprompt != NULL)
tok-prompt = tok-nextprompt;
if (newtok == NULL)
Kevin Barry
--
Added file:
http://bugs.python.org/file26501/Python-2.6.6-Run_Interactive-fix.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14916
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry ta0k...@gmail.com added the comment: I've attached a simplified example program (working3.c) that demonstrates both the original problem and that the patch (Python-2.6.6-Run_Interactive-fix.patch) works. It eliminates the need for a pty, 'xterm', and redirection. Compile the attached program with: gcc `python-config --cflags` working3.c -o working3 `python-config --ldflags` and run it with (before and after patching): ./working3 Also, for verification, run 'python' with no arguments to show that default interactivity is preserved. Kevin Barry -- Added file: http://bugs.python.org/file26503/working3.c ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14916 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry ta0k...@gmail.com added the comment:
Here is a patch that corrects the problem (quoted below and attached.) This
only corrects the problem when 'PyOS_ReadlineFunctionPointer' is set, e.g. you
must 'import readline', otherwise Python will defer to stdin/stdout with
'PyOS_StdioReadline'.
The patch:
--- Python-2.6.8/Parser/tokenizer.c 2012-04-10 11:32:11.0 -0400
+++ Python-2.6.8-patched/Parser/tokenizer.c 2012-07-23 19:56:39.645992101
-0400
@@ -805,7 +805,7 @@
return Py_CHARMASK(*tok-cur++);
}
if (tok-prompt != NULL) {
-char *newtok = PyOS_Readline(stdin, stdout, tok-prompt);
+char *newtok = PyOS_Readline(tok-fp? tok-fp : stdin, tok-fp?
tok-fp : stdout, tok-prompt);
if (tok-nextprompt != NULL)
tok-prompt = tok-nextprompt;
if (newtok == NULL)
Kevin Barry
--
keywords: +patch
status: open - pending
Added file:
http://bugs.python.org/file26491/Python-2.6.8-Run_Interactive-fix.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14916
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
Kevin Barry ta0k...@gmail.com added the comment: I've attached a new example source file to demonstrate the fix. Compile the attached program with (*after* patching and installing Python): gcc `python-config --cflags` working2.c -o working2 `python-config --ldflags` and run it with: ./working2 xterm -S/0 /dev/null /dev/null (The redirection shows that it works when stdin/stdout aren't a tty.) I looked at the most-recent revision of tokenizer.c (http://hg.python.org/cpython/file/52032b13243e/Parser/tokenizer.c) and see that the change in my patch above hasn't been made already. Kevin Barry -- status: pending - open Added file: http://bugs.python.org/file26492/working2.c ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14916 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14916] PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout
New submission from Kevin Barry ta0k...@gmail.com: I have been trying to get PyRun_InteractiveLoop to run on a pty (Linux) without replacing stdin and stdout with that pty; however, it seems like Python (2.6.6) is hard-coded to only run interactively on stdin and stdout. Compile the attached program with: gcc `python-config --cflags` working.c -o working `python-config --ldflags` and run it with: ./working xterm -S/0 and you should see that there is no interactivity in the xterm that's opened. Compile the attached file with: gcc -DREADLINE_HACK `python-config --cflags` working.c -o working `python-config --ldflags` -lreadline -lcurses and run it with: ./working xterm -S/0 to see how it runs with my best attempt to get it to function properly with a readline hack. Additionally, try running: ./working xterm -S/0 /dev/null ./working xterm -S/0 /dev/null both of which should cause interactivity in the xterm to fail, indicating that Python is checking stdin/stdout for tty status when determining if it should run interactively (i.e. it's not checking the tty status of the file passed to PyRun_InteractiveLoop.) Am I somehow using this function wrong? I've been trying to work around this problem for a while, and I don't think I should be using readline hacks (especially since they don't port to other OSes with ptys, e.g. OS X.) I even tried to patch the call to PyOS_Readline in tok_nextc (Parser/tokenizer.c) to use tok-fp instead of stdin/stdout, which caused I/O to use the pty but it still failed to make interactivity work. Thanks! Kevin Barry -- components: Interpreter Core, Library (Lib) files: working.c messages: 161586 nosy: Kevin.Barry priority: normal severity: normal status: open title: PyRun_InteractiveLoop fails to run interactively when using a Linux pty that's not tied to stdin/stdout type: behavior versions: Python 2.6 Added file: http://bugs.python.org/file25706/working.c ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14916 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
