[issue11820] idle3 shell os.system swallows shell command output

[kent]

New submission from kent :

attempting to run an os.system command under the idle 3 shell swallows the out 
put.
Idle 3 is running on a 32 bit kde mandriva linux.

>>> import os
>>> os.system('ls')
0
>>> os.system('pwd')
0


as you can see it returns a 0 indicating successful completion, but no output. 
However os.getcwd works perfectly.

>>> os.getcwd()
'/home/kent/Documents' 

running the same code from python in an xwindow terminal works fine.

apparently the idle shell does not echo the the standard output or error output 
as the python interpreter does.

--
components: IDLE
messages: 133452
nosy: Thekent
priority: normal
severity: normal
status: open
title: idle3 shell os.system swallows shell command output
type: behavior
versions: Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[kent]

kent  added the comment:

running it as a file from idle gives the same result.
import os
print (os.system('pwd'))

0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[kent]

kent  added the comment:

When starting idle from a terminal the output from the command is sent to the 
terminal.  When starting idle from the desktop, the output disappears except  
for the exit status.  Same behavior with 2.65

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I am not sure if this should be called a bug or feature request, but that does 
not matter so much with IDLE. 

Os.system is documented as executing in a subshell and returning the exit code, 
which is does. The doc also says "If command generates any output, it will be 
sent to the interpreter standard output stream."

IDLE tries to imitate the interpreter, but it is not the interpreter, and I am 
not sure if that is always possible. The problem is that IDLE sends code (or, I 
presume, a filename) to a windowless interpreter (via socket or pipe) and 
receives and displays whatever is sent back. So I suspect the problem and fix 
is and would have to be with how a windowless interpreter executes os.system 
(in a third process). But for all I know, it may be the OS that decides not to 
hook the output of a system process to a no-window process that calls it.

On Windows, os.system('dir') ('dir' == 'ls') within IDLE pops up a command 
window to display the output, which immediately disappears. The same within the 
interactive interpreter (in a Command Prompt window) displays the output, just 
as with your XTerminal case.

Os.getcwd is documented as returning a string, so of course it does. It is not 
relevant to this issue.

Because of problems with os.system, the docs end with a suggestion to use 
subprocess instead. So there may be reluctance to 'fix' os.system calls.

The subprocess doc has an example for 'ls'. For 3.2 it is

>>> subprocess.check_output(["ls", "-l", "/dev/null"])
b'crw-rw-rw- 1 root root 1, 3 Oct 18  2007 /dev/null\n'

but the quotes and \n suggest that multiline output would not display properly.

On Windows with 3.2, the following works

>>> print(subprocess.check_output(['dir'], shell=True).decode())
 Volume in drive C is HP_PAVILION
 Volume Serial Number is 6C44-B700

 Directory of C:\Programs\Python32

shell=True is needed for 'dir' to be recognized.
Both print and .decode() are needed for proper line breaks.

The same info for the current directory is also available in an Open File or 
Save File dialog, so the ls/dir is really not needed.

--
components: +Interpreter Core
nosy: +terry.reedy
versions: +Python 3.2, Python 3.3 -Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[kent]

kent  added the comment:

I tried using subprocess.Popen and subprocess.call, both of which did the same 
behavior. Under the interpreter I get the desired string output:
>>> subprocess.call('ls')
bin  Documents  eclipse  local  Pictures   tmp workspace
Desktop  Downloads  hamlib   Music  Templates  Videos
0

Under Idle:

>>> subprocess.call('ls')
0
Thus the subprocess.call method provides the string output.  Why should I have 
to use the subprocess.check_output('command').decode() when subprocess.call() 
gives me what I want?

Thus it does not seem to be an os.system issue, but a failure of Idle to 
capture the sysoutput string as the interpreter does  The xterminal example 
highlights this.  The sysoutput is echoed at the xterminal not in idle.

You are correct there are other ways to get the specific information for ls.  I 
was using that as test command.  I was experimenting with the commands in order 
to a write a program which will run an executable under either windows or 
linux.  I wanted to see the output in the interpreter in order to test it.  I 
was using Idle for the testing.  It does NOT work the same as the interpreter.  
If Idle is to be useful as an IDE, shouldn't it's shell work the same as the 
interpreter? If it doesn't why use Idle ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[Ned Deily]

Ned Deily  added the comment:

I agree that it is kind of odd behavior and, after a quick look back through 
open issues, I was a bit surprised to not find an open issue about it (although 
I may have overlooked one).

"Thus it does not seem to be an os.system issue, but a failure of Idle to 
capture the sysoutput string as the interpreter does  The xterminal example 
highlights this.  The sysoutput is echoed at the xterminal not in idle."

The reason this is an issue is because the interpreter does *not* do anything 
special to capture stdout from subprocesses.  If you're running the interpreter 
in a terminal window both stdout from the interpreter process and, by default, 
stdout from any child processes, like those created by os.system or subprocess 
(if you don't override it), inherit the same stdout and so the output from both 
processes show up in the same place (i.e. terminal session) automatically.  
When running under IDLE though, IDLE uses its own proxy output object to 
intercept the output from the interpreter but does not do anything special for 
stdout from subprocesses so that output ends up in unexpected places: the 
terminal session from which IDLE was launched or the transient window on 
Windows or into the system log with the OS X IDLE.app.  I think IDLE could do a 
better job of intercepting stdout for these cases but it might be a bit tricky 
to make it work across all platforms.

--
components:  -Interpreter Core
nosy: +ned.deily
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[kent]

kent  added the comment:

I had kind of figured it might be something like this.  I ran the following 
code in the xterm interpreter:
>>> x=subprocess.call('ls')
bin  Documents  eclipse  local  Pictures   tmp workspace
Desktop  Downloads  hamlib   Music  Templates  Videos
>>> print(x)
0
It does not capture the output.

--
components: +Interpreter Core

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[kent]

kent  added the comment:

The getoutput and getstatusoutput provide the expect output which can be 
captured

>>> x=subprocess.getoutput('ls')
>>> print(x)
hs_err_pid28274.log
LP4E-examples
mydir.pth
mydir.pth~
PP4E-Examples-1.2
ProgMan
Python_dir


Would it be a good thing to have the interpreter capture the sysout of all 
child processes automatically?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[Terry J. Reedy]

Terry J. Reedy  added the comment:

After thinking some more and re-reading the os.system doc, I am closing this. I 
think things are working exactly as documented, and in a way that will not be 
changed. The os.system doc initially says

1. "Execute the command (a string) in a subshell."

It does that.

2. "This is implemented by calling the Standard C function system(), and has 
the same limitations."

Read Standard C for more on that.

Nearly everything in os is a thin wrapper of operating system data and 
functions. Enhanced wrappers that add user-friendliness, go elsewhere.

3. "Changes to sys.stdin, etc. are not reflected in the environment of the 
executed command."

The sys module is Python-specific and has no effect on system functions. In 
particular, whether sys.stdout, initially copied from the process, is initally 
null or the terminal, Idle changing sys.stdout has no effect on os.system 
output, and it will not see it. This is, obviously, a known and expected 
limitation.

4. "If command generates any output, it will be sent to the interpreter 
standard output stream."

The interpreter process stdout is not sys.stdout. If it is null, bye-bye output.

The doc ends with

5. "The subprocess module provides more powerful facilities for spawning new 
processes and retrieving their results; using that module is preferable to 
using this function."

In other words, the limitations of os.system are known, and it is 
semi-deprecated, especially when it does not work as one would wish, and a 
replacement has been provided. The subprocess module is an example of an 
'enhanced wrapper'.

6 "See the Replacing Older Functions with the subprocess Module section in the 
subprocess documentation for some helpful recipes."

These recipes are replacements for things that work. The .check_output method 
enables things that one could not very easily do before. I already gave 
examples of how to use it.

--
resolution:  -> invalid
stage: needs patch -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[Ned Deily]

Ned Deily  added the comment:

I disagree. I think you are confusing Python's sys.std* objects with the std* 
file descriptors of a process.  The important points here are that IDLE is 
effectively substituting its rpcproxy for the std* file descriptors of the IDLE 
shell process that it creates *and* that, in POSIX world, a forked subprocess 
is required to inherit copies of the file descriptors (in particular, sys.std*) 
of its parent (the IDLE shell subprocess).  So, if the IDLE shell subprocess 
displays the output written to the stdout and stderr in the Python Shell window 
and directs typed-in input from that window to the stdin file descriptor, it is 
reasonable to assume that the same thing should happen by default to processes 
launched within that IDLE shell subprocess, be they by os.system or by 
subprocess.Popen.  subprocess.Popen lets you substitute sys* file associations 
when creating the subprocess but that is irrelevant for the default case.  The 
default for subprocess.Popen is exactly the same as os.system: "With the 
default settings of None, no redirection will occur; the child’s file handles 
will be inherited from the parent."

--
resolution: invalid -> 
stage: committed/rejected -> needs patch
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I thought I was un-confusing sys.std* Python objects (such as 
idlelib.rpc.RPCProxy) from std* integer file descriptors, whereas you seem to 
say they are the same: "copies of the file descriptors (in particular, 
sys.std*)". The rest of your message seems to imply that 
subprocess.check_output should not work, but it does, at least on Windows. And 
subprocess *is* a recommended replacement for os.system.

Roger, do you have any thoughts on this issue?

--
nosy: +serwy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[Ned Deily]

Ned Deily  added the comment:

I didn't mean to imply anything about subprocess.check_output().  It works as 
expected, that is by explicitly intercepting anything written to the stdout of 
the forked subprocess and returning it as the call result.  What doesn't work 
is if you try a default subprocess.call(), for example.  The results are the 
same as os.system and for the same reason.  In a Posix terminal session:

$ python3.3
>>> import subprocess
>>> subprocess.call('pwd')
/Users/nad
0
>>> 

But in an IDLE shell window:
>>> import subprocess
>>> subprocess.call('pwd')
0
>>> 

That's confusing to users.  IDLE should handle that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11820] idle3 shell os.system swallows shell command output

[Roger Serwy]

Roger Serwy  added the comment:

It might be possible to fix this problem by creating pipes to redirect stdout 
and stderr to the text widget in PyShell. 

The ShellWindow.py demo in Demo/tkinter/guido may be helpful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com