Akira Li added the comment: > setting "universal_newlines=True" switches to UTF-8 encoded text pipes
It uses locale.getpreferredencoding(False) encoding -- something like cp1252,cp1251,etc on Windows, and UTF-8 on *nix with proper locale settings. It is ASCII (C/POSIX locale default) if the locale is not set in cron, ssh, init.d scripts, etc. If you need a different character encoding, you could use (instead of universal_newlines=True): pipe = io.TextIOWrapper(process.stdout, encoding=character_encoding) A better spelling for universal_newlines=True would be text_mode=True. A summary table (like in itertools' module) would be nice. check_output() name is unfortunate but it does the right thing and it is not hard to use for a beginner -- once somebody discovers it e.g., via "Running shell command from Python and capturing the output" Stack Overflow question http://stackoverflow.com/questions/4760215/running-shell-command-from-python-and-capturing-the-output Compare: output = check_output([sys.executable, '-c', 'print("abc")']) and output = run([sys.executable, '-c', 'print("abc)'], stdout=PIPE).stdout The latter command doesn't raise exception if the child process fails. A beginner has to know about check=True to do the right thing: output = run([sys.executable, '-c', 'print("abc")'], stdout=PIPE, check=True).stdout It is easier to refer to check_output() if someone asks "How do I get command's output in Python?" I wish call() did what check_call() does and the current call() behavior would be achieved by the opposite parameter e.g. no_raise_on_status=False being the default: rc = call(command, no_raise_on_status=True) If we can't change the interface then check_call() is the answer to "Calling an external command in Python" question http://stackoverflow.com/questions/89228/calling-an-external-command-in-python - check_call(command) -- run command, raise if it fails - output = check_output(command) -- get command's output, raise if it fails. To pass *data* to the command via its standard input, pass input=data. To get/pass text (Unicode) instead of bytes, pass universal_newlines=True - check_call("a -- *.jpg | b 2>&1 >output | c", shell=True) -- run a shell command as is It is a pity that a list argument such as ["ls", "-l"] is allowed with shell=True These cover the most common operations with a subprocess. Henceforth, run() is more convenient if we don't need to interact with the child process while it is running. For example, if we introduce the word PIPE (a magic object in the kernel that connects processes) then to capture both standard and error streams of the command: cp = run(command, stdout=PIPE, stderr=PIPE) output, errors = cp.stdout, cp.stderr run() allows to get the output and to get the exit status easily: cp.returncode. Explicit cp.stdout_text, cp.stdout_bytes regardless the text mode would be nice. To interact with a child process while it is running, Popen() have to be used directly. There could be buffering and other issues (tty vs. pipe), see "Q: Why not just use a pipe (popen())?" http://pexpect.readthedocs.io/en/latest/FAQ.html#whynotpipe Working with both stdout/stderr or a non-blocking read require threads or asyncio, fcntl, etc. A couple of words should be said about killing a command started with shell=True. (to kill a process tree: set start_new_session=True parameter and call os.killpg()). timeout option doesn't work in this case (it uses Popen.kill()). check_output() unlike check_call() may wait for grandchildren if they inherit the pipe. Mention Job object on Windows e.g., http://stackoverflow.com/questions/23434842/python-how-to-kill-child-processes-when-parent-dies ---------- nosy: +akira _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27050> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
