Re: Buffering of sys.stdout and sys.stderr in python3 (and documentation)
It's surprising and broken that stderr should be buffered in python3. python3 calls setvbuf(3) on stderr at startup to achieve this chuckle-headed behavior. It makes stderr line buffered if on a terminal, and fully buffered if redirected to a log file. A fully buffered stderr is a very bad idea. This change goes against the C89 and C99 standards, 40+ years of precedent, and the behavior of nearly every other current programming language. Error messages will not show up in logs on time, and may be completely lost if the process is terminated before the buffer is flushed, e.g. with SIGTERM How about fixing this? Until then, we should write this in every script: sys.stderr = os.fdopen(sys.stderr.fileno(), 'w', 0) -- https://mail.python.org/mailman/listinfo/python-list
Re: Buffering of sys.stdout and sys.stderr in python3 (and documentation)
sys.stderr = os.fdopen(sys.stderr.fileno(), 'w', 0) which unfortunately doesn't work! I guess will resort to python3 -u, although I don't want stdout to be unbuffered. -- https://mail.python.org/mailman/listinfo/python-list
Re: Buffering of sys.stdout and sys.stderr in python3 (and documentation)
Hi Terry, The difference from 2.x should be in What's New in 3.0, except that the new i/o module is in 2.6, so it was not exactly new. The io module existed in 2.6, but it was not used by default for standard output and standard error. The only mention of this in What's New in 3.0 is in the section marked for changes that were already in 2.6 (which is wrong in this case), and it notes only that io.TextIOWrapper is now used, but not what implications that has for its behaviour and backward compatibility. You might be able to find more inhttp://python.org/dev/peps/pep-3116/ I skimmed through it but couldn't find anything relevant. It seems more advanced and implementation-focussed. You *should* be able to find sufficient info in the 3.x docs. If, after you get other responses (or not), you think the docs need upgrading, open an issue on the tracker at bugs.python.org with suggestions as specific as possible, including changed or new lines of text based on your experience and experiments. OK, I'll do that if nobody points me at some existing docs here. Regards, Geoff Bache -- http://mail.python.org/mailman/listinfo/python-list
Buffering of sys.stdout and sys.stderr in python3 (and documentation)
Hi all, Short version: I'm a bit confused in general as to the changes between python2 and python3 regarding how standard output and standard error do buffering. A few things seem to have changed and I've failed to find any documentation of how and why. Also, the meaning of python -u seems to have changed and the docs don't seem to reflect the new behaviour (and I can't find any docs about the change either)... Long version: From rude experiment it seems that: 1) In Python 2.x, standard error was always unbuffered while standard output was buffered by default. In python3, both are buffered. In both cases, buffered means line-buffered when writing to the console and not line-buffered when redirected to files. 2) In Python 2.x, the -u flag meant everything was totally unbuffered. In Python 3.x, it means that both stdout and stderr are line-buffered also when redirected to files. Are either of these changes documented anywhere? (1) seems important : it can lead to not seeing exception printouts, if stderr is redirected to a file and the program is subsequently terminated with SIGTERM. I just wasted quite a bit of time due to this situation... This is what the Python 3 docs have to say about the -u flag: Force the binary layer of the stdin, stdout and stderr streams (which is available as their buffer attribute) to be unbuffered. The text I/O layer will still be line-buffered. The still seems misleading to me, as it is only relevant if writing to the console. It would be useful to contrast the behaviour with and without -u when writing to files I would say. Regards, Geoff Bache -- http://mail.python.org/mailman/listinfo/python-list
Re: Buffering of sys.stdout and sys.stderr in python3 (and documentation)
On 12/9/2011 2:32 PM, Geoff Bache wrote: Hi all, Short version: I'm a bit confused in general as to the changes between python2 and python3 regarding how standard output and standard error do buffering. A few things seem to have changed and I've failed to find any documentation of how and why. Also, the meaning of python -u seems to have changed and the docs don't seem to reflect the new behaviour (and I can't find any docs about the change either)... Long version: From rude experiment it seems that: 1) In Python 2.x, standard error was always unbuffered while standard output was buffered by default. In python3, both are buffered. In both cases, buffered means line-buffered when writing to the console and not line-buffered when redirected to files. 2) In Python 2.x, the -u flag meant everything was totally unbuffered. In Python 3.x, it means that both stdout and stderr are line-buffered also when redirected to files. Are either of these changes documented anywhere? (1) seems important : it can lead to not seeing exception printouts, if stderr is redirected to a file and the program is subsequently terminated with SIGTERM. I just wasted quite a bit of time due to this situation... This is what the Python 3 docs have to say about the -u flag: Force the binary layer of the stdin, stdout and stderr streams (which is available as their buffer attribute) to be unbuffered. The text I/O layer will still be line-buffered. The still seems misleading to me, as it is only relevant if writing to the console. It would be useful to contrast the behaviour with and without -u when writing to files I would say. The difference from 2.x should be in What's New in 3.0, except that the new i/o module is in 2.6, so it was not exactly new. You might be able to find more in http://python.org/dev/peps/pep-3116/ You *should* be able to find sufficient info in the 3.x docs. If, after you get other responses (or not), you think the docs need upgrading, open an issue on the tracker at bugs.python.org with suggestions as specific as possible, including changed or new lines of text based on your experience and experiments. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
