Oh, I thought that stderr was unbuffered. Like the following C program:
#include <stdio.h>
int main() {
fprintf(stdout, "stdout0");
fprintf(stderr, "stderr0");
fprintf(stdout, "stdout1");
return 0;
}
This outputs:
stderr0stdout0stdout1
Turns out fprintf in glibc will also implicitly flush at newlines also. So, the
following program:
#include <stdio.h>
int main() {
fprintf(stdout, "stdout0\nstdout12");
fprintf(stderr, "stderr0");
fprintf(stdout, "stdout1");
return 0;
}
outputs:
stdout0
stderr0stdout12stdout1
This is in line with python. The following program:
import sys
sys.stdout.write('stdout0\nstdout01')
sys.stderr.write('stderr0')
sys.stdout.write('stdout1')
outputs:
stdout0
stderr0stdout01stdout1
While the following program:
import sys
sys.stdout.write('stdout0')
sys.stderr.write('stderr0')
sys.stdout.write('stdout1')
outputs:
stderr0stdout0stdout1
So, I guess that the errfile is explicitly flushed due to the fact that stdout
may or may not be flushed during output.
So, I have 2 questions: (1) do we need to keep the errfile param to make this
useful (I think we do), and if #1 is yes, (2) what is an appropriate way to doc
this?
wt
________________________________
From: Amit Green <amit.mi...@gmail.com>
Sent: Friday, September 29, 2017 9:17:39 AM
To: Wren Turkal
Cc: Steven D'Aprano; python-ideas@python.org
Subject: Re: [Python-ideas] allow overriding files used for the input builtin
Hmm, very good point.
Flushing standard error is essential:
* This is because output of standard output & standard error are often
redirected to the same file -- frequently the terminal -- and its important to
flush one of them before output to the other (since these are typically line
buffered you only see this if you output partial lines).
* Here is an example of not flushing & the mess it causes:
>>> import sys
>>> sys.stdout.write('hi'); sys.stderr.write(' HMM ');
>>> sys.stdout.write('there\n')
HMM hithere
* Here is the same example with flushing:
>>> sys.stdout.write('hi'); sys.stdout.flush(); sys.stderr.write(' HMM ');
>>> sys.stderr.flush(); sys.stdout.write('there\n')
hi HMM there
In fact, I think you need to improve your interface and documentation:
* If either of the parameters 'outfile', or 'errfile' are passed in & not
the 'none' value then the following happens:
* Both sys.stdout & sys.stderr are flushed (in that order) before being
redirected. (NOTE: Both are flushed, even if only one is redirected).
* Before restoring sys.stdout & sys.stderr; then outfile & errfile are
flushed (if both have been used then they are flushed in that order).
You would of course need to write the documentation clearer than I did above
(writing documentation well is not my skill) -- I wrote it to convey exactly
what has to happen.
On Fri, Sep 29, 2017 at 12:01 PM, Wren Turkal <w...@fb.com<mailto:w...@fb.com>>
wrote:
Steven and Amit,
I originally configured the mailing list for digest delivery and can't reply
directly to his message. However, I now seen it, I will update the PR with the
suggested name changes as soon as my "make test" finishes. FWIW, I've changed
to direct message delivery.
With regard to ferr (the corresponding variable in the builtin_input c function
before the change), I saw this code:
/* First of all, flush stderr */
tmp = _PyObject_CallMethodId(ferr, &PyId_flush, NULL);
if (tmp == NULL)
PyErr_Clear();
else
Py_DECREF(tmp);
And I assumed that it was important to be able to override a stderr as a result.
I think there are few options here to resolve that:
1. Remove the errfile parameter and just do what happened before.
2. Remove the errfile and the above code (FWIW, I am not sure I understand
the importance of flushing stderr before taking input).
3. Document the explicit purpose of the errfile. Amit, you'd responded with
something about this (again digest reply, sorry). I am not sure how to
concisely do that given Amit's descriptions. 😊
I am honestly leaning toward 2 unless I can figure out why the flushing of
stderr is actually needed.
wt
________________________________
From: Amit Green <amit.mi...@gmail.com<mailto:amit.mi...@gmail.com>>
Sent: Friday, September 29, 2017 8:13:21 AM
To: Wren Turkal
Cc: python-ideas@python.org<mailto:python-ideas@python.org>
Subject: Re: [Python-ideas] allow overriding files used for the input builtin
Yes, infile, outfile & errfile would be consistent with python naming
convention (also mentioned by Steven D'Aprano above)
One of python's greatest strength is its library, the consistency of the
library, and how well documented the library is (in fact, I think the library
is a greater strength than even the very nice syntax of python in general).
By "consistency of the library" I mean: functions do pretty much what you
expect, they use consistent error mechanism & the documentation pretty much
accurately documents what the function does -- especially as to showing its
results & how it handles errors.
Regarding this though, this then brings up the question (above from Steven
D'Aprano) -- what would the the "errfile" parameter do?
* As a general principle of consistency python library functions, and
python itself, do not output to errfile, but instead throw errors.
* (There are very minor exceptions such as exceptions thrown in __del__
functions; which are caught by python & then printed to standard error).
I would thus think you don't want the errfile parameter -- unless it would be
for catching these __del__ method that get triggered by input failing (for
example your 'infile' parameter when called, allocated an object, which gets
deallocated & throws an exception inside of its __del__ method).
If this is the purpose, then (back to 'how well documented the library is') --
it should be documented this is the purpose of the "errfile" parameter ;-)
[A secondary reason you might want to redirect "errfile" is that the passed in
input or output file's, themselves do output to standard error ...]
On Fri, Sep 29, 2017 at 10:50 AM, Wren Turkal <w...@fb.com<mailto:w...@fb.com>>
wrote:
I am happy to rename the args. What do you think about infile, outfile, and
errfile?
FWIW, I did consider "in", "out", and "err", but "in" is a keyword, and I
didn't think those quite captured the full meaning.
wt
________________________________
From: Amit Green <amit.mi...@gmail.com<mailto:amit.mi...@gmail.com>>
Sent: Thursday, September 28, 2017 11:18:18 PM
To: Wren Turkal
Cc: python-ideas@python.org<mailto:python-ideas@python.org>
Subject: Re: [Python-ideas] allow overriding files used for the input builtin
I'm fine with the idea in general of extra keyword parameters to the input
function.
A few points:
Your example code, needs try/catch to match what the input with parameters does
-- and yes, its way nicer to be able to use it the example you have shown than
play games with try/catch (Personally I also refuse to ever change sys.stdin,
or sys.stdout, as I consider that a bad coding style).
Mostly though I would like to ask, please do not name keyword arguments with
names like 'fin' & 'fout'. This is almost unreadable and make's code almost
indecipherable to others the first time they see the function & its keyword
arguments (First impressions are very important).
Both a function name & its keyword parameters need to be as understandable as
possible when a user encounters them for the first time.
On Fri, Sep 29, 2017 at 1:53 AM, Wren Turkal <w...@fb.com<mailto:w...@fb.com>>
wrote:
Hi there,
I have posted an idea for improvement with a PR of an implementation to
https://bugs.python.org/issue31603<https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.python.org_issue31603&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=OAN5uLR4JWXbIgcvx315Zw&m=K1-OD0dhslOiAqYQdEyr0Oppl8TaroBPvXr8h_Z8XxM&s=sjDmBcI00MbWPPjuLMPlBnZHoFZOHoTxaCo2KCYlEd4&e=>.
The basic idea is to add fin, fout, and ferr file object parameters and default
to using what is used today when the args are not specified. I believe this
would be useful to allow captures input and send output to specific files when
using input. The input builtin has some logic to use readline if it's
available. It would be nice to be able to use this same logic no matter what
files are being used for input/output.
This is meant to turn code like the following:
orig_stdin = sys.stdin
orig_stdout = sys.stdout
with open('/dev/tty', 'r+') as f:
sys.stdin = f
sys.stdout = f
name = input('Name? ')
sys.stdin = orig_stdin
sys.stdout = orig_stdout
print(name)
into something more like this:
with open('/dev/tty', 'r+') as f:
name = input('Name? ', fin=f, fout=f)
print(name)
It's nice that it makes the assignment to a global variable to change the file
used for input/output to no longer be needed.
I had this idea the other day, and I realized that it would be super easy to
implement, so I went ahead the threw up a PR also.
Would love to see if anyone else is interested in this. I think it's pretty
cool that the core logic really didn't need to be changed other than plumbing
in the new args.
FWIW, this change introduces no regressions and adds a few more tests to test
the new functionality. Honestly, I think this functionality could probably be
used to simplify some of the other tests as well, but I wanted to gauge what
folks thought of the change before going farther.
Wren Turkal
Existential Production Engineer of the Ages
Facebook, Inc.
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org<mailto:Python-ideas@python.org>
https://mail.python.org/mailman/listinfo/python-ideas<https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailman_listinfo_python-2Dideas&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=OAN5uLR4JWXbIgcvx315Zw&m=K1-OD0dhslOiAqYQdEyr0Oppl8TaroBPvXr8h_Z8XxM&s=dvbtuL0kPe5UxakqosCnQQCIxlKprMP6JTqx4ZLXx4g&e=>
Code of Conduct:
http://python.org/psf/codeofconduct/<https://urldefense.proofpoint.com/v2/url?u=http-3A__python.org_psf_codeofconduct_&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=OAN5uLR4JWXbIgcvx315Zw&m=K1-OD0dhslOiAqYQdEyr0Oppl8TaroBPvXr8h_Z8XxM&s=px67QeYtdnaejiTP9VcY5IuSiJ4pk3XEXcbDzZWbsnI&e=>
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
