On 2020-06-03 22:40, Augie Fackler wrote:
On Jun 3, 2020, at 07:13, Yuya Nishihara <y...@tcha.org> wrote:
On Wed, 03 Jun 2020 05:46:05 +0200, Manuel Jacob wrote:
On 2020-06-03 00:04, Augie Fackler wrote:
On Jun 2, 2020, at 14:07, Manuel Jacob <m...@manueljacob.de> wrote:
# HG changeset patch
# User Manuel Jacob <m...@manueljacob.de>
# Date 1591120869 -7200
# Tue Jun 02 20:01:09 2020 +0200
# Branch stable
# Node ID ebbc45544673c33ea3beb887ed4d5230b015102a
# Parent 91e509a12dbc4cd6cf2dcb9dae3ed383932132ac
py3: always flush ui streams on Python 3
On Python 2, we rely on that stdout and stderr are line buffered.
Python 3’s
io module doesn’t offer line buffered binary streams.
We use the underlying non-buffer thing though for std{err, out}
though, or so I thought. Are you observing behavior that this
corrects?
pycompat says:
stdout = sys.stdout.buffer
stderr = sys.stderr.buffer
They are <_io.BufferedWriter name='<stdout>'> and <_io.BufferedWriter
name='<stderr>'>, respectively.
I observed the error in TortoiseHg, which is a long-running process.
To trigger the problem with the hg command, you would need to do
something slow that prints on `self._fout`, e.g. `hg pull
https://www.mercurial-scm.org/repo/hg --debug`.
Denis, do you have any idea? My assumption was sys.stdout.buffer was
magically
line-buffered, but it appears not at least with Python 3.8.3 on Linux.
https://www.mercurial-scm.org/repo/hg/rev/227ba1afcb65
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1198,9 +1198,14 @@ class ui(object):
label = opts.get('label', b'')
msg = self.label(msg, label)
dest.write(msg)
+ # On Python 2, stdout and stderr are usually line
buffered, but
# stderr may be buffered under win32 when redirected to
files,
# including stdout.
- if dest is self._ferr and not getattr(self._ferr,
'closed', False):
+ # On Python 3, we use the underlying binary buffer,
which
does not
+ # support line buffering.
+ if (pycompat.ispy3 or dest is self._ferr) and not
getattr(
+ self._ferr, 'closed', False
+ ):
dest.flush()
Flushing stdout per write() sounds bad. If that's the only way to work
around Py3 io module, we'll probably need to check if the stdout is
a tty or not.
Agreed. Maybe we could sniff for a \n in the output stream, but I
dunno if that's likely to be a win.
For status output (in the general sense, not necessarily in the
ui.status() sense), flushing the stream would IMHO be fine: the messages
should be shown as fast as possible, often end with a '\n' (so the
stream would be flushed anyway in case of line buffering) and often use
string interpolation (therefore _writenobuf() gets called once per
message), and the amount of output is limited (so printing the lines
won’t dominate the total time spent).
For "data" output (like diffs), flushing the stream can indeed be
wasteful: showing single lines immediately is often not as important,
ui.write() is often called with small data (not complete lines), and the
amount of output can be large. Maybe we can introduce special methods
for this kind of data that uses fully buffered writes? Code that uses
these methods would need to flush manually.
Just sniffing for '\n', as Augie suggested, is pretty much what the io
module does when line buffering is enabled
(https://github.com/python/cpython/blob/v3.8.3/Modules/_io/textio.c#L1597).
Of course, we should do that only if the stream is interactive. I’m
concerned that calling isatty() on every write is too expensive, but I
need to benchmark this. BTW, there’s ui._isatty() that checks the
ui.nontty config. This config is not checked when setting sys.stdout to
line-buffered on Python 2 (by the interpreter), so am I right that we
should not care about this config?
BTW, I’m not sure I understand the comment about stderr on Windows. I
couldn't find any further information on this topic. Is it the case
that, when stderr is redirected to stdout on windows, Python fails to
recognize that stderr is actually interactive and doesn't set line
buffering?
Is there some command that calls ui.write() many times with small data
and doesn’t do much else, to be used as a benchmark?
_______________________________________________
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
