[PATCH] hg: copy buffer state to remote ui

[Elmar Bartel]

# HG changeset patch
# User Elmar Bartel 
# Date 1591826940 -7200
#  Thu Jun 11 00:09:00 2020 +0200
# Node ID 563268fce8e7502e17812fea7b90325a37c854f4
# Parent  83e41b73d115e3717943c2e5a83d36d05670384c
hg: copy buffer state to remote ui

When remoteui() creates a new ui, the buffer state of the
source ui should also be copied to the newly created ui.
Otherwise, when pushbuffer() was called on the source ui,
this "push" gets lost for the remote ui and output from the
remote side will go to default stdout but not to the pushed
buffer as is expected by the caller of pushbuffer().

diff -r 83e41b73d115 -r 563268fce8e7 mercurial/hg.py
--- a/mercurial/hg.py   Tue Jun 09 17:13:26 2020 -0400
+++ b/mercurial/hg.py   Thu Jun 11 00:09:00 2020 +0200
@@ -1362,6 +1362,12 @@ def remoteui(src, opts):
 if v:
 dst.setconfig(b'web', b'cacerts', util.expandpath(v), b'copied')
 
+# copy buffer state to the remote ui
+if src._buffers:
+dst._buffers = src._buffers
+dst._bufferstates = src._bufferstates
+dst._bufferapplylabels = src._bufferstates[-1][2]
+
 return dst
 
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] py3: make stdout line-buffered if connected to a TTY

[Augie Fackler]

> On Jun 10, 2020, at 09:47, Yuya Nishihara  wrote:
> 
> On Wed, 10 Jun 2020 14:28:09 +0200, Manuel Jacob wrote:
>> # HG changeset patch
>> # User Manuel Jacob 
>> # Date 1591786959 -7200
>> #  Wed Jun 10 13:02:39 2020 +0200
>> # Node ID 31cf564772d5f170a7c88267c92e632b32da28d7
>> # Parent  61719b9658b156548634c216ffb2b9188266293b
>> # EXP-Topic py3-line_buffered_stdout
>> py3: make stdout line-buffered if connected to a TTY
> 
> Nice. Queued for default, thanks.

A huge thanks to both of you for working through this nightmare.

> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@44924: new changeset (1 on stable)

[Mercurial Commits]

New changeset (1 on stable) in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/6c8384afbf77
changeset:   44924:6c8384afbf77
branch:  stable
tag: tip
user:Manuel Jacob 
date:Tue Jun 09 05:24:45 2020 +0200
summary: resourceutil: fix location of line comments

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8353: debugcommands: create new debugantivirusrunning command

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  In D8353#128838 , @durin42 wrote:
  
  > Sadly  I have no idea on that.
  
  Do we have a poor windows user that could serve an a guinea pig here ?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8353/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8353

To: durin42, #hg-reviewers, marmoute
Cc: mercurial-patches, marmoute, mharbison72, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8353: debugcommands: create new debugantivirusrunning command

[durin42 (Augie Fackler)]

durin42 added a comment.


  Sadly  I have no idea on that.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8353/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8353

To: durin42, #hg-reviewers, marmoute
Cc: mercurial-patches, marmoute, mharbison72, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] py3: make stdout line-buffered if connected to a TTY

[Yuya Nishihara]

On Wed, 10 Jun 2020 14:28:09 +0200, Manuel Jacob wrote:
> # HG changeset patch
> # User Manuel Jacob 
> # Date 1591786959 -7200
> #  Wed Jun 10 13:02:39 2020 +0200
> # Node ID 31cf564772d5f170a7c88267c92e632b32da28d7
> # Parent  61719b9658b156548634c216ffb2b9188266293b
> # EXP-Topic py3-line_buffered_stdout
> py3: make stdout line-buffered if connected to a TTY

Nice. Queued for default, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] py3: make stdout line-buffered if connected to a TTY

[Manuel Jacob]

# HG changeset patch
# User Manuel Jacob 
# Date 1591786959 -7200
#  Wed Jun 10 13:02:39 2020 +0200
# Node ID 31cf564772d5f170a7c88267c92e632b32da28d7
# Parent  61719b9658b156548634c216ffb2b9188266293b
# EXP-Topic py3-line_buffered_stdout
py3: make stdout line-buffered if connected to a TTY

Status messages that are to be shown on the terminal should be written to the
file descriptor before anything further is done, to keep the user updated.

One common way to achieve this is to make stdout line-buffered if it is
connected to a TTY. This is done on Python 2 (except on Windows, where libc,
which the CPython 2 streams depend on, does not properly support this).

Python 3 rolls it own I/O streams. On Python 3, buffered binary streams can't be
set line-buffered. The previous code (added in 227ba1afcb65) incorrectly
assumed that on Python 3, pycompat.stdout (sys.stdout.buffer) is already
line-buffered. However the interpreter initializes it with a block-buffered
stream or an unbuffered stream (when the -u option or the PYTHONUNBUFFERED
environment variable is set), never with a line-buffered stream.

One example where the current behavior is unacceptable is when running
`hg pull https://www.mercurial-scm.org/repo/hg` on Python 3, where the line
"pulling from https://www.mercurial-scm.org/repo/hg; does not appear on the
terminal before the hg process blocks while waiting for the server.

Various approaches to fix this problem are possible, including:

1. Weaken the contract of procutil.stdout to not give any guarantees about
   buffering behavior. In this case, users of procutil.stdout need to be
   changed to do enough flushes. In particular,
  1. either ui must insert enough flushes for ui.write() and friends, or
  2. ui.write() and friends get split into flushing and fully buffered
 methods, or
  3. users of ui.write() and friends must flush explicitly.
2. Make stdout unbuffered.
3. Make stdout line-buffered. Since Python 3 does not natively support that for
   binary streams, we must implement it ourselves.

(2.) is problematic because using unbuffered I/O changes the performance
characteristics significantly compared to line-buffered (which is used on
Python 2) and this would be a regression.

(1.2.) and (1.3) are a substantial amount of work. It’s unclear whether the
added complexity would be justified, given that raw performance doesn’t matter
that much when writing to a terminal much faster than the user could read it.

(1.1.) pushes complexity into the ui class instead of separating the concern of
how stdout is buffered. Other users of procutil.stdout would still need to take
care of the flushes.

This patch implements (3.). The general performance considerations are very
similar to (1.1.). The extra method invocation and method forwarding add a
little more overhead if the class is used. In exchange, it doesn’t add overhead
if not used.


For the benchmarks, I compared the previous implementation (incorrect on Python
3), (1.1.), (3.) and (2.). The command was chosen so that the streams were
configured as if they were writing to a TTY, but actually write to a pager,
which is also the default:
HGRCPATH=/dev/null python3 ./hg --cwd ~/vcs/mozilla-central --time --pager yes 
--config pager.pager='cat > /dev/null' status --all

previous:
time: real 7.880 secs (user 7.290+0.050 sys 0.580+0.170)
time: real 7.830 secs (user 7.220+0.070 sys 0.590+0.140)
time: real 7.800 secs (user 7.210+0.050 sys 0.570+0.170)

(1.1.) using Yuya Nishihara’s patch:
time: real 9.860 secs (user 8.670+0.350 sys 1.160+0.830)
time: real 9.540 secs (user 8.430+0.370 sys 1.100+0.770)
time: real 9.830 secs (user 8.630+0.370 sys 1.180+0.840)

(3.) using this patch:
time: real 9.580 secs (user 8.480+0.350 sys 1.090+0.770)
time: real 9.670 secs (user 8.480+0.330 sys 1.170+0.860)
time: real 9.640 secs (user 8.500+0.350 sys 1.130+0.810)

(2.) using a previous patch by me:
time: real 10.480 secs (user 8.850+0.720 sys 1.590+1.500)
time: real 10.490 secs (user 8.750+0.750 sys 1.710+1.470)
time: real 10.240 secs (user 8.600+0.700 sys 1.590+1.510)


As expected, there’s no difference on Python 2, as exactly the same code paths
are used:

previous:
time: real 6.950 secs (user 5.870+0.330 sys 1.070+0.770)
time: real 7.040 secs (user 6.040+0.360 sys 0.980+0.750)
time: real 7.070 secs (user 5.950+0.360 sys 1.100+0.760)

this patch:
time: real 7.010 secs (user 5.900+0.390 sys 1.070+0.730)
time: real 7.000 secs (user 5.850+0.350 sys 1.120+0.760)
time: real 7.000 secs (user 5.790+0.380 sys 1.170+0.710)

diff --git a/mercurial/utils/procutil.py b/mercurial/utils/procutil.py
--- a/mercurial/utils/procutil.py
+++ b/mercurial/utils/procutil.py
@@ -49,6 +49,25 @@
 return False
 
 
+if pycompat.ispy3:
+
+class LineBufferedWrapper(object):
+def __init__(self, orig):
+self.orig = orig
+
+def __getattr__(self, attr):
+return getattr(self.orig, attr)
+
+def write(self, s):
+orig = 

Re: [PATCH stable v2] py3: stop using block-buffered stdout to TTY on Python 3

[Yuya Nishihara]

On Wed, 10 Jun 2020 11:32:29 +0200, Manuel Jacob wrote:
> Yuya and me agreed that it is better to not use unbuffered I/O. See the
> discussion on v1 and subsequent improved approaches.

Yes. IMO, unbuffered I/O is the last option given how casually we do use
ui.write(). I have little concern about the speed of terminal output, which
is inherently slow or unreadable, but it shouldn't be unnecessarily slow if
a pager is attached.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 6353] New: hg help config should mention --edit

[mercurial-bugs]

https://bz.mercurial-scm.org/show_bug.cgi?id=6353

Bug ID: 6353
   Summary: hg help config should mention --edit
   Product: Mercurial
   Version: unspecified
  Hardware: PC
OS: Linux
Status: UNCONFIRMED
  Severity: feature
  Priority: wish
 Component: Mercurial
  Assignee: bugzi...@mercurial-scm.org
  Reporter: pierre-yves.da...@ens-lyon.org
CC: mercurial-devel@mercurial-scm.org
Python Version: ---

The --edit and --local flag are very useful but they are not advertised by the
config documentation.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH stable v2] py3: stop using block-buffered stdout to TTY on Python 3

[Manuel Jacob]

Yuya and me agreed that it is better to not use unbuffered I/O. See the 
discussion on v1 and subsequent improved approaches.


On 2020-06-09 23:24, Augie Fackler wrote:

Agreed, this LGTM. Yuya? I know you've been good at spotting issues in
this area that I miss.

On Jun 9, 2020, at 07:21, Pierre-Yves David 
 wrote:


Sigh, this seems like a good enough fix for now. We can rewrite the UI 
layer in rust later ;-)


On 6/7/20 12:55 AM, Manuel Jacob wrote:

# HG changeset patch
# User Manuel Jacob 
# Date 1591333053 -7200
#  Fri Jun 05 06:57:33 2020 +0200
# Branch stable
# Node ID d1754bf05591d2aee0f5f9dec185faf912a21190
# Parent  1f114c797961733ea03d0de88390f9555a7edef4
# EXP-Topic fix_block-buffered_stdout
py3: stop using block-buffered stdout to TTY on Python 3
The previous code (added in 227ba1afcb65) incorrectly assumed that
sys.stdout.buffer is line-buffered. However the interpreter 
initializes it
with a block-buffered stream or an unbuffered stream (when the -u 
option or
the PYTHONUNBUFFERED environment variable is set), never with a 
line-buffered

stream.
To see the difference with the hg command, you need to do something 
slow, e.g.
`hg pull https://www.mercurial-scm.org/repo/hg`. On Python 3 on 
Linux, without
this patch, all output is printed right before the command ends. With 
this

patch, output is printed almost immediately.
I did some quick and dirty benchmarking, printing on my terminal.
Unfavorable case:
* for 100 times:
  * for 1000 times:
* call `ui.write(b'x')`
  * call `ui.write(b'\n')`
* ui.flush()
Results:
CPython 2.7 is unaffected (line-buffered): wall 0.231795 comb 
0.23 user 0.23 sys 0.00
CPython 3.8 before patch (block-buffered): wall 0.190882 comb 
0.19 user 0.19 sys 0.00
CPython 3.8 after patch (unbuffered):  wall 0.654466 comb 
0.65 user 0.48 sys 0.17
CPython 3.8 hand-rolled line-buffered [*]: wall 0.218919 comb 
0.22 user 0.22 sys 0.00

Favorable case:
* for 10 times:
  * call `ui.write(b'Testing write performance\n')`
Results:
CPython 2.7 is unaffected (line-buffered): wall 0.717060 comb 
0.71 user 0.51 sys 0.20
CPython 3.8 before patch (block-buffered): wall 0.255527 comb 
0.26 user 0.18 sys 0.08
CPython 3.8 after patch (unbuffered):  wall 0.616500 comb 
0.62 user 0.45 sys 0.17
CPython 3.8 hand-rolled line-buffered [*]: wall 0.811805 comb 
0.81 user 0.57 sys 0.24
[*] For this, I implemented a simplified version of what is suggested 
in the
TODO: stdout was set to block-buffered and ui.write() flushed the 
stream if

the output contains a newline character.
diff -r 1f114c797961 -r d1754bf05591 mercurial/utils/procutil.py
--- a/mercurial/utils/procutil.py   Thu Apr 30 00:33:00 2020 -0400
+++ b/mercurial/utils/procutil.py   Fri Jun 05 06:57:33 2020 +0200
@@ -55,12 +55,20 @@
 # on Windows).
 # Python 3 rolls its own standard I/O streams.
 if isatty(stdout):
+# TODO: Consider mitigating the performance impact of using 
unbuffered
+# writes on Python 3 and Windows, e.g. by using fully buffered 
streams and

+# ensuring on a higher level that lines are flushed.
 if pycompat.iswindows:
 # Windows doesn't support line buffering
 stdout = os.fdopen(stdout.fileno(), 'wb', 0)
-elif not pycompat.ispy3:
-# on Python 3, stdout (sys.stdout.buffer) is already line 
buffered and

-# buffering=1 is not handled in binary mode
+elif pycompat.ispy3:
+# Python 3 doesn't support line buffering on binary streams. 
Use the
+# underlying unbuffered raw stream. If the ``raw`` attribute 
doesn't
+# exist, stdout is already unbuffered or replaced by a 
custom stream.
+# TODO: After Mercurial 5.4, use this code path on Python 3 
on Windows.
+# We shouldn't risk changing working behavior in the stable 
branch.

+stdout = getattr(stdout, 'raw', stdout)
+else:
 stdout = os.fdopen(stdout.fileno(), 'wb', 1)
   if pycompat.iswindows:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8353: debugcommands: create new debugantivirusrunning command

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  My main question here is : what does "AV engine would lose its lunch" 
translate in terms of the python runtime ?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8353/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8353

To: durin42, #hg-reviewers, marmoute
Cc: mercurial-patches, marmoute, mharbison72, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7177: rebase: introduce optional parent mapping

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  In D7177#128760 , @martinvonz 
wrote:
  
  > In D7177#128757 , @marmoute 
wrote:
  >
  >> In D7177#128756 , @martinvonz 
wrote:
  >>
  >>> In D7177#128740 , @marmoute 
wrote:
  >>>
   In D7177#128389 , 
@martinvonz wrote:
  
  > Maybe another option is to allow multiple `-d` arguments for this case? 
Something like `hg rebase -r C -d B -d D`. I haven't thought through BC, but I 
think that's what I'd prefer if we were writing rebase from scratch. I know we 
can't support `hg rebase -r C -d 'B + D'` for backward-compatibility reasons 
(because we already support that -- it rebases to the highest revnum in the 
set).
  
   That's interresting. How would that works with a practical case ?
  >>>
  >>> The example I have was meant to be a practical example for how to do it 
in the case given in the patch description :) But let me know if you meant some 
other aspect of "how it works". As I said, I'm not sure ant the BC bit of it at 
least.
  >>
  >> I meant how that would work in general, esepcially case more complicated 
than the basic base shown by @joerg.sonnenberger.
  >
  > I think the parents chosen by `-d` should only apply to the roots of the 
rebase set. If your rebase set has multiple roots and only some of them are 
merge commits, then we should probably error out.
  
  So you suggestion would be to allow to specify two -d that would definive the 
two new parents of the root ? I suspect it would be a bit too narrow compared 
to the varienty of case we might have to handle. Maybe this is a mission for 
PlanPlan.
  
  > Anyway, I'm not sure it's realistic to make it work that way now, 
especially because of how inconsistent it would be with other arguments that 
accept revsets and only care about the highest revnum in the set (I think that 
was a mistake to do, but I understand why it was done that way and it's 
obviously too late to change it).
  
  I think the multiple -d value approach would works fine. (but for the other 
issue).
  
  >> Not that in @joerg base, B being ancestors of both C and C', we could 
probably behave better by default directly. Could we not ?
  >
  > As @joerg said in the patch description, there's no reasonable way to 
choose which of A and B to preserve (the current version of rebase wouldn't 
preserve either and would instead rebase all of A, B, and C).
  
  The description got me confused, (I though C was rebased on C' (finding the 
naming strange)) while this is about C being rebased on D.

INLINE COMMENTS

> rebase.py:884
> +_(b'map old parent to new parent (ADVANCED)'),
> +_(b'REV:REV')),
>  (b'k', b'keep', False, _(b'keep original changesets')),

`REV:REV` is valid revset so the result is quite confusing, if we go that 
route, we need anothr syntax.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7177/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7177

To: joerg.sonnenberger, martinvonz, #hg-reviewers, baymax
Cc: mercurial-patches, marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel