Re: [PATCH 8 of 9 pager v2] annotate: migrate to modern pager API
Excerpts from Yuya Nishihara's message of 2017-02-19 11:04:19 +0900:
> On Sat, 18 Feb 2017 15:39:08 -0500, Augie Fackler wrote:
> >
> > > On Feb 18, 2017, at 3:11 PM, Martin von Zweigbergk
> > > wrote:
> > >
> > >> > --- a/mercurial/commands.py
> > >> > +++ b/mercurial/commands.py
> > >> > @@ -361,6 +361,7 @@ def annotate(ui, repo, *pats, **opts):
> > >> >
> > >> > Returns 0 on success.
> > >> > """
> > >> > +ui.pager('annotate')
> > >> > if not pats:
> > >> > raise error.Abort(_('at least one filename or pattern is
> > >> > required'))
> > >>
> > >> Just to make sure. Do we plan to delay ui.pager() call so short error
> > >> messages
> > >> (and password prompt, etc.) won't be paged?
> > >>
> > > I was wondering the same, but was hoping the pager would be configured to
> > > exit if the full text would fit. I have no idea if that's true for most
> > > systems out there, though. I don't believe I've manually configured mine
> > > that way, so I suspect it's that way be default on our custom Ubuntu. I
> > > guess pager on by default will be pretty annoying if that is not a common
> > > default configuration, so I really hope it is.
> >
> > I don’t think that’s a safe assumption, sadly. I’ll do a follow-up to try
> > and make sure that things are sane at least at a high level.
>
> IIRC, "more" exits automatically, but "less" doesn't by default. Also, "less"
> clears the paged contents on exit.
less can be configured to not clear the screen (-X), and exit automatically
(-F). And another common flag is -R, which will allow colors.
git used to just hard-code LESS=FRX as the default in pager.c [1]. That got
moved to an override-able build variable by [2].
I guess we could do the same, provide "LESS=FRX" as the default to be more
user-friendly.
[1]:
https://github.com/git/git/blob/c3b1e8d85133e2a19d372b7c166d5b49fcbbfef2/pager.c#L70
[2]: https://github.com/git/git/commit/995bc22d7f8c611e342095a211065f8585a08e65
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 8 of 9 pager v2] annotate: migrate to modern pager API
On Sat, 18 Feb 2017 15:39:08 -0500, Augie Fackler wrote:
>
> > On Feb 18, 2017, at 3:11 PM, Martin von Zweigbergk
> > wrote:
> >
> >> > --- a/mercurial/commands.py
> >> > +++ b/mercurial/commands.py
> >> > @@ -361,6 +361,7 @@ def annotate(ui, repo, *pats, **opts):
> >> >
> >> > Returns 0 on success.
> >> > """
> >> > +ui.pager('annotate')
> >> > if not pats:
> >> > raise error.Abort(_('at least one filename or pattern is
> >> > required'))
> >>
> >> Just to make sure. Do we plan to delay ui.pager() call so short error
> >> messages
> >> (and password prompt, etc.) won't be paged?
> >>
> > I was wondering the same, but was hoping the pager would be configured to
> > exit if the full text would fit. I have no idea if that's true for most
> > systems out there, though. I don't believe I've manually configured mine
> > that way, so I suspect it's that way be default on our custom Ubuntu. I
> > guess pager on by default will be pretty annoying if that is not a common
> > default configuration, so I really hope it is.
>
> I don’t think that’s a safe assumption, sadly. I’ll do a follow-up to try and
> make sure that things are sane at least at a high level.
IIRC, "more" exits automatically, but "less" doesn't by default. Also, "less"
clears the paged contents on exit.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 2] smartset: convert set to list lazily
Excerpts from Martin von Zweigbergk's message of 2017-02-18 17:03:12 -0800: > @@ -205,5 +215,5 @@ class baseset(abstractsmartset): > > def __nonzero__(self): > -return bool(self._list) > +return bool(len(self)) > > Could delete method, I think. Just FYI. No, because abstractsmartset implements __nonzero__ to "raise NotImplementedError()". I think that's reasonable as it will force people to not forget overriding the __nonzero__ implementation, to make things more explicit. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] smartset: reorder initialization of baseset in more intuitive way
Looks good to me. Thanks for the clean-up! Excerpts from Yuya Nishihara's message of 2017-02-18 18:48:06 +0900: > # HG changeset patch > # User Yuya Nishihara > # Date 1487407072 -32400 > # Sat Feb 18 17:37:52 2017 +0900 > # Node ID 1ca3469fdd08c0d5d814a4bc359869bc157c7fc9 > # Parent 3f4bdcfe63e9f1e357187fa728ae0da9d35b3973 > smartset: reorder initialization of baseset in more intuitive way > > What we want to do is to assign either _set or _list per the given data > type. > > diff --git a/mercurial/smartset.py b/mercurial/smartset.py > --- a/mercurial/smartset.py > +++ b/mercurial/smartset.py > @@ -166,16 +166,14 @@ class baseset(abstractsmartset): > """ > self._ascending = None > self._istopo = istopo > -if not isinstance(data, list): > -if isinstance(data, set): > -self._set = data > -# set has no order we pick one for stability purpose > -self._ascending = True > -# converting set to list has a cost, do it lazily > -data = None > -else: > +if isinstance(data, set): > +# converting set to list has a cost, do it lazily > +self._set = data > +# set has no order we pick one for stability purpose > +self._ascending = True > +else: > +if not isinstance(data, list): > data = list(data) > -if data is not None: > self._list = data > self._datarepr = datarepr > ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2 v2] match: making visitdir() deal with non-recursive entries
On Feb 17, 2017 10:17 PM, "Yuya Nishihara" wrote:
On Mon, 13 Feb 2017 17:05:24 -0800, Rodrigo Damazio Bovendorp via
Mercurial-devel wrote:
> # HG changeset patch
> # User Rodrigo Damazio Bovendorp
> # Date 1487034194 28800
> # Mon Feb 13 17:03:14 2017 -0800
> # Node ID e90de197586d0749e64cef752613e6fe41d1c8e3
> # Parent 94264a6e6672c917d42518f7ae9322445868d067
> match: making visitdir() deal with non-recursive entries
> @@ -241,7 +250,7 @@
> return 'all'
> if dir in self._excluderoots:
> return False
> -if (self._includeroots and
> +if ((self._includeroots or self._includedirs != set(['.'])) and
> '.' not in self._includeroots and
> dir not in self._includeroots and
> dir not in self._includedirs and
Maybe we'll need to distinguish the explicitly-set 'rootfilesin:.' from
the default value. Since visitdir() is for optimization, this shouldn't be
a blocker, so queued.
Good point, rootfilesin:. doesn't currently get optimized, it seems.
Unrelated to this patch is that the optimization only applies to -I, not if
specified outside of that. I remember working on fixing that two years ago,
but I don't remember how far I got. Seems like it should be possible.
It would also be nice if the dirstate walk could take advantage of the
visitdir logic.
> @@ -642,11 +650,48 @@
> break
> root.append(p)
> r.append('/'.join(root) or '.')
> -elif kind in ('relpath', 'path', 'rootfilesin'):
> +elif kind in ('relpath', 'path'):
> r.append(pat or '.')
> +elif kind in ('rootfilesin'):
^^^
Fixed this as well.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 8 v4] mercurial: use best available timer for perf measurements
On Feb 16, 2017 10:48 AM, "Bryan O'Sullivan" wrote: On Thu, Feb 16, 2017 at 10:35 AM, Martin von Zweigbergk via Mercurial-devel wrote: > As discussed on #mercurial, Augie and I find "timer" confusing, > because it sounds like it returns a timer object, but in fact it just > returns a time. Can we have a followup patch that renames it to e.g. > perftime()? > If we're gonna bikeshed, it should be called time, because that's what it really emulates :-) Not quite, IIUC. On recent Pythons, it calls a function that returns a value that's useful for timing, but that's not a time since the epoch. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 2 V2] smartset: add some doctests
# HG changeset patch # User Jun Wu # Date 1487464207 28800 # Sat Feb 18 16:30:07 2017 -0800 # Node ID deb48622b857d621606a1cdda4adc868b1c663d4 # Parent 22722c297b33c2f68a8a34e0b0be0e6a9e0f4759 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r deb48622b857 smartset: add some doctests Add doctests explaining the set / list behavior. This will make the following changes more confident. diff --git a/mercurial/smartset.py b/mercurial/smartset.py --- a/mercurial/smartset.py +++ b/mercurial/smartset.py @@ -159,4 +159,49 @@ class baseset(abstractsmartset): Every method in this class should be implemented by any smartset class. + +This class could be constructed by an (unordered) set, or an (ordered) +list-like object. If a set is provided, it'll be sorted lazily. + +>>> x = [4, 0, 7, 6] +>>> y = [5, 6, 7, 3] + +Construct by a set: +>>> xs = baseset(set(x)) +>>> ys = baseset(set(y)) +>>> [list(i) for i in [xs + ys, xs & ys, xs - ys]] +[[0, 4, 6, 7, 3, 5], [6, 7], [0, 4]] +>>> [type(i).__name__ for i in [xs + ys, xs & ys, xs - ys]] +['addset', 'filteredset', 'filteredset'] + +Construct by a list-like: +>>> xs = baseset(x) +>>> ys = baseset(i for i in y) +>>> [list(i) for i in [xs + ys, xs & ys, xs - ys]] +[[4, 0, 7, 6, 5, 3], [7, 6], [4, 0]] +>>> [type(i).__name__ for i in [xs + ys, xs & ys, xs - ys]] +['addset', 'filteredset', 'filteredset'] + +Populate "_set" fields in the lists so set optimization may be used: +>>> [1 in xs, 3 in ys] +[False, True] + +Without sort(), results won't be changed: +>>> [list(i) for i in [xs + ys, xs & ys, xs - ys]] +[[4, 0, 7, 6, 5, 3], [7, 6], [4, 0]] +>>> [type(i).__name__ for i in [xs + ys, xs & ys, xs - ys]] +['addset', 'filteredset', 'filteredset'] + +With sort(): +>>> xs.sort(reverse=True) +>>> [list(i) for i in [xs + ys, xs & ys, xs - ys]] +[[7, 6, 4, 0, 5, 3], [7, 6], [4, 0]] +>>> [type(i).__name__ for i in [xs + ys, xs & ys, xs - ys]] +['addset', 'filteredset', 'filteredset'] + +>>> ys.sort() +>>> [list(i) for i in [xs + ys, xs & ys, xs - ys]] +[[7, 6, 4, 0, 3, 5], [7, 6], [4, 0]] +>>> [type(i).__name__ for i in [xs + ys, xs & ys, xs - ys]] +['addset', 'filteredset', 'filteredset'] """ def __init__(self, data=(), datarepr=None, istopo=False): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2 V2] smartset: use native set operations as fast paths
# HG changeset patch
# User Jun Wu
# Date 1487467423 28800
# Sat Feb 18 17:23:43 2017 -0800
# Node ID 6f57ef05c74567db95f42426499640ad29bc878f
# Parent deb48622b857d621606a1cdda4adc868b1c663d4
# Available At https://bitbucket.org/quark-zju/hg-draft
# hg pull https://bitbucket.org/quark-zju/hg-draft -r 6f57ef05c745
smartset: use native set operations as fast paths
For set operations like "&" and "-", where we know both basesets have their
sets ready, and the first set is sorted, use the native Python set
operations as a fast path.
Note: "+" is not optimized as that will break the ordering.
This leads to noticeable improvements on performance:
revset| before | after | delta
draft() & draft() & draft() & draft() |776 | 477 | -39%
draft() + draft() + draft() + draft() | 2849 | 2864 |
draft() - draft() + draft() - draft() |943 | 240 | -75%
draft() - draft() - draft() - draft() |557 | 197 | -64%
(time measured in microseconds)
diff --git a/mercurial/smartset.py b/mercurial/smartset.py
--- a/mercurial/smartset.py
+++ b/mercurial/smartset.py
@@ -172,5 +172,5 @@ class baseset(abstractsmartset):
[[0, 4, 6, 7, 3, 5], [6, 7], [0, 4]]
>>> [type(i).__name__ for i in [xs + ys, xs & ys, xs - ys]]
-['addset', 'filteredset', 'filteredset']
+['addset', 'baseset', 'baseset']
Construct by a list-like:
@@ -192,10 +192,10 @@ class baseset(abstractsmartset):
['addset', 'filteredset', 'filteredset']
-With sort():
+With sort(), set optimization could be used:
>>> xs.sort(reverse=True)
>>> [list(i) for i in [xs + ys, xs & ys, xs - ys]]
[[7, 6, 4, 0, 5, 3], [7, 6], [4, 0]]
>>> [type(i).__name__ for i in [xs + ys, xs & ys, xs - ys]]
-['addset', 'filteredset', 'filteredset']
+['addset', 'baseset', 'baseset']
>>> ys.sort()
@@ -203,5 +203,5 @@ class baseset(abstractsmartset):
[[7, 6, 4, 0, 3, 5], [7, 6], [4, 0]]
>>> [type(i).__name__ for i in [xs + ys, xs & ys, xs - ys]]
-['addset', 'filteredset', 'filteredset']
+['addset', 'baseset', 'baseset']
"""
def __init__(self, data=(), datarepr=None, istopo=False):
@@ -323,4 +323,20 @@ class baseset(abstractsmartset):
return None
+def _fastsetop(self, other, op):
+# try to use native set operations as fast paths
+if (type(other) is baseset and '_set' in other.__dict__ and '_set' in
+self.__dict__ and self._ascending is not None):
+s = baseset(data=getattr(self._set, op)(other._set))
+s._ascending = self._ascending
+else:
+s = getattr(super(baseset, self), op)(other)
+return s
+
+def __and__(self, other):
+return self._fastsetop(other, '__and__')
+
+def __sub__(self, other):
+return self._fastsetop(other, '__sub__')
+
def __repr__(self):
d = {None: '', False: '-', True: '+'}[self._ascending]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 2] smartset: convert set to list lazily
On Feb 17, 2017 10:02 PM, "Jun Wu" wrote: # HG changeset patch # User Jun Wu # Date 1487393969 28800 # Fri Feb 17 20:59:29 2017 -0800 # Node ID b32af6cafb9b958decb7ae0db7dd754aa5a5f883 # Parent 01eebb65a61d9edcad1665ed747c7092f1ddb8b9 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r b32af6cafb9b smartset: convert set to list lazily If the caller only wants to construct a baseset via a set, and then do "__contains__" tests. It's unnecessary to initialize the list. Testing on my unfiltered hg-committed repo where len(draft()) is 2600, this patch shows about 6% improvement on set intensive queries: Before: $ for i in `seq 5`; hg perfrevset 'draft() & draft() & draft() & draft()' ! wall 0.001196 comb 0.00 user 0.00 sys 0.00 (best of 2011) ! wall 0.001191 comb 0.00 user 0.00 sys 0.00 (best of 2099) ! wall 0.001186 comb 0.01 user 0.01 sys 0.00 (best of 1953) ! wall 0.001182 comb 0.00 user 0.00 sys 0.00 (best of 2135) ! wall 0.001193 comb 0.00 user 0.00 sys 0.00 (best of 2177) After: $ for i in `seq 5`; hg perfrevset 'draft() & draft() & draft() & draft()' ! wall 0.001128 comb 0.00 user 0.00 sys 0.00 (best of 2247) ! wall 0.001119 comb 0.00 user 0.00 sys 0.00 (best of 2317) ! wall 0.001115 comb 0.00 user 0.00 sys 0.00 (best of 2244) ! wall 0.001131 comb 0.00 user 0.00 sys 0.00 (best of 2093) ! wall 0.001124 comb 0.00 user 0.00 sys 0.00 (best of 2134) It could have bigger impact on larger sets in theory. diff --git a/mercurial/smartset.py b/mercurial/smartset.py --- a/mercurial/smartset.py +++ b/mercurial/smartset.py @@ -172,6 +172,10 @@ class baseset(abstractsmartset): # set has no order we pick one for stability purpose self._ascending = True -data = list(data) -self._list = data +# converting set to list has a cost, do it lazily +data = None +else: +data = list(data) +if data is not None: +self._list = data self._datarepr = datarepr @@ -186,4 +190,10 @@ class baseset(abstractsmartset): return asclist +@util.propertycache +def _list(self): +# _list is only lazily constructed if we have _set +assert '_set' in self.__dict__ +return list(self._set) + def __iter__(self): if self._ascending is None: @@ -205,5 +215,5 @@ class baseset(abstractsmartset): def __nonzero__(self): -return bool(self._list) +return bool(len(self)) Could delete method, I think. Just FYI. def sort(self, reverse=False): @@ -219,5 +229,8 @@ class baseset(abstractsmartset): def __len__(self): -return len(self._list) +if '_list' in self.__dict__: +return len(self._list) +else: +return len(self._set) def isascending(self): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH py3] ui: construct _keepalnum list in a python3-friendly way
On 16 Feb 2017, at 16:35, Augie Fackler mailto:r...@durin42.com>> wrote:
> +if pycompat.ispy3:
> +_unicodes = [bytes([c]).decode('latin1') for c in range(256)]
> +_notalnum = [s.encode('latin1') for s in _unicodes if not s.isalnum()]
...
> +_keepalnum = ''.join(_notalnum)
This could be more cheaply calculated as
_keepalnum = bytes(c for c in range(256) if not chr(c).isalnum())
This takes a third of the time.
Martijn___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 8 of 9 pager v2] annotate: migrate to modern pager API
> On Feb 18, 2017, at 3:11 PM, Martin von Zweigbergk
> wrote:
>
>> > --- a/mercurial/commands.py
>> > +++ b/mercurial/commands.py
>> > @@ -361,6 +361,7 @@ def annotate(ui, repo, *pats, **opts):
>> >
>> > Returns 0 on success.
>> > """
>> > +ui.pager('annotate')
>> > if not pats:
>> > raise error.Abort(_('at least one filename or pattern is
>> > required'))
>>
>> Just to make sure. Do we plan to delay ui.pager() call so short error
>> messages
>> (and password prompt, etc.) won't be paged?
>>
> I was wondering the same, but was hoping the pager would be configured to
> exit if the full text would fit. I have no idea if that's true for most
> systems out there, though. I don't believe I've manually configured mine that
> way, so I suspect it's that way be default on our custom Ubuntu. I guess
> pager on by default will be pretty annoying if that is not a common default
> configuration, so I really hope it is.
I don’t think that’s a safe assumption, sadly. I’ll do a follow-up to try and
make sure that things are sane at least at a high level.
AF
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 8 of 9 pager v2] annotate: migrate to modern pager API
On Feb 18, 2017 03:18, "Yuya Nishihara" wrote:
On Thu, 16 Feb 2017 11:59:17 -0500, Augie Fackler wrote:
> # HG changeset patch
> # User Augie Fackler
> # Date 1486439567 18000
> # Mon Feb 06 22:52:47 2017 -0500
> # Node ID 4c2f3f1b67a71401faff082dbca79a3f212b5499
> # Parent 935dccd057eb61b980901b5c7a806fac9fb55d0f
> annotate: migrate to modern pager API
>
> diff --git a/hgext/pager.py b/hgext/pager.py
> --- a/hgext/pager.py
> +++ b/hgext/pager.py
> @@ -110,4 +110,4 @@ def uisetup(ui):
> extensions.wrapfunction(dispatch, '_runcommand', pagecmd)
> extensions.afterloaded('color', afterloaded)
>
> -attended = ['annotate', 'cat', 'diff', 'export', 'glog', 'log', 'qdiff']
> +attended = ['cat', 'diff', 'export', 'glog', 'log', 'qdiff']
> diff --git a/mercurial/commands.py b/mercurial/commands.py
> --- a/mercurial/commands.py
> +++ b/mercurial/commands.py
> @@ -361,6 +361,7 @@ def annotate(ui, repo, *pats, **opts):
>
> Returns 0 on success.
> """
> +ui.pager('annotate')
> if not pats:
> raise error.Abort(_('at least one filename or pattern is
required'))
Just to make sure. Do we plan to delay ui.pager() call so short error
messages
(and password prompt, etc.) won't be paged?
I was wondering the same, but was hoping the pager would be configured to
exit if the full text would fit. I have no idea if that's true for most
systems out there, though. I don't believe I've manually configured mine
that way, so I suspect it's that way be default on our custom Ubuntu. I
guess pager on by default will be pretty annoying if that is not a common
default configuration, so I really hope it is.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2] chg: deduplicate error handling of ui.system()
# HG changeset patch # User Yuya Nishihara # Date 1487434605 -32400 # Sun Feb 19 01:16:45 2017 +0900 # Node ID d38553996c178b051dd8ed9aa99149b3952d09d3 # Parent c6dd20729bb3fd2f7745139c27488ce7279f13e1 chg: deduplicate error handling of ui.system() This moves 'onerr' handling from low-level util.system() to higher level, which seems better API separation. diff --git a/mercurial/chgserver.py b/mercurial/chgserver.py --- a/mercurial/chgserver.py +++ b/mercurial/chgserver.py @@ -179,7 +179,7 @@ def _newchgui(srcui, csystem, attachio): else: self._csystem = csystem -def _runsystem(self, cmd, environ, cwd, onerr, errprefix, out): +def _runsystem(self, cmd, environ, cwd, out): # fallback to the original system method if the output needs to be # captured (to self._buffers), or the output stream is not stdout # (e.g. stderr, cStringIO), because the chg client is not aware of @@ -187,17 +187,9 @@ def _newchgui(srcui, csystem, attachio): if (out is not self.fout or not util.safehasattr(self.fout, 'fileno') or self.fout.fileno() != util.stdout.fileno()): -return util.system(cmd, environ=environ, cwd=cwd, onerr=onerr, - errprefix=errprefix, out=out) +return util.system(cmd, environ=environ, cwd=cwd, out=out) self.flush() -rc = self._csystem(cmd, util.shellenviron(environ), cwd) -if rc and onerr: -errmsg = '%s %s' % (os.path.basename(cmd.split(None, 1)[0]), -util.explainexit(rc)[0]) -if errprefix: -errmsg = '%s: %s' % (errprefix, errmsg) -raise onerr(errmsg) -return rc +return self._csystem(cmd, util.shellenviron(environ), cwd) def _runpager(self, cmd): self._csystem(cmd, util.shellenviron(), type='pager', diff --git a/mercurial/ui.py b/mercurial/ui.py --- a/mercurial/ui.py +++ b/mercurial/ui.py @@ -1246,6 +1246,9 @@ class ui(object): blockedtag=None): '''execute shell command with appropriate output stream. command output will be redirected if fout is not stdout. + +if command fails and onerr is None, return status, else raise onerr +object as exception. ''' if blockedtag is None: blockedtag = 'unknown_system_' + cmd.translate(None, _keepalnum) @@ -1253,14 +1256,19 @@ class ui(object): if any(s[1] for s in self._bufferstates): out = self with self.timeblockedsection(blockedtag): -return self._runsystem(cmd, environ=environ, cwd=cwd, onerr=onerr, - errprefix=errprefix, out=out) +rc = self._runsystem(cmd, environ=environ, cwd=cwd, out=out) +if rc and onerr: +errmsg = '%s %s' % (os.path.basename(cmd.split(None, 1)[0]), +util.explainexit(rc)[0]) +if errprefix: +errmsg = '%s: %s' % (errprefix, errmsg) +raise onerr(errmsg) +return rc -def _runsystem(self, cmd, environ, cwd, onerr, errprefix, out): +def _runsystem(self, cmd, environ, cwd, out): """actually execute the given shell command (can be overridden by extensions like chg)""" -return util.system(cmd, environ=environ, cwd=cwd, onerr=onerr, - errprefix=errprefix, out=out) +return util.system(cmd, environ=environ, cwd=cwd, out=out) def traceback(self, exc=None, force=False): '''print exception traceback if traceback printing enabled or forced. diff --git a/mercurial/util.py b/mercurial/util.py --- a/mercurial/util.py +++ b/mercurial/util.py @@ -1011,20 +1011,16 @@ def shellenviron(environ=None): env['HG'] = hgexecutable() return env -def system(cmd, environ=None, cwd=None, onerr=None, errprefix=None, out=None): +def system(cmd, environ=None, cwd=None, out=None): '''enhanced shell command execution. run with environment maybe modified, maybe in different dir. -if command fails and onerr is None, return status, else raise onerr -object as exception. - if out is specified, it is assumed to be a file-like object that has a write() method. stdout and stderr will be redirected to out.''' try: stdout.flush() except Exception: pass -origcmd = cmd cmd = quotecommand(cmd) if pycompat.sysplatform == 'plan9' and (sys.version_info[0] == 2 and sys.version_info[1] < 7): @@ -1048,12 +1044,6 @@ def system(cmd, environ=None, cwd=None, rc = proc.returncode if pycompat.sysplatform == 'OpenVMS' and rc & 1: rc = 0 -if rc and onerr: -errmsg = '%s %s' % (os.path.basename(origcmd.split(None
[PATCH 1 of 2] chg: refactor ui.system() to be partly overridden
# HG changeset patch
# User Yuya Nishihara
# Date 1487433610 -32400
# Sun Feb 19 01:00:10 2017 +0900
# Node ID c6dd20729bb3fd2f7745139c27488ce7279f13e1
# Parent 22722c297b33c2f68a8a34e0b0be0e6a9e0f4759
chg: refactor ui.system() to be partly overridden
Since fd598149112b changed the signature of ui.system(), chgui.system()
should have been updated. This patch factors out the util.system() call
so that chg can override how a shell command is executed.
diff --git a/mercurial/chgserver.py b/mercurial/chgserver.py
--- a/mercurial/chgserver.py
+++ b/mercurial/chgserver.py
@@ -179,17 +179,16 @@ def _newchgui(srcui, csystem, attachio):
else:
self._csystem = csystem
-def system(self, cmd, environ=None, cwd=None, onerr=None,
- errprefix=None):
+def _runsystem(self, cmd, environ, cwd, onerr, errprefix, out):
# fallback to the original system method if the output needs to be
# captured (to self._buffers), or the output stream is not stdout
# (e.g. stderr, cStringIO), because the chg client is not aware of
# these situations and will behave differently (write to stdout).
-if (any(s[1] for s in self._bufferstates)
+if (out is not self.fout
or not util.safehasattr(self.fout, 'fileno')
or self.fout.fileno() != util.stdout.fileno()):
-return super(chgui, self).system(cmd, environ, cwd, onerr,
- errprefix)
+return util.system(cmd, environ=environ, cwd=cwd, onerr=onerr,
+ errprefix=errprefix, out=out)
self.flush()
rc = self._csystem(cmd, util.shellenviron(environ), cwd)
if rc and onerr:
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1253,8 +1253,14 @@ class ui(object):
if any(s[1] for s in self._bufferstates):
out = self
with self.timeblockedsection(blockedtag):
-return util.system(cmd, environ=environ, cwd=cwd, onerr=onerr,
- errprefix=errprefix, out=out)
+return self._runsystem(cmd, environ=environ, cwd=cwd, onerr=onerr,
+ errprefix=errprefix, out=out)
+
+def _runsystem(self, cmd, environ, cwd, onerr, errprefix, out):
+"""actually execute the given shell command (can be overridden by
+extensions like chg)"""
+return util.system(cmd, environ=environ, cwd=cwd, onerr=onerr,
+ errprefix=errprefix, out=out)
def traceback(self, exc=None, force=False):
'''print exception traceback if traceback printing enabled or forced.
diff --git a/tests/test-chg.t b/tests/test-chg.t
--- a/tests/test-chg.t
+++ b/tests/test-chg.t
@@ -32,6 +32,46 @@ long socket path
$ cd ..
+editor
+--
+
+ $ cat >> pushbuffer.py < def reposetup(ui, repo):
+ > repo.ui.pushbuffer(subproc=True)
+ > EOF
+
+ $ chg init editor
+ $ cd editor
+
+by default, system() should be redirected to the client:
+
+ $ touch foo
+ $ CHGDEBUG= HGEDITOR=cat chg ci -Am channeled --edit 2>&1 \
+ > | egrep "HG:|run 'cat"
+ chg: debug: run 'cat "*"' at '$TESTTMP/editor' (glob)
+ HG: Enter commit message. Lines beginning with 'HG:' are removed.
+ HG: Leave message empty to abort commit.
+ HG: --
+ HG: user: test
+ HG: branch 'default'
+ HG: added foo
+
+but no redirection should be made if output is captured:
+
+ $ touch bar
+ $ CHGDEBUG= HGEDITOR=cat chg ci -Am bufferred --edit \
+ > --config extensions.pushbuffer="$TESTTMP/pushbuffer.py" 2>&1 \
+ > | egrep "HG:|run 'cat"
+ [1]
+
+check that commit commands succeeded:
+
+ $ hg log -T '{rev}:{desc}\n'
+ 1:bufferred
+ 0:channeled
+
+ $ cd ..
+
pager
-
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: RFC: bitmap storage for precursors and phases
Excerpts from Jun Wu's message of 2017-02-17 19:14:12 -0800: > Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: > > On Fri, Feb 17, 2017 at 10:30 AM, Jun Wu wrote: > > > > > Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > > > > As I said before we will load all non-public revs in one set and all > > > > > > The problem is, loading a Python set from disk is O(size-of-the-set). > > > > > > Bitmap's loading cost should be basically 0 (with mmap). I think that's > > > why > > > we want bitmap at the first place. There are other choices like packfile > > > index, hash tables, but bitmap is the simplest and most efficient. > > > > > > > Hey folks, > > > > I haven't yet seen mention of some considerations that seem very important > > in driving the decision-making, so I'd appreciate it if someone could fill > > me in. > > > > Firstly, what's our current understanding of the sizes and compositions of > > these sets of numbers? In theory, we have a lot of data from practical > > application at Facebook, but nobody's brought it up. > > > > To clarify why this matters, if an obsstore contains say 100 entries, then > > a very naive implementation should be fine. If it's related to something > > else, such as the size of, amount of activity in, or local age of a repo, > > then maybe we have a different set of decisions to make. > > First, I agree that if N is small, O(N) or O(log(N)), or O(1), do not matter > much in practice. I tend to always reason about future-proof solutions, > which may or may not be a good habit - I'll adjust in the future. > > I think there are multiple topics being discussed: > > 1. How to solve the overhead loading hiddenrevs with minimal changes? > 2. Why is the bitmap format interesting (future use-cases)? > > For 1, > > I once tried to make one bit of revlog flag as the source of truth of > "hidden", which looks a nice solution. But it got rejected because revlog > has to be append-only. Moving the flag bits out formed the bitmap idea > naturally. > > Bitmap fits here but it does not mean we have to use it to solve this > particular problem. For example, stateful chg will get us some easy wins, > and see the problem analysis and proposed solution below. > > Note that we actually have a simple caching format for the "hiddenrevs". > See repoview.py, computehidden. > > But what's wrong with "computehidden" is the choice of the cache key - it > calls "hideablerevs", which takes 0.5 seconds in obsolete.getrevs(repo, > 'obsolete'), which reads the giant obsstore (in hg-committed) and do some > complex computation. The result is then hashed and used as a cache key of > "computehidden". > > So, if we do not want to know the "obsolete()" revisions, but just want to > know what's hidden. Changing the cache key would be a very straightforward > and effective solution which does not require a new file format. Hm, changing the cache key to smth like phaseroots + obsstore + changelog mtime + size may be an easy and good win. That's a good idea. Although it will still get invalidated after each write. > > That'll help commands like "hg st", "hg log -T foo", etc. But it won't > help complex "log"s which do require the "obsolete()" revisions. To speed > up the latter, we can probably just copy the caching infrastructure from > repoview.py. > > For 2, > > len(filteredrevs) in my hg-committed is 2155. It's small. I tend to think > about solutions that scale longer and are not too complex to build. That > may or may not be a good habit. > > That said, I agree if the set is small and perf, then whatever easier to > implement makes sense. > > I have some crazy ideas about the future where a giant bitmap is > practically useful. For example, source controlling "visible heads" and > the obsstore, then adding some C code to quickly fill a bitmap (requires > "set" and "setancestors/descendants"), then we can show the user whenever > their repo used to look like back in any time by using the bitmap filter. > That'll help investigate hard issues, and implement "hg undo". > > > Secondly, what operations are being performed on these sets today, and do > > we know if those operations are going to change? > > > > For instance, if a may be extended via new elements added to its end, then > > an mmap-based implementation immediately becomes awkward. If a set is > > Not that bad if we preallocate space. > > > traversed or we perform operations like union or intersection, then a > > bitmap might make sense if the set is very small and dense, but not if it's > > big and sparse. > > Not necessarily true if we use an advanced variant, like the "roaringbitmap" > mentioned by you last year [1]. > > [1]: > https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-September/087864.html > > > (Jun's idea that mmap is effectively free is also not borne out in > > practice, but that's
Re: [PATCH 2 of 3 V2] revset: use phasecache.getrevset
On Sat, 18 Feb 2017 00:56:09 -0800, Jun Wu wrote:
> # HG changeset patch
> # User Jun Wu
> # Date 1487407171 28800
> # Sat Feb 18 00:39:31 2017 -0800
> # Node ID 432992303718ec631b81a351795071c34cfe68f6
> # Parent 7e30a7c38d845e9a526a7ac64c648f18ec51687e
> # Available At https://bitbucket.org/quark-zju/hg-draft
> # hg pull https://bitbucket.org/quark-zju/hg-draft -r
> 432992303718
> revset: use phasecache.getrevset
> For the pure Python implementation, some operations are faster while "not
> draft()" is noticeably slower:
>
> revset | public() | not public() | draft() | not draft()
> hidden | yes | no | yes | no | yes | no | yes | no
>
> before | 18852 | 17183 | 17758 | 15921 | 17505 | 15973 | 41521 | 39822
> after | 18924 | 17380 | 17558 | 14545 | 16727 | 13593 | 48356 | 43992
> delta | | -9% | -5% | -15% | +16% | +10%
>
> That may be the different performance characters of generatorset vs.
> filteredset. The "not draft()" query could be optimized in this case where
> both "public" and "secret" are passed to "getrevsets" so it won't iterate
> the whole repo twice.
Yeah, generatorset is somewhat costly for membership tests. Instead, phasecache
could return a filteredset, but it wouldn't be always faster.
s = smartset.fullreposet(repo)
s.filter(condition, condrepr=('', phases), cache=False)
I see no regression in contrib/base-revsets.txt, and the performance difference
is only seen in pure build, so this series seems good. Queued, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@31011: 13 new changesets
13 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/334cf948c758 changeset: 30999:334cf948c758 user:Augie Fackler date:Mon Feb 06 22:52:47 2017 -0500 summary: annotate: migrate to modern pager API https://www.mercurial-scm.org/repo/hg/rev/84ec2d6a2831 changeset: 31000:84ec2d6a2831 user:Augie Fackler date:Mon Feb 06 23:22:04 2017 -0500 summary: tests: prove that ignore works https://www.mercurial-scm.org/repo/hg/rev/e7eca6e1372e changeset: 31001:e7eca6e1372e user:Rainer Woitok date:Fri Feb 17 11:08:36 2017 +0100 summary: doc: correct example concerning "hg purge" alias in man page "hgrc.5" https://www.mercurial-scm.org/repo/hg/rev/a489ee9b2852 changeset: 31002:a489ee9b2852 user:Jun Wu date:Fri Feb 17 00:59:09 2017 -0800 summary: runtests: prefer IPv4 to IPv6 https://www.mercurial-scm.org/repo/hg/rev/225f574e0645 changeset: 31003:225f574e0645 user:Jun Wu date:Wed Feb 15 21:03:42 2017 -0800 summary: runtests: export HGIPV6 to hint test scripts whether to use IPv6 https://www.mercurial-scm.org/repo/hg/rev/d05fefbb5ab3 changeset: 31004:d05fefbb5ab3 user:Jun Wu date:Wed Feb 15 21:09:00 2017 -0800 summary: dumbhttp: use IPv6 if HGIPV6 is set to 1 https://www.mercurial-scm.org/repo/hg/rev/d8d698bcdcd6 changeset: 31005:d8d698bcdcd6 user:Jun Wu date:Wed Feb 15 22:53:45 2017 -0800 summary: tinyproxy: use IPv6 if HGIPV6 is set to 1 https://www.mercurial-scm.org/repo/hg/rev/d4916aebf3d0 changeset: 31006:d4916aebf3d0 user:Jun Wu date:Thu Feb 16 08:01:19 2017 -0800 summary: runtests: export LOCALIP https://www.mercurial-scm.org/repo/hg/rev/bfdb0741f9f2 changeset: 31007:bfdb0741f9f2 user:Jun Wu date:Wed Feb 15 23:24:03 2017 -0800 summary: dummyssh: use LOCALIP https://www.mercurial-scm.org/repo/hg/rev/636cf3f7620d changeset: 31008:636cf3f7620d user:Jun Wu date:Thu Feb 16 09:38:52 2017 -0800 summary: tests: use LOCALIP https://www.mercurial-scm.org/repo/hg/rev/161ab32b44a1 changeset: 31009:161ab32b44a1 user:Jun Wu date:Thu Feb 16 00:13:29 2017 -0800 summary: runtests: set web.address to localhost https://www.mercurial-scm.org/repo/hg/rev/09ec648cd2a9 changeset: 31010:09ec648cd2a9 user:Jun Wu date:Thu Feb 16 08:43:59 2017 -0800 summary: runtests: always set web.ipv6 https://www.mercurial-scm.org/repo/hg/rev/01eebb65a61d changeset: 31011:01eebb65a61d bookmark:@ tag: tip user:Jun Wu date:Fri Feb 17 01:21:15 2017 -0800 summary: runtests: add an IPv6 command line flag -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 07 of 10 ipv6 V2] tests: use LOCALIP
On Fri, 17 Feb 2017 10:51:21 -0800, Jun Wu wrote: > # HG changeset patch > # User Jun Wu > # Date 1487266732 28800 > # Thu Feb 16 09:38:52 2017 -0800 > # Node ID 1ea3da89ecc44b59e96cb366e6388c1ac178d3af > # Parent bb7f41b2c8043a3115f242fb9b9f71ee461c93a9 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > 1ea3da89ecc4 > tests: use LOCALIP > > This patch replaces hardcoded 127.0.0.1 with $LOCALIP in all tests. > > Till now, the IPv6 series should make tests pass on common IPv6 systems > where the local device has the address "::1" and the hostname "localhost" > resolves to "::1". > - $ hg clone http://127.0.0.1:3121/a b > + $ hg clone http://$LOCALIP:3121/a b I'm a little surprised that "http://::1:3121/"; just works thanks to ":3121". Perhaps it's invalid and should be "http://[::1]:3121/";. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 8 of 9 pager v2] annotate: migrate to modern pager API
On Thu, 16 Feb 2017 11:59:17 -0500, Augie Fackler wrote:
> # HG changeset patch
> # User Augie Fackler
> # Date 1486439567 18000
> # Mon Feb 06 22:52:47 2017 -0500
> # Node ID 4c2f3f1b67a71401faff082dbca79a3f212b5499
> # Parent 935dccd057eb61b980901b5c7a806fac9fb55d0f
> annotate: migrate to modern pager API
>
> diff --git a/hgext/pager.py b/hgext/pager.py
> --- a/hgext/pager.py
> +++ b/hgext/pager.py
> @@ -110,4 +110,4 @@ def uisetup(ui):
> extensions.wrapfunction(dispatch, '_runcommand', pagecmd)
> extensions.afterloaded('color', afterloaded)
>
> -attended = ['annotate', 'cat', 'diff', 'export', 'glog', 'log', 'qdiff']
> +attended = ['cat', 'diff', 'export', 'glog', 'log', 'qdiff']
> diff --git a/mercurial/commands.py b/mercurial/commands.py
> --- a/mercurial/commands.py
> +++ b/mercurial/commands.py
> @@ -361,6 +361,7 @@ def annotate(ui, repo, *pats, **opts):
>
> Returns 0 on success.
> """
> +ui.pager('annotate')
> if not pats:
> raise error.Abort(_('at least one filename or pattern is required'))
Just to make sure. Do we plan to delay ui.pager() call so short error messages
(and password prompt, etc.) won't be paged?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@30998: 17 new changesets
17 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/08e6f4dac2ca changeset: 30982:08e6f4dac2ca user:Simon Farnsworth date:Wed Feb 15 13:34:06 2017 -0800 summary: extdiff: log time spent in external diff program https://www.mercurial-scm.org/repo/hg/rev/d4825798818b changeset: 30983:d4825798818b user:Simon Farnsworth date:Wed Feb 15 13:34:06 2017 -0800 summary: histedit: log the time taken to read in the commands list https://www.mercurial-scm.org/repo/hg/rev/15f9084a9a0c changeset: 30984:15f9084a9a0c user:Jun Wu date:Wed Feb 15 16:18:31 2017 -0800 summary: runtests: add a function to test if IPv6 is available https://www.mercurial-scm.org/repo/hg/rev/1f803482844a changeset: 30985:1f803482844a user:Jun Wu date:Wed Feb 15 16:22:22 2017 -0800 summary: runtests: checkportisavailable should only check one family https://www.mercurial-scm.org/repo/hg/rev/f07ca071a058 changeset: 30986:f07ca071a058 user:Jun Wu date:Wed Feb 15 16:43:27 2017 -0800 summary: runtests: set web.ipv6 if we use IPv6 https://www.mercurial-scm.org/repo/hg/rev/1ee685defe80 changeset: 30987:1ee685defe80 user:Jun Wu date:Wed Feb 15 16:29:58 2017 -0800 summary: runtests: unindent an "if True" block https://www.mercurial-scm.org/repo/hg/rev/d194f0dba7ac changeset: 30988:d194f0dba7ac user:Pulkit Goyal <7895pul...@gmail.com> date:Thu Feb 16 17:30:35 2017 +0530 summary: py3: convert the mode argument of os.fdopen to unicodes https://www.mercurial-scm.org/repo/hg/rev/74af89c66834 changeset: 30989:74af89c66834 user:Jun Wu date:Wed Feb 15 19:41:14 2017 -0800 summary: localrepo: move extension loading to a separate method https://www.mercurial-scm.org/repo/hg/rev/cb899ee133d8 changeset: 30990:cb899ee133d8 user:Jun Wu date:Thu Feb 16 23:10:47 2017 -0800 summary: chgserver: move comments in config example https://www.mercurial-scm.org/repo/hg/rev/3844b3299a53 changeset: 30991:3844b3299a53 user:Jun Wu date:Thu Feb 16 10:33:59 2017 -0800 summary: test-logtoprocess: use cat to wait for outputs https://www.mercurial-scm.org/repo/hg/rev/61b4122019d3 changeset: 30992:61b4122019d3 user:Augie Fackler date:Wed Feb 15 17:47:51 2017 -0500 summary: pager: move pager-initiating code into core https://www.mercurial-scm.org/repo/hg/rev/9c2977ceaa46 changeset: 30993:9c2977ceaa46 user:Augie Fackler date:Wed Feb 15 17:47:57 2017 -0500 summary: pager: move more behavior into core https://www.mercurial-scm.org/repo/hg/rev/3ed6e43998df changeset: 30994:3ed6e43998df user:Augie Fackler date:Wed Feb 15 17:48:03 2017 -0500 summary: ui: introduce neverpager() call https://www.mercurial-scm.org/repo/hg/rev/5e85bab867a7 changeset: 30995:5e85bab867a7 user:Augie Fackler date:Mon Feb 06 23:21:45 2017 -0500 summary: ui: add ignore-single-command functionality https://www.mercurial-scm.org/repo/hg/rev/cc3c9b6f1e09 changeset: 30996:cc3c9b6f1e09 user:Augie Fackler date:Mon Feb 06 23:45:30 2017 -0500 summary: tests: clean up a bunch of pager testing that is about to be invalidated https://www.mercurial-scm.org/repo/hg/rev/29a4a8d01bc9 changeset: 30997:29a4a8d01bc9 user:Augie Fackler date:Mon Feb 06 23:42:04 2017 -0500 summary: ui: respect historic pager.attend-$COMMAND=no https://www.mercurial-scm.org/repo/hg/rev/75e325ce538e changeset: 30998:75e325ce538e bookmark:@ tag: tip user:Augie Fackler date:Mon Feb 06 23:55:54 2017 -0500 summary: ui: add a debug print right before we start the pager -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@30981: 11 new changesets
11 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/bb6385882cfa changeset: 30971:bb6385882cfa user:Pierre-Yves David date:Thu Dec 22 02:34:22 2016 +0100 summary: color: move configstyles into the core module https://www.mercurial-scm.org/repo/hg/rev/a3c7e42c7a1f changeset: 30972:a3c7e42c7a1f user:Pierre-Yves David date:Thu Dec 22 02:37:18 2016 +0100 summary: color: move '_effect_str' function into the core module https://www.mercurial-scm.org/repo/hg/rev/e5363cb96233 changeset: 30973:e5363cb96233 user:Pierre-Yves David date:Thu Dec 22 02:38:53 2016 +0100 summary: color: move the '_render_effects' function to the core module https://www.mercurial-scm.org/repo/hg/rev/ae5d60bb70c9 changeset: 30974:ae5d60bb70c9 user:Simon Farnsworth date:Wed Feb 15 11:53:59 2017 -0800 summary: util: introduce timer() https://www.mercurial-scm.org/repo/hg/rev/22fbca1d11ed changeset: 30975:22fbca1d11ed user:Simon Farnsworth date:Wed Feb 15 13:17:39 2017 -0800 summary: mercurial: switch to util.timer for all interval timings https://www.mercurial-scm.org/repo/hg/rev/e92daf156d5c changeset: 30976:e92daf156d5c user:Simon Farnsworth date:Wed Feb 15 13:17:45 2017 -0800 summary: ui: provide a mechanism to track and log blocked time https://www.mercurial-scm.org/repo/hg/rev/5a9e4dc8e4fd changeset: 30977:5a9e4dc8e4fd user:Simon Farnsworth date:Wed Feb 15 13:07:26 2017 -0800 summary: contrib: add a write microbenchmark to perf.py https://www.mercurial-scm.org/repo/hg/rev/fdecd24ca4dc changeset: 30978:fdecd24ca4dc user:Simon Farnsworth date:Wed Feb 15 13:50:06 2017 -0800 summary: ui: log time spent blocked on stdio https://www.mercurial-scm.org/repo/hg/rev/fd598149112b changeset: 30979:fd598149112b user:Simon Farnsworth date:Wed Feb 15 13:29:12 2017 -0800 summary: ui: time calls to ui.system https://www.mercurial-scm.org/repo/hg/rev/60b5db2ab586 changeset: 30980:60b5db2ab586 user:Simon Farnsworth date:Wed Feb 15 13:38:00 2017 -0800 summary: ui: give editor() a tag of its own https://www.mercurial-scm.org/repo/hg/rev/a0c4e57074fe changeset: 30981:a0c4e57074fe bookmark:@ tag: tip user:Simon Farnsworth date:Wed Feb 15 13:34:06 2017 -0800 summary: crecord: log blocked time waiting for curses input -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 03 of 10 v5] ui: provide a mechanism to track and log blocked time
On Wed, 15 Feb 2017 14:06:53 -0800, Simon Farnsworth wrote:
> # HG changeset patch
> # User Simon Farnsworth
> # Date 1487193465 28800
> # Wed Feb 15 13:17:45 2017 -0800
> # Node ID f3a219226ba0658f72801329d07c1ba516152b70
> # Parent 1c71bddbe01e76c1c48b5479ff67d47645afd7b6
> ui: provide a mechanism to track and log blocked time
> finally:
> duration = util.timer() - starttime
> req.ui.flush()
> +if req.ui.logblockedtimes:
> +req.ui._blockedtimes['command_duration'] = duration * 1000
> +req.ui.log('uiblocked', 'ui blocked ms', **req.ui._blockedtimes)
[snip]
> self._reportuntrusted = self.debugflag or self.configbool("ui",
> "report_untrusted", True)
> self.tracebackflag = self.configbool('ui', 'traceback', False)
> +self.logblockedtimes = self.configbool('ui', 'logblockedtimes')
Just curious. Why this config knob is needed only to suppress the output?
I thought this would stop tracking the blocked time at all.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] help: add pointer how to narrow list of resolved/unresolved files (issue5469)
# HG changeset patch # User Yuya Nishihara # Date 1487408401 -32400 # Sat Feb 18 18:00:01 2017 +0900 # Node ID ceef988474afe31056a2d8fcef67526cf43c2d6f # Parent 1ca3469fdd08c0d5d814a4bc359869bc157c7fc9 help: add pointer how to narrow list of resolved/unresolved files (issue5469) diff --git a/mercurial/commands.py b/mercurial/commands.py --- a/mercurial/commands.py +++ b/mercurial/commands.py @@ -4259,6 +4259,8 @@ def resolve(ui, repo, *pats, **opts): - :hg:`resolve -l`: list files which had or still have conflicts. In the printed list, ``U`` = unresolved and ``R`` = resolved. + You can use ``set:unresolved()`` or ``set:resolved()`` to filter + the list. See :hg:`help filesets` for details. .. note:: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] smartset: reorder initialization of baseset in more intuitive way
# HG changeset patch # User Yuya Nishihara # Date 1487407072 -32400 # Sat Feb 18 17:37:52 2017 +0900 # Node ID 1ca3469fdd08c0d5d814a4bc359869bc157c7fc9 # Parent 3f4bdcfe63e9f1e357187fa728ae0da9d35b3973 smartset: reorder initialization of baseset in more intuitive way What we want to do is to assign either _set or _list per the given data type. diff --git a/mercurial/smartset.py b/mercurial/smartset.py --- a/mercurial/smartset.py +++ b/mercurial/smartset.py @@ -166,16 +166,14 @@ class baseset(abstractsmartset): """ self._ascending = None self._istopo = istopo -if not isinstance(data, list): -if isinstance(data, set): -self._set = data -# set has no order we pick one for stability purpose -self._ascending = True -# converting set to list has a cost, do it lazily -data = None -else: +if isinstance(data, set): +# converting set to list has a cost, do it lazily +self._set = data +# set has no order we pick one for stability purpose +self._ascending = True +else: +if not isinstance(data, list): data = list(data) -if data is not None: self._list = data self._datarepr = datarepr ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@30970: 5 new changesets
5 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/520af4cc34de changeset: 30966:520af4cc34de user:Pierre-Yves David date:Fri Nov 18 18:43:39 2016 +0100 summary: color: spread '_effect' values for readability https://www.mercurial-scm.org/repo/hg/rev/20990991d384 changeset: 30967:20990991d384 bookmark:@ user:Pierre-Yves David date:Fri Nov 18 18:48:38 2016 +0100 summary: color: move '_effect' mapping into core https://www.mercurial-scm.org/repo/hg/rev/0d2a58a04080 changeset: 30968:0d2a58a04080 user:Pierre-Yves David date:Thu Dec 22 02:23:23 2016 +0100 summary: color: move '_terminfo_params' into the core 'color' module https://www.mercurial-scm.org/repo/hg/rev/ddc80d1777a6 changeset: 30969:ddc80d1777a6 user:Pierre-Yves David date:Thu Dec 22 02:26:50 2016 +0100 summary: color: move 'valideffect' function into the core module https://www.mercurial-scm.org/repo/hg/rev/683606a829e8 changeset: 30970:683606a829e8 tag: tip user:Pierre-Yves David date:Thu Dec 22 02:30:03 2016 +0100 summary: color: rework conditional 'valideffect' -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 3 V2] obsolete: avoid using revset language to compute the obsolete revset
# HG changeset patch
# User Jun Wu
# Date 1487408120 28800
# Sat Feb 18 00:55:20 2017 -0800
# Node ID 286567ff59dbf6cc5645f878b8b223c94c3e4116
# Parent 432992303718ec631b81a351795071c34cfe68f6
# Available At https://bitbucket.org/quark-zju/hg-draft
# hg pull https://bitbucket.org/quark-zju/hg-draft -r 286567ff59db
obsolete: avoid using revset language to compute the obsolete revset
This is part of a refactoring that moves some phase query optimization from
revset.py to phases.py. See previous patches for the motivation.
Now we have APIs in phasecache to get the non-public set efficiently, let's
use it directly instead of going through the "not public()" revset language
in "obsolete()" computation.
This patch was meaured using:
for i in 'public()' 'not public()' 'draft()' 'not draft()'; do
hg perfrevset "$i"; hg perfrevset "$i" --hidden;
done
and no noticeable (> 1%) performance difference was observed.
diff --git a/mercurial/obsolete.py b/mercurial/obsolete.py
--- a/mercurial/obsolete.py
+++ b/mercurial/obsolete.py
@@ -1121,5 +1121,5 @@ def _computeobsoleteset(repo):
obs = set()
getnode = repo.changelog.node
-notpublic = repo.revs("not public()")
+notpublic = repo._phasecache.getrevset(repo, (phases.draft, phases.secret))
for r in notpublic:
if getnode(r) in repo.obsstore.successors:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 3 V2] phases: add a getrevset method to phasecache
# HG changeset patch # User Jun Wu # Date 1487400545 28800 # Fri Feb 17 22:49:05 2017 -0800 # Node ID 7e30a7c38d845e9a526a7ac64c648f18ec51687e # Parent 1315d68b1dcf55247f51208608d139029eaae632 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r 7e30a7c38d84 phases: add a getrevset method to phasecache This is part of a refactoring that moves some phase query optimization from revset.py to phases.py. The motivation behind this was chg repo preloading - to make the obsstore depend on less things (like the revset language). The refactoring also looks good by itself - phasecache does not expose its private field "_phasesets" via public methods and revset.py is accessing it in a hacky way. This patch adds a "getrevset" method, which takes multiple phases and returns a revset in an best-effort efficient way - for "public" phase, it returns a lazy generatorset; for "draft" and "secret", it returns efficient "baseset". diff --git a/mercurial/phases.py b/mercurial/phases.py --- a/mercurial/phases.py +++ b/mercurial/phases.py @@ -116,4 +116,5 @@ from . import ( encoding, error, +smartset, ) @@ -171,4 +172,25 @@ class phasecache(object): self.opener = repo.svfs +def getrevset(self, repo, phases): +"""return a smartset for the given phases""" +self.loadphaserevs(repo) # ensure phase's sets are loaded + +if self._phasesets and all(self._phasesets[p] is not None + for p in phases): +# fast path - use _phasesets +revs = self._phasesets[phases[0]] +if len(phases) > 1: +revs = revs.copy() # only copy when needed +for p in phases[1:]: +revs.update(self._phasesets[p]) +if repo.changelog.filteredrevs: +revs = revs - repo.changelog.filteredrevs +return smartset.baseset(revs) +else: +# slow path - enumerate all revisions +phase = self.phase +revs = (r for r in repo if phase(repo, r) in phases) +return smartset.generatorset(revs, iterasc=True) + def copy(self): # Shallow copy meant to ensure isolation in ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 3 V2] revset: use phasecache.getrevset
# HG changeset patch
# User Jun Wu
# Date 1487407171 28800
# Sat Feb 18 00:39:31 2017 -0800
# Node ID 432992303718ec631b81a351795071c34cfe68f6
# Parent 7e30a7c38d845e9a526a7ac64c648f18ec51687e
# Available At https://bitbucket.org/quark-zju/hg-draft
# hg pull https://bitbucket.org/quark-zju/hg-draft -r 432992303718
revset: use phasecache.getrevset
This is part of a refactoring that moves some phase query optimization from
revset.py to phases.py. See the previous patch for motivation.
This patch changes revset code to use phasecache.getrevset so it no longer
accesses the private field: _phasecache._phasesets directly.
For performance impact, this patch was tested using the following query, on
my hg-committed repo:
for i in 'public()' 'not public()' 'draft()' 'not draft()'; do
echo $i;
hg perfrevset "$i";
hg perfrevset "$i" --hidden;
done
For the CPython implementation, most operations are unchanged (within
+/- 1%), while "not public()" and "draft()" is noticeably faster on an
unfiltered repo. It may be because the new code avoids a set copy if
filteredrevs is empty.
revset | public() | not public() | draft()| not draft()
hidden | yes | no | yes | no | yes | no | yes | no
--
before | 19006 | 17352 | 239 | 286 | 180 | 228 | 7690 | 5745
after | 19137 | 17231 | 240 | 207 | 182 | 150 | 7687 | 5658
delta | | -38% | | -52% |
(timed in microseconds)
For the pure Python implementation, some operations are faster while "not
draft()" is noticeably slower:
revset | public() | not public() | draft() | not draft()
hidden | yes | no | yes | no | yes | no | yes | no
before | 18852 | 17183 | 17758 | 15921 | 17505 | 15973 | 41521 | 39822
after | 18924 | 17380 | 17558 | 14545 | 16727 | 13593 | 48356 | 43992
delta | | -9% | -5% | -15% | +16% | +10%
That may be the different performance characters of generatorset vs.
filteredset. The "not draft()" query could be optimized in this case where
both "public" and "secret" are passed to "getrevsets" so it won't iterate
the whole repo twice.
diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -1645,17 +1645,8 @@ def parents(repo, subset, x):
return subset & ps
-def _phase(repo, subset, target):
-"""helper to select all rev in phase """
-repo._phasecache.loadphaserevs(repo) # ensure phase's sets are loaded
-if repo._phasecache._phasesets:
-s = repo._phasecache._phasesets[target] - repo.changelog.filteredrevs
-s = baseset(s)
-s.sort() # set are non ordered, so we enforce ascending
-return subset & s
-else:
-phase = repo._phasecache.phase
-condition = lambda r: phase(repo, r) == target
-return subset.filter(condition, condrepr=('', target),
- cache=False)
+def _phase(repo, subset, *targets):
+"""helper to select all rev in phases"""
+s = repo._phasecache.getrevset(repo, targets)
+return subset & s
@predicate('draft()', safe=True)
@@ -1718,18 +1709,5 @@ def present(repo, subset, x):
def _notpublic(repo, subset, x):
getargs(x, 0, 0, "_notpublic takes no arguments")
-repo._phasecache.loadphaserevs(repo) # ensure phase's sets are loaded
-if repo._phasecache._phasesets:
-s = set()
-for u in repo._phasecache._phasesets[1:]:
-s.update(u)
-s = baseset(s - repo.changelog.filteredrevs)
-s.sort()
-return subset & s
-else:
-phase = repo._phasecache.phase
-target = phases.public
-condition = lambda r: phase(repo, r) != target
-return subset.filter(condition, condrepr=('', target),
- cache=False)
+return _phase(repo, subset, phases.draft, phases.secret)
@predicate('public()', safe=True)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2] smartset: use native set operations as fast paths
Excerpts from Yuya Nishihara's message of 2017-02-18 17:27:51 +0900: > A baseset may be ordered. Native set ops are valid only in unordered context > (or when both sets have the same explicit ascending flag.) > > Can you add some doctests? Good catch. I'll add them later. I'm preparing the phasecache patches now. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 2] smartset: convert set to list lazily
On Fri, 17 Feb 2017 22:02:11 -0800, Jun Wu wrote: > # HG changeset patch > # User Jun Wu > # Date 1487393969 28800 > # Fri Feb 17 20:59:29 2017 -0800 > # Node ID b32af6cafb9b958decb7ae0db7dd754aa5a5f883 > # Parent 01eebb65a61d9edcad1665ed747c7092f1ddb8b9 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > b32af6cafb9b > smartset: convert set to list lazily Queued this, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH py3] ui: construct _keepalnum list in a python3-friendly way
On Thu, 16 Feb 2017 11:35:41 -0500, Augie Fackler wrote: > # HG changeset patch > # User Augie Fackler > # Date 1487262890 18000 > # Thu Feb 16 11:34:50 2017 -0500 > # Node ID 791b4e846a7b9a0783440b9504585438777fe2d2 > # Parent 1ee685defe80117cf6aafea1ede6c33c478abceb > ui: construct _keepalnum list in a python3-friendly way Looks good. Queued, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2] smartset: use native set operations as fast paths
On Fri, 17 Feb 2017 22:02:12 -0800, Jun Wu wrote: > # HG changeset patch > # User Jun Wu > # Date 1487396855 28800 > # Fri Feb 17 21:47:35 2017 -0800 > # Node ID 1315d68b1dcf55247f51208608d139029eaae632 > # Parent b32af6cafb9b958decb7ae0db7dd754aa5a5f883 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > 1315d68b1dcf > smartset: use native set operations as fast paths > > For cases where we know both basesets have their sets ready, use native > Python set operations as a fast path. > > This leads to noticeable improvements on some set operations: > > revset| before | after | after/before > - > draft() & draft() & draft() & draft() | 1123 | 818 | 0.72 > draft() + draft() + draft() + draft() | 3288 | 817 | 0.25 > draft() - draft() + draft() - draft() | 1277 | 551 | 0.43 > draft() - draft() - draft() - draft() |864 | 517 | 0.60 > > (time unit: micro second) > > diff --git a/mercurial/smartset.py b/mercurial/smartset.py > --- a/mercurial/smartset.py > +++ b/mercurial/smartset.py > @@ -278,4 +278,28 @@ class baseset(abstractsmartset): > return None > > +def _fastsetop(self, other, op): > +# try to use native set operations as fast paths > +if (type(other) is baseset and '_set' in other.__dict__ and '_set' in > +self.__dict__): > +if op == '__add__': > +setop = 'union' > +else: > +setop = op # __sub__ or __and__ > +s = baseset(data=getattr(self._set, setop)(other._set), > +istopo=self._istopo) > +s._ascending = self._ascending > +else: > +s = getattr(super(baseset, self), op)(other) > +return s > + > +def __add__(self, other): > +return self._fastsetop(other, '__add__') > + > +def __and__(self, other): > +return self._fastsetop(other, '__and__') > + > +def __sub__(self, other): > +return self._fastsetop(other, '__sub__') A baseset may be ordered. Native set ops are valid only in unordered context (or when both sets have the same explicit ascending flag.) Can you add some doctests? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
