Re: [PATCH 3 of 5] tags: cache `repo.changelog` access when checking tags nodes
On Tue, Nov 20, 2018 at 10:23 PM Boris Feld wrote: > # HG changeset patch > # User Boris Feld > # Date 1542710295 0 > # Tue Nov 20 10:38:15 2018 + > # Node ID 1ad7353bec2584a53f2bb4fbca44310fa827f2d0 > # Parent 7760b12bd4fb467f4fb7e4d0d5094fc324f30f49 > # EXP-Topic perf-tags > # Available At https://bitbucket.org/octobus/mercurial-devel/ > # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r > 1ad7353bec25 > tags: cache `repo.changelog` access when checking tags nodes > > The tags reading process checks if the nodes referenced in tags exist. > Caching > the access to `repo.changelog` provides a large speedup for repositories > with > many tags. > > before: ! wall 0.393464 comb 0.39 user 0.33 sys 0.06 (median > of 25) > after: ! wall 0.267711 comb 0.27 user 0.21 sys 0.06 (median > of 38) > Can you please add the perf command of which these results are in this patch also? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH STABLE] tests: stabilize the recent checkexec changes on Windows
On Tue, 20 Nov 2018 18:49:01 -0500, Matt Harbison wrote: > # HG changeset patch > # User Matt Harbison > # Date 1542757639 18000 > # Tue Nov 20 18:47:19 2018 -0500 > # Branch stable > # Node ID 25b7c4cb5de1593e16bbd7a7dafbd28baa743995 > # Parent bd0874977a5e5921f7f8615f2eb96d267f641c2e > tests: stabilize the recent checkexec changes on Windows Queued for stable, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 7 V4] sparse-revlog: add a `index_segment_span` function in C
On Tue, 20 Nov 2018 21:44:34 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld
> # Date 1541785396 -3600
> # Fri Nov 09 18:43:16 2018 +0100
> # Node ID 5bb00d137c14d311c91c7a4f32f58378cd9195ec
> # Parent 8863dc1f7d078827708d178bbbfee5519b4c9b0f
> # EXP-Topic sparse-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 5bb00d137c14
> sparse-revlog: add a `index_segment_span` function in C
>
> We are about to implement a native version of `slicechunktodensity`. For
> clarity, we introduce the helper functions first. This new function provides
> an efficient way to retrieve some of the information needed by
> `slicechunktodensity`.
>
> diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
> --- a/mercurial/cext/revlog.c
> +++ b/mercurial/cext/revlog.c
> @@ -1068,6 +1068,27 @@ bail:
> return NULL;
> }
>
> +static inline int64_t
> +index_segment_span(indexObject *self, Py_ssize_t start_rev, Py_ssize_t
> end_rev)
> +{
> + int64_t start_offset;
> + int64_t end_offset;
> + int end_size;
> + start_offset = index_get_start(self, start_rev);
> + if (start_offset < 0) {
> + return -1;
> + }
> + end_offset = index_get_start(self, end_rev);
> + if (end_offset < 0) {
> + return -1;
> + }
> + end_size = index_get_length(self, end_rev);
> + if (end_size < 0) {
> + return -1;
> + }
> + return (end_offset - start_offset) + (int64_t)end_size;
Nit: maybe raise error if end_offset - start_offset) + (int64_t)end_size < 0 ?
Other than that, the series looks good to me.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 7 V4] sparse-revlog: add a `index_get_start` function in C
On Tue, 20 Nov 2018 21:44:32 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld
> # Date 1542725358 0
> # Tue Nov 20 14:49:18 2018 +
> # Node ID 60a55da39befa4996c1a88ca6f663765ea143ef2
> # Parent 337a389953366f059f2cb88129031ba3e67e0bbc
> # EXP-Topic sparse-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 60a55da39bef
> sparse-revlog: add a `index_get_start` function in C
> diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
> --- a/mercurial/cext/revlog.c
> +++ b/mercurial/cext/revlog.c
> @@ -185,6 +185,46 @@ static inline int index_get_parents(inde
> return 0;
> }
>
> +static inline int64_t index_get_start(indexObject *self, Py_ssize_t rev)
I meant "int index_get_start(*self, rev, *offset)" just like
index_get_parents(),
but this is also good.
> +{
> + uint64_t offset;
> + if (rev >= self->length) {
> + PyObject *tuple;
> + PyObject *pylong;
> + unsigned PY_LONG_LONG tmp;
> + tuple = PyList_GET_ITEM(self->added, rev - self->length);
> + pylong = PyTuple_GET_ITEM(tuple, 0);
> +#ifndef IS_PY3K
> + if (PyInt_Check(pylong)) {
> + long tmp2 = PyInt_AsLong(pylong);
> + if (tmp2 < 0) {
> + return -1;
> + }
> + tmp = (unsigned PY_LONG_LONG)tmp2;
> + } else {
> +#endif
> + tmp = PyLong_AsUnsignedLongLong(pylong);
> + if (tmp == (unsigned PY_LONG_LONG) - 1) {
> + return -1;
> + }
> +
> +#ifndef IS_PY3K
> + }
> +#endif
> + offset = (uint64_t)tmp;
Sigh, PyLong API is so inconsistent on Python 2. Let's switch to
PyLong_AsLongLong(), which can handle PyInt as well. I think it's
okay to raise OverflowError if offset >= 1 << (63 - 16).
FWIW, PyLong_AsUnsignedLongLong() of Python 2 would raise BadInternalCall
if non-long type were passed.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 7 V4] sparse-revlog: add a `index_get_length` function in C
On Tue, 20 Nov 2018 21:44:33 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld
> # Date 1541785378 -3600
> # Fri Nov 09 18:42:58 2018 +0100
> # Node ID 8863dc1f7d078827708d178bbbfee5519b4c9b0f
> # Parent 60a55da39befa4996c1a88ca6f663765ea143ef2
> # EXP-Topic sparse-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 8863dc1f7d07
> sparse-revlog: add a `index_get_length` function in C
>
> We are about to implement a native version of `slicechunktodensity`. For
> clarity, we introduce the helper functions first. This new function provides
> an efficient way to retrieve some of the information needed by
> `slicechunktodensity`.
>
> diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
> --- a/mercurial/cext/revlog.c
> +++ b/mercurial/cext/revlog.c
> @@ -225,6 +225,44 @@ static inline int64_t index_get_start(in
> return (int64_t)(offset >> 16);
> }
>
> +static inline int index_get_length(indexObject *self, Py_ssize_t rev)
> +{
> + if (rev >= self->length) {
> + PyObject *tuple;
> + PyObject *pylong;
> + unsigned PY_LONG_LONG tmp;
> + tuple = PyList_GET_ITEM(self->added, rev - self->length);
> + pylong = PyTuple_GET_ITEM(tuple, 1);
> +#ifndef IS_PY3K
> + if (PyInt_Check(pylong)) {
> + long tmp2 = PyInt_AsLong(pylong);
> + if (tmp2 < 0) {
> + return -1;
Nit: exception wouldn't be set if the value was actually -1.
> + }
> + tmp = (unsigned PY_LONG_LONG)tmp2;
> + } else {
> +#endif
> + tmp = PyLong_AsUnsignedLongLong(pylong);
> + if (tmp == (unsigned PY_LONG_LONG) - 1) {
> + return -1;
> + }
IIUC, we can simply use PyInt_AsLong() here since a valid length should be
in int range. PyInt_AsLong is aliased to PyLong_AsLong on Python 3.
To be clear, what I have in mind is something like this:
long ret = PyInt_AsLong(PyTuple_GET_ITEM(tuple, 1));
if (ret == -1 && PyErr_Occurred())
return -1;
if (ret < 0 || ret > (long)INT_MAX)
set overflow error and return -1;
> + if (tmp > INT_MAX) {
> + PyErr_Format(PyExc_OverflowError,
> + "revlog entry size too large (%llu)",
> + (long long)tmp);
> + return -1;
> + }
> + return (int)tmp;
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5287: py3: use node.hex(m.digest()) instead of m.hexdigest()
> > hgext/remotefilelog/basepack.py:472: import 'node' from line 12 shadowed > by loop variable > > Looks like a False positive/negative. Might be, but it's better to not shadow global names by local ones. Queued, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5287: py3: use node.hex(m.digest()) instead of m.hexdigest()
yuja added a comment. > > hgext/remotefilelog/basepack.py:472: import 'node' from line 12 shadowed by loop variable > > Looks like a False positive/negative. Might be, but it's better to not shadow global names by local ones. Queued, thanks. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5287 To: pulkit, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5287: py3: use node.hex(m.digest()) instead of m.hexdigest()
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9fcf8084ada8: py3: use node.hex(m.digest()) instead of
m.hexdigest() (authored by pulkit, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D5287?vs=12574&id=12575
REVISION DETAIL
https://phab.mercurial-scm.org/D5287
AFFECTED FILES
hgext/fastannotate/context.py
hgext/largefiles/lfcommands.py
hgext/lfs/blobstore.py
hgext/remotefilelog/basepack.py
CHANGE DETAILS
diff --git a/hgext/remotefilelog/basepack.py b/hgext/remotefilelog/basepack.py
--- a/hgext/remotefilelog/basepack.py
+++ b/hgext/remotefilelog/basepack.py
@@ -10,6 +10,7 @@
from mercurial.i18n import _
from mercurial import (
+node as nodemod,
policy,
pycompat,
util,
@@ -412,7 +413,7 @@
return
try:
-sha = self.sha.hexdigest()
+sha = nodemod.hex(self.sha.digest())
self.packfp.close()
self.writeindex()
diff --git a/hgext/lfs/blobstore.py b/hgext/lfs/blobstore.py
--- a/hgext/lfs/blobstore.py
+++ b/hgext/lfs/blobstore.py
@@ -20,6 +20,7 @@
from mercurial import (
encoding,
error,
+node,
pathutil,
pycompat,
url as urlmod,
@@ -156,7 +157,7 @@
fp.write(chunk)
sha256.update(chunk)
-realoid = sha256.hexdigest()
+realoid = node.hex(sha256.digest())
if realoid != oid:
raise LfsCorruptionError(_('corrupt remote lfs object: %s')
% oid)
@@ -206,7 +207,7 @@
# Don't abort if corruption is detected, because `hg verify` will
# give more useful info about the corruption- simply don't add the
# hardlink.
-if verify or hashlib.sha256(blob).hexdigest() == oid:
+if verify or node.hex(hashlib.sha256(blob).digest()) == oid:
self.ui.note(_('lfs: found %s in the usercache\n') % oid)
lfutil.link(self.cachevfs.join(oid), self.vfs.join(oid))
else:
@@ -230,7 +231,7 @@
for chunk in util.filechunkiter(fp, size=1048576):
sha256.update(chunk)
-return oid == sha256.hexdigest()
+return oid == node.hex(sha256.digest())
def has(self, oid):
"""Returns True if the local blobstore contains the requested blob,
@@ -587,7 +588,7 @@
return reduced.values()
def _verify(oid, content):
-realoid = hashlib.sha256(content).hexdigest()
+realoid = node.hex(hashlib.sha256(content).digest())
if realoid != oid:
raise LfsCorruptionError(_('detected corrupt lfs object: %s') % oid,
hint=_('run hg verify'))
diff --git a/hgext/largefiles/lfcommands.py b/hgext/largefiles/lfcommands.py
--- a/hgext/largefiles/lfcommands.py
+++ b/hgext/largefiles/lfcommands.py
@@ -240,7 +240,7 @@
# largefile was modified, update standins
m = hashlib.sha1('')
m.update(ctx[f].data())
-hash = m.hexdigest()
+hash = node.hex(m.digest())
if f not in lfiletohash or lfiletohash[f] != hash:
rdst.wwrite(f, ctx[f].data(), ctx[f].flags())
executable = 'x' in ctx[f].flags()
diff --git a/hgext/fastannotate/context.py b/hgext/fastannotate/context.py
--- a/hgext/fastannotate/context.py
+++ b/hgext/fastannotate/context.py
@@ -138,7 +138,7 @@
(k, getattr(diffopts, k))
for k in mdiff.diffopts.defaults
))
-return hashlib.sha1(diffoptstr).hexdigest()[:6]
+return node.hex(hashlib.sha1(diffoptstr).digest())[:6]
_defaultdiffopthash = hashdiffopts(mdiff.defaultopts)
To: pulkit, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 5] extensions: add "uipopulate" hook, called per instance, not per process
On Sun, 18 Nov 2018 23:19:20 +0900, Yuya Nishihara wrote: > # HG changeset patch > # User Yuya Nishihara > # Date 1542024651 -32400 > # Mon Nov 12 21:10:51 2018 +0900 > # Node ID 1e1427e07940f57d4146c38a0757e5104a88985c > # Parent 7b0e4efc0a7140a3d6f0814214e0c2bfa6c60acf > extensions: add "uipopulate" hook, called per instance, not per process > +Ui instance setup > +- > + > +The optional ``uipopulate`` is called for each ``ui`` instance after > +configuration is loaded, where extensions can set up additional ui members, > +update configuration by ``ui.setconfig()``, and extend the class dynamically. > + > +Typically there are three ``ui`` instances involved in command execution: > + > +``req.ui`` (or ``repo.baseui``) > +Only system and user configurations are loaded into it. > +``lui`` > +Local repository configuration is loaded as well. This will be used at > +early dispatching stage where a repository isn't available. > +``repo.ui`` > +The fully-loaded ``ui`` used after a repository is instantiated. This > +will be created from the ``req.ui`` per repository. > + > +Be aware that the ``req.ui`` wouldn't see local repository configuration, > +and the ``repo.ui`` will be copied from the base ``req.ui`` *after* > +``uipopulate`` is called. Use ``reposetup`` to set up ``repo.ui`` instance > +with repository-local configuration. I'll send V2 which will simply call uipopulate(repo.ui) after repo/hgrc is loaded. That seems less error-prone. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 5 V2] ui: unify argument name of ui.log()
# HG changeset patch # User Yuya Nishihara # Date 1541926763 -32400 # Sun Nov 11 17:59:23 2018 +0900 # Node ID 84a17834f4aa3bafba00b1b448d1a90dadd6d9e7 # Parent 96d5d57ab30e831b11e48f92936c3ebcfb3e7510 ui: unify argument name of ui.log() It's called an "event" in both blackbox and logtoprocess. diff --git a/mercurial/ui.py b/mercurial/ui.py --- a/mercurial/ui.py +++ b/mercurial/ui.py @@ -1709,10 +1709,10 @@ class ui(object): '''exists only so low-level modules won't need to import scmutil''' return scmutil.progress(self, topic, unit, total) -def log(self, service, *msg, **opts): +def log(self, event, *msg, **opts): '''hook for logging facility extensions -service should be a readily-identifiable subsystem, which will +event should be a readily-identifiable subsystem, which will allow filtering. *msg should be a newline-terminated format string to log, and ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 5 V2] logtoprocess: extract logger class from ui wrapper
# HG changeset patch
# User Yuya Nishihara
# Date 1541925918 -32400
# Sun Nov 11 17:45:18 2018 +0900
# Node ID 96d5d57ab30e831b11e48f92936c3ebcfb3e7510
# Parent 1a6bb5a85e3055d745d83e72fe17f3a6a8a13f63
logtoprocess: extract logger class from ui wrapper
It provides the same interface as the blackboxlogger. The ui wrapper will
be removed shortly.
diff --git a/hgext/logtoprocess.py b/hgext/logtoprocess.py
--- a/hgext/logtoprocess.py
+++ b/hgext/logtoprocess.py
@@ -38,6 +38,7 @@ import os
from mercurial import (
pycompat,
+util,
)
from mercurial.utils import (
procutil,
@@ -49,27 +50,50 @@ from mercurial.utils import (
# leave the attribute unspecified.
testedwith = 'ships-with-hg-core'
+class processlogger(object):
+"""Map log events to external commands
+
+Arguments are passed on as environment variables.
+"""
+
+def __init__(self, ui):
+self._scripts = dict(ui.configitems(b'logtoprocess'))
+
+def tracked(self, event):
+return bool(self._scripts.get(event))
+
+def log(self, ui, event, msg, opts):
+script = self._scripts.get(event)
+if not script:
+return
+env = {
+b'EVENT': event,
+b'HGPID': os.getpid(),
+b'MSG1': msg[0] % msg[1:],
+}
+# keyword arguments get prefixed with OPT_ and uppercased
+env.update((b'OPT_%s' % key.upper(), value)
+ for key, value in pycompat.byteskwargs(opts).items())
+fullenv = procutil.shellenviron(env)
+procutil.runbgcommand(script, fullenv, shell=True)
+
def uisetup(ui):
class logtoprocessui(ui.__class__):
-def log(self, event, *msg, **opts):
-"""Map log events to external commands
-
-Arguments are passed on as environment variables.
+def __init__(self, src=None):
+super(logtoprocessui, self).__init__(src)
+if src and r'_ltplogger' in src.__dict__:
+self._ltplogger = src._ltplogger
-"""
-script = self.config('logtoprocess', event)
-if script:
-env = {
-b'EVENT': event,
-b'HGPID': os.getpid(),
-b'MSG1': msg[0] % msg[1:],
-}
-# keyword arguments get prefixed with OPT_ and uppercased
-env.update((b'OPT_%s' % key.upper(), value)
- for key, value in
pycompat.byteskwargs(opts).items())
-fullenv = procutil.shellenviron(env)
-procutil.runbgcommand(script, fullenv, shell=True)
+# trick to initialize logger after configuration is loaded, which
+# can be replaced later with processlogger(ui) in uisetup(), where
+# both user and repo configurations should be available.
+@util.propertycache
+def _ltplogger(self):
+return processlogger(self)
+
+def log(self, event, *msg, **opts):
+self._ltplogger.log(self, event, msg, opts)
return super(logtoprocessui, self).log(event, *msg, **opts)
# Replace the class for this instance and all clones created from it:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 5 V2] hgweb: load globally-enabled extensions explicitly
# HG changeset patch # User Yuya Nishihara # Date 1542449505 -32400 # Sat Nov 17 19:11:45 2018 +0900 # Node ID 39c91c2915f3452039109d182c91c8082aabc81a # Parent 84a17834f4aa3bafba00b1b448d1a90dadd6d9e7 hgweb: load globally-enabled extensions explicitly Before, extensions were loaded as a side effect of hg.repository() if the hgweb was executed as a CGI/WSGI. I want to make it explicit so that another ui hook can be inserted after extensions.loadall(). diff --git a/mercurial/hgweb/hgweb_mod.py b/mercurial/hgweb/hgweb_mod.py --- a/mercurial/hgweb/hgweb_mod.py +++ b/mercurial/hgweb/hgweb_mod.py @@ -22,6 +22,7 @@ from .common import ( from .. import ( encoding, error, +extensions, formatter, hg, hook, @@ -212,6 +213,7 @@ class hgweb(object): u = baseui.copy() else: u = uimod.ui.load() +extensions.loadall(u) r = hg.repository(u, repo) else: # we trust caller to give us a private copy diff --git a/mercurial/hgweb/hgwebdir_mod.py b/mercurial/hgweb/hgwebdir_mod.py --- a/mercurial/hgweb/hgwebdir_mod.py +++ b/mercurial/hgweb/hgwebdir_mod.py @@ -30,6 +30,7 @@ from .. import ( configitems, encoding, error, +extensions, hg, profiling, pycompat, @@ -268,6 +269,9 @@ class hgwebdir(object): self.lastrefresh = 0 self.motd = None self.refresh() +if not baseui: +# set up environment for new ui +extensions.loadall(self.ui) def refresh(self): if self.ui: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 5 V2] extensions: add "uipopulate" hook, called per instance, not per process
# HG changeset patch
# User Yuya Nishihara
# Date 1542024651 -32400
# Mon Nov 12 21:10:51 2018 +0900
# Node ID 9e8a18bf900b65f91f1cb9b9e65bb9e654fa3dd8
# Parent 39c91c2915f3452039109d182c91c8082aabc81a
extensions: add "uipopulate" hook, called per instance, not per process
In short, this is the "reposetup" function for ui. It allows us to modify
ui attributes without extending ui.__class__. Before, the only way to do
that was to abuse the config dictionary, which is copied across ui instances.
See the next patch for usage example.
diff --git a/mercurial/chgserver.py b/mercurial/chgserver.py
--- a/mercurial/chgserver.py
+++ b/mercurial/chgserver.py
@@ -246,6 +246,10 @@ def _loadnewui(srcui, args):
rpath = options['repository']
path, newlui = dispatch._getlocal(newui, rpath, wd=cwd)
+extensions.populateui(newui)
+if newui is not newlui:
+extensions.populateui(newlui)
+
return (newui, newlui)
class channeledsystem(object):
diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -866,6 +866,9 @@ def _dispatch(req):
# Check abbreviation/ambiguity of shell alias.
shellaliasfn = _checkshellalias(lui, ui, args)
if shellaliasfn:
+# no additional configs will be set, set up the ui instances
+for ui_ in uis:
+extensions.populateui(ui_)
return shellaliasfn()
# check for fallback encoding
@@ -948,6 +951,10 @@ def _dispatch(req):
for ui_ in uis:
ui_.disablepager()
+# configs are fully loaded, set up the ui instances
+for ui_ in uis:
+extensions.populateui(ui_)
+
if options['version']:
return commands.version_(ui)
if options['help']:
diff --git a/mercurial/extensions.py b/mercurial/extensions.py
--- a/mercurial/extensions.py
+++ b/mercurial/extensions.py
@@ -405,6 +405,25 @@ def afterloaded(extension, callback):
else:
_aftercallbacks.setdefault(extension, []).append(callback)
+def populateui(ui):
+"""Run extension hooks on the given ui to populate additional members,
+extend the class dynamically, etc.
+
+This will be called after the configuration is loaded, and/or extensions
+are loaded. In general, it's once per ui instance, but in command-server
+and hgweb, this may be called more than once with the same ui.
+"""
+for name, mod in extensions(ui):
+hook = getattr(mod, 'uipopulate', None)
+if not hook:
+continue
+try:
+hook(ui)
+except Exception as inst:
+ui.traceback(force=True)
+ui.warn(_('*** failed to populate ui by extension %s: %s\n')
+% (name, stringutil.forcebytestr(inst)))
+
def bind(func, *args):
'''Partial function application
diff --git a/mercurial/help/internals/extensions.txt
b/mercurial/help/internals/extensions.txt
--- a/mercurial/help/internals/extensions.txt
+++ b/mercurial/help/internals/extensions.txt
@@ -183,6 +183,29 @@ Command table setup
After ``extsetup``, the ``cmdtable`` is copied into the global command table
in Mercurial.
+Ui instance setup
+-
+
+The optional ``uipopulate`` is called for each ``ui`` instance after
+configuration is loaded, where extensions can set up additional ui members,
+update configuration by ``ui.setconfig()``, and extend the class dynamically.
+
+Typically there are three ``ui`` instances involved in command execution:
+
+``req.ui`` (or ``repo.baseui``)
+Only system and user configurations are loaded into it.
+``lui``
+Local repository configuration is loaded as well. This will be used at
+early dispatching stage where a repository isn't available.
+``repo.ui``
+The fully-loaded ``ui`` used after a repository is instantiated. This
+will be created from the ``req.ui`` per repository.
+
+In command server and hgweb, this may be called more than once for the same
+``ui`` instance.
+
+(New in Mercurial 4.9)
+
Repository setup
@@ -304,7 +327,8 @@ uisetup
a change made here will be visible by other extensions during ``extsetup``.
* Monkeypatches or function wraps (``extensions.wrapfunction``) of ``dispatch``
module members
-* Setup of ``pre-*`` and ``post-*`` hooks
+* Set up ``pre-*`` and ``post-*`` hooks. (DEPRECATED. ``uipopulate`` is
+ preferred on Mercurial 4.9 and later.)
* ``pushkey`` setup
extsetup
@@ -314,9 +338,17 @@ extsetup
* Add a global option to all commands
* Extend revsets
+uipopulate
+--
+
+* Modify ``ui`` instance attributes and configuration variables.
+* Changes to ``ui.__class__`` per instance.
+* Set up all hooks per scoped configuration.
+
reposetup
-
-* All hooks but ``pre-*`` and ``post-*``
+* Set up all hooks but ``pre-*`` and ``post-*``. (DEPRECATED. ``uipopulate`` is
+ preferred on Mercurial 4.9 and later.)
* Modi
[PATCH 5 of 5 V2] ui: manage logger instances and event filtering by core ui
# HG changeset patch
# User Yuya Nishihara
# Date 1541927313 -32400
# Sun Nov 11 18:08:33 2018 +0900
# Node ID a580ee2c7ef735dfc0efb69a30ec1482576ea1df
# Parent 9e8a18bf900b65f91f1cb9b9e65bb9e654fa3dd8
ui: manage logger instances and event filtering by core ui
The setup code in blackbox needs more tweaks since it has lots of black
magics. I'll fix them by follow-up patches.
To be clear, the goal of this series is to provide a proper way for command
server to install its own logger. I need it to debug in-memory repository
cache.
diff --git a/hgext/blackbox.py b/hgext/blackbox.py
--- a/hgext/blackbox.py
+++ b/hgext/blackbox.py
@@ -53,7 +53,6 @@ from mercurial import (
pycompat,
registrar,
ui as uimod,
-util,
)
from mercurial.utils import (
dateutil,
@@ -147,9 +146,6 @@ class blackboxlogger(object):
def log(self, ui, event, msg, opts):
global _lastlogger
-if not self.tracked(event):
-return
-
if self._bbvfs:
_lastlogger = self
elif _lastlogger and _lastlogger._bbvfs:
@@ -201,33 +197,20 @@ class blackboxlogger(object):
def wrapui(ui):
class blackboxui(ui.__class__):
-def __init__(self, src=None):
-super(blackboxui, self).__init__(src)
-if src and r'_bblogger' in src.__dict__:
-self._bblogger = src._bblogger
-
-# trick to initialize logger after configuration is loaded, which
-# can be replaced later with blackboxlogger(ui) in uisetup(), where
-# both user and repo configurations should be available.
-@util.propertycache
-def _bblogger(self):
-return blackboxlogger(self)
-
def debug(self, *msg, **opts):
super(blackboxui, self).debug(*msg, **opts)
if self.debugflag:
self.log('debug', '%s', ''.join(msg))
-def log(self, event, *msg, **opts):
-super(blackboxui, self).log(event, *msg, **opts)
-self._bblogger.log(self, event, msg, opts)
-
ui.__class__ = blackboxui
uimod.ui = blackboxui
def uisetup(ui):
wrapui(ui)
+def uipopulate(ui):
+ui.setlogger(b'blackbox', blackboxlogger(ui))
+
def reposetup(ui, repo):
# During 'hg pull' a httppeer repo is created to represent the remote repo.
# It doesn't have a .hg directory to put a blackbox in, so we don't do
@@ -235,7 +218,10 @@ def reposetup(ui, repo):
if not repo.local():
return
-logger = getattr(ui, '_bblogger', None)
+# Since blackbox.log is stored in the repo directory, the logger should be
+# instantiated per repository.
+logger = blackboxlogger(ui)
+ui.setlogger(b'blackbox', logger)
if logger:
logger.setrepo(repo)
diff --git a/hgext/logtoprocess.py b/hgext/logtoprocess.py
--- a/hgext/logtoprocess.py
+++ b/hgext/logtoprocess.py
@@ -38,7 +38,6 @@ import os
from mercurial import (
pycompat,
-util,
)
from mercurial.utils import (
procutil,
@@ -63,9 +62,7 @@ class processlogger(object):
return bool(self._scripts.get(event))
def log(self, ui, event, msg, opts):
-script = self._scripts.get(event)
-if not script:
-return
+script = self._scripts[event]
env = {
b'EVENT': event,
b'HGPID': os.getpid(),
@@ -77,24 +74,5 @@ class processlogger(object):
fullenv = procutil.shellenviron(env)
procutil.runbgcommand(script, fullenv, shell=True)
-def uisetup(ui):
-
-class logtoprocessui(ui.__class__):
-def __init__(self, src=None):
-super(logtoprocessui, self).__init__(src)
-if src and r'_ltplogger' in src.__dict__:
-self._ltplogger = src._ltplogger
-
-# trick to initialize logger after configuration is loaded, which
-# can be replaced later with processlogger(ui) in uisetup(), where
-# both user and repo configurations should be available.
-@util.propertycache
-def _ltplogger(self):
-return processlogger(self)
-
-def log(self, event, *msg, **opts):
-self._ltplogger.log(self, event, msg, opts)
-return super(logtoprocessui, self).log(event, *msg, **opts)
-
-# Replace the class for this instance and all clones created from it:
-ui.__class__ = logtoprocessui
+def uipopulate(ui):
+ui.setlogger(b'logtoprocess', processlogger(ui))
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -235,6 +235,7 @@ class ui(object):
self._fmsgout = src._fmsgout
self._fmsgerr = src._fmsgerr
self._finoutredirected = src._finoutredirected
+self._loggers = src._loggers.copy()
self.pageractive = src.pageractive
self._disablepager = src._disablepager
self._tweaked = src._tweaked
@@ -263,6 +264,7 @@ class ui(object):
self._fmsgout
D5291: branchmap: build the revbranchcache._namesreverse() only when required
pulkit created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY On big repositories with a lot of named branches and that also increasing over time, building of this dict can be expensive and shows up in profile. For our internal repository, this saves ~0.05 seconds. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5291 AFFECTED FILES mercurial/branchmap.py CHANGE DETAILS diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py --- a/mercurial/branchmap.py +++ b/mercurial/branchmap.py @@ -397,13 +397,13 @@ self._names = [] self._rbcnamescount = len(self._names) # number of names read at # _rbcsnameslen -self._namesreverse = dict((b, r) for r, b in enumerate(self._names)) +self._namesreverse = None def _clear(self): self._rbcsnameslen = 0 del self._names[:] self._rbcnamescount = 0 -self._namesreverse.clear() +self._namesreverse = None self._rbcrevslen = len(self._repo.changelog) self._rbcrevs = bytearray(self._rbcrevslen * _rbcrecsize) @@ -453,6 +453,8 @@ """Retrieve branch info from changelog and update _rbcrevs""" changelog = self._repo.changelog b, close = changelog.branchinfo(rev) +if self._namesreverse is None: +self._namesreverse = dict((b, r) for r, b in enumerate(self._names)) if b in self._namesreverse: branchidx = self._namesreverse[b] else: @@ -467,6 +469,8 @@ def setdata(self, branch, rev, node, close): """add new data information to the cache""" +if self._namesreverse is None: +self._namesreverse = dict((b, r) for r, b in enumerate(self._names)) if branch in self._namesreverse: branchidx = self._namesreverse[branch] else: To: pulkit, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5288: tests: make test-alias.t pass with re2
> Thanks. FWIW, re2 options allow to choose an encoding which is latin1 or > utf8 (https://github.com/google/re2/blob/master/re2/re2.h#L609). Presumably > latin1 means that "." matches a byte, which would seem more compatible with > re, but python bindings don't provide the ability to choose this encoding. Indeed. We could pass in a fat unicode array (i.e. each byte as latin-1 char) to re2, but that would sacrifice the performance. Thanks for the info. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5288: tests: make test-alias.t pass with re2
yuja added a comment. > Thanks. FWIW, re2 options allow to choose an encoding which is latin1 or utf8 (https://github.com/google/re2/blob/master/re2/re2.h#L609). Presumably latin1 means that "." matches a byte, which would seem more compatible with re, but python bindings don't provide the ability to choose this encoding. Indeed. We could pass in a fat unicode array (i.e. each byte as latin-1 char) to re2, but that would sacrifice the performance. Thanks for the info. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5288 To: valentin.gatienbaron, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5291: branchmap: build the revbranchcache._namesreverse() only when required
> +if self._namesreverse is None: > +self._namesreverse = dict((b, r) for r, b in > enumerate(self._names)) > if b in self._namesreverse: > branchidx = self._namesreverse[b] > else: > @@ -467,6 +469,8 @@ > > def setdata(self, branch, rev, node, close): > """add new data information to the cache""" > +if self._namesreverse is None: > +self._namesreverse = dict((b, r) for r, b in > enumerate(self._names)) Maybe _namesreverse can be a propertycache, instead of duplicating it. `del self.__dict__[r'...']` can be used to discard the cache. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5291: branchmap: build the revbranchcache._namesreverse() only when required
yuja added a comment. > +if self._namesreverse is None: > +self._namesreverse = dict((b, r) for r, b in enumerate(self._names)) > > if b in self._namesreverse: > branchidx = self._namesreverse[b] > else: > > @@ -467,6 +469,8 @@ > > def setdata(self, branch, rev, node, close): > """add new data information to the cache""" > > +if self._namesreverse is None: > +self._namesreverse = dict((b, r) for r, b in enumerate(self._names)) Maybe _namesreverse can be a propertycache, instead of duplicating it. `del self.__dict__[r'...']` can be used to discard the cache. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5291 To: pulkit, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5292: branchmap: make it easier for extensions not to break branchcache
mjpieters created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY The _branchcache global gives us a reference for super() to use even if an extension subclasses branchmap.branchcache then replaces the class in the module. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5292 AFFECTED FILES mercurial/branchmap.py CHANGE DETAILS diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py --- a/mercurial/branchmap.py +++ b/mercurial/branchmap.py @@ -211,7 +211,7 @@ def __init__(self, entries=(), tipnode=nullid, tiprev=nullrev, filteredhash=None, closednodes=None): -super(branchcache, self).__init__(entries) +super(_branchcache, self).__init__(entries) self.tipnode = tipnode self.tiprev = tiprev self.filteredhash = filteredhash @@ -369,6 +369,11 @@ self.write(repo) +# keep a private reference to the class object so that subclasses that +# replace the branchcache class do not break super() +_branchcache = branchcache + + class remotebranchcache(branchcache): """Branchmap info for a remote connection, should not write locally""" def write(self, repo): To: mjpieters, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 7 V4] sparse-revlog: add a `index_get_start` function in C
On 21/11/2018 13:30, Yuya Nishihara wrote:
> On Tue, 20 Nov 2018 21:44:32 +0100, Boris Feld wrote:
>> # HG changeset patch
>> # User Boris Feld
>> # Date 1542725358 0
>> # Tue Nov 20 14:49:18 2018 +
>> # Node ID 60a55da39befa4996c1a88ca6f663765ea143ef2
>> # Parent 337a389953366f059f2cb88129031ba3e67e0bbc
>> # EXP-Topic sparse-perf
>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
>> 60a55da39bef
>> sparse-revlog: add a `index_get_start` function in C
>> diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
>> --- a/mercurial/cext/revlog.c
>> +++ b/mercurial/cext/revlog.c
>> @@ -185,6 +185,46 @@ static inline int index_get_parents(inde
>> return 0;
>> }
>>
>> +static inline int64_t index_get_start(indexObject *self, Py_ssize_t rev)
> I meant "int index_get_start(*self, rev, *offset)" just like
> index_get_parents(),
> but this is also good.
Since the return is simpler than index_get_parent, it seemed simpler to
return the value directly.
>
>> +{
>> +uint64_t offset;
>> +if (rev >= self->length) {
>> +PyObject *tuple;
>> +PyObject *pylong;
>> +unsigned PY_LONG_LONG tmp;
>> +tuple = PyList_GET_ITEM(self->added, rev - self->length);
>> +pylong = PyTuple_GET_ITEM(tuple, 0);
>> +#ifndef IS_PY3K
>> +if (PyInt_Check(pylong)) {
>> +long tmp2 = PyInt_AsLong(pylong);
>> +if (tmp2 < 0) {
>> +return -1;
>> +}
>> +tmp = (unsigned PY_LONG_LONG)tmp2;
>> +} else {
>> +#endif
>> +tmp = PyLong_AsUnsignedLongLong(pylong);
>> +if (tmp == (unsigned PY_LONG_LONG) - 1) {
>> +return -1;
>> +}
>> +
>> +#ifndef IS_PY3K
>> +}
>> +#endif
>> +offset = (uint64_t)tmp;
> Sigh, PyLong API is so inconsistent on Python 2. Let's switch to
> PyLong_AsLongLong(), which can handle PyInt as well. I think it's
> okay to raise OverflowError if offset >= 1 << (63 - 16).
Okay, so dropping the special case and directly using:
offset = PyLong_AsLongLong(pylong);
if (offset < 0){ … }
Right?
>
> FWIW, PyLong_AsUnsignedLongLong() of Python 2 would raise BadInternalCall
> if non-long type were passed.
Yep, I saw that, and it's how I ended up adding the PyInt case.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 7 V4] sparse-revlog: add a `index_segment_span` function in C
On 21/11/2018 13:27, Yuya Nishihara wrote:
> On Tue, 20 Nov 2018 21:44:34 +0100, Boris Feld wrote:
>> # HG changeset patch
>> # User Boris Feld
>> # Date 1541785396 -3600
>> # Fri Nov 09 18:43:16 2018 +0100
>> # Node ID 5bb00d137c14d311c91c7a4f32f58378cd9195ec
>> # Parent 8863dc1f7d078827708d178bbbfee5519b4c9b0f
>> # EXP-Topic sparse-perf
>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
>> 5bb00d137c14
>> sparse-revlog: add a `index_segment_span` function in C
>>
>> We are about to implement a native version of `slicechunktodensity`. For
>> clarity, we introduce the helper functions first. This new function provides
>> an efficient way to retrieve some of the information needed by
>> `slicechunktodensity`.
>>
>> diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
>> --- a/mercurial/cext/revlog.c
>> +++ b/mercurial/cext/revlog.c
>> @@ -1068,6 +1068,27 @@ bail:
>> return NULL;
>> }
>>
>> +static inline int64_t
>> +index_segment_span(indexObject *self, Py_ssize_t start_rev, Py_ssize_t
>> end_rev)
>> +{
>> +int64_t start_offset;
>> +int64_t end_offset;
>> +int end_size;
>> +start_offset = index_get_start(self, start_rev);
>> +if (start_offset < 0) {
>> +return -1;
>> +}
>> +end_offset = index_get_start(self, end_rev);
>> +if (end_offset < 0) {
>> +return -1;
>> +}
>> +end_size = index_get_length(self, end_rev);
>> +if (end_size < 0) {
>> +return -1;
>> +}
>> +return (end_offset - start_offset) + (int64_t)end_size;
> Nit: maybe raise error if end_offset - start_offset) + (int64_t)end_size < 0 ?
Unless the revlog is corrupted, we have:
start_rev <= end_rev
and:
start_offset <= end_offset
So this value will be positive. If the revlog is corrupt I assume we'll
have more troubles sooner.
Or do you want extra checks on the above assertion?
>
> Other than that, the series looks good to me.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5293: py3: whitelist three more passing test caught by buildbot
pulkit created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY The number of failing test is back to two digits. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5293 AFFECTED FILES contrib/python3-whitelist CHANGE DETAILS diff --git a/contrib/python3-whitelist b/contrib/python3-whitelist --- a/contrib/python3-whitelist +++ b/contrib/python3-whitelist @@ -306,8 +306,11 @@ test-largefiles-small-disk.t test-largefiles-update.t test-largefiles.t +test-lfconvert.t +test-lfs-bundle.t test-lfs-largefiles.t test-lfs-pointer.py +test-lfs.t test-linelog.py test-linerange.py test-locate.t To: pulkit, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] blackbox: add configitem for format of log timestamps
Ping. Did you have any trouble applying this updated patch? This one should not
have any mangled spaces.
> On October 23, 2018 at 1:49 PM Matthew DeVore wrote:
>
>
> From: Matt DeVore
>
> My e-mail client was mangling spaces, which is why you couldn't apply the
> patch earlier. I'm sending this again with a different tool and e-mail
> address.
>
>
> # HG changeset patch
> # User Matt DeVore
> # Date 1539816481 25200
> # Wed Oct 17 15:48:01 2018 -0700
> # Node ID 13c2fe6d3b30a743daa1984404a70ed769779d11
> # Parent a0e7fa019290d5348c4a83f6b287f2612d645025
> blackbox: add configitem for format of log timestamps
>
> Sometimes blackbox logs are used to report performance problems, but the
> timestamps are only at second granularity, so often the timings have to
> stated separately by the reporter. This is inconvenient and error-prone,
> so I would like to include %f in the date format. This patch makes that
> possible.
>
> diff --git a/hgext/blackbox.py b/hgext/blackbox.py
> --- a/hgext/blackbox.py
> +++ b/hgext/blackbox.py
> @@ -33,6 +33,11 @@ Examples::
> # rotate up to N log files when the current one gets too big
> maxfiles = 3
>
> + [blackbox]
> + # Include nanoseconds in log entries with %f (see Python function
> + # datetime.datetime.strftime)
> + date-format = '%Y-%m-%d @ %H:%M:%S.%f'
> +
> """
>
> from __future__ import absolute_import
> @@ -82,6 +87,9 @@ configitem('blackbox', 'maxfiles',
> configitem('blackbox', 'track',
> default=lambda: ['*'],
> )
> +configitem('blackbox', 'date-format',
> + default='%Y/%m/%d %H:%M:%S',
> +)
>
> lastui = None
>
> @@ -169,7 +177,9 @@ def wrapui(ui):
> return
> ui._bbinlog = True
> default = self.configdate('devel', 'default-date')
> - date = dateutil.datestr(default, '%Y/%m/%d %H:%M:%S')
> + format = ui.config('blackbox', 'date-format')
> + date = dateutil.datestr(default,
> + ui.config('blackbox', 'date-format'))
> user = procutil.getuser()
> pid = '%d' % procutil.getpid()
> formattedmsg = msg[0] % msg[1:]
> diff --git a/tests/test-blackbox.t b/tests/test-blackbox.t
> --- a/tests/test-blackbox.t
> +++ b/tests/test-blackbox.t
> @@ -82,6 +82,16 @@ recursive aliases work correctly
> 1970/01/01 00:00:00 bob @ (5000)>
> so-confusing exited 0 after * seconds (glob)
> 1970/01/01 00:00:00 bob @ (5000)>
> blackbox
>
> +custom date format
> + $ rm ./.hg/blackbox.log
> + $ hg --config blackbox.date-format='%Y-%m-%d @ %H:%M:%S' \
> + > --config devel.default-date='1334347993 0' --traceback status
> + A a
> + $ hg blackbox
> + 2012-04-13 @ 20:13:13 bob @ (5000)>
> --config 'blackbox.date-format=%Y-%m-%d @ %H:%M:%S' --config
> 'devel.default-date=1334347993 0' --traceback status
> + 2012-04-13 @ 20:13:13 bob @ (5000)>
> --config 'blackbox.date-format=%Y-%m-%d @ %H:%M:%S' --config
> 'devel.default-date=1334347993 0' --traceback status exited 0 after * seconds
> (glob)
> + 1970/01/01 00:00:00 bob @ (5000)>
> blackbox
> +
> incoming change tracking
>
> create two heads to verify that we only see one change in the log later
>
>___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] blackbox: add configitem for format of log timestamps
On Wed, Nov 21, 2018 at 8:32 AM MATTHEW DEVORE wrote:
> Ping. Did you have any trouble applying this updated patch? This one
> should not have any mangled spaces.
>
This was queued as https://www.mercurial-scm.org/repo/hg/rev/25f1c7bd649d
(Yuya replied to you at Oct 24, 2018, 4:15 AM Pacific time).
> > On October 23, 2018 at 1:49 PM Matthew DeVore wrote:
> >
> >
> > From: Matt DeVore
> >
> > My e-mail client was mangling spaces, which is why you couldn't apply
> the
> > patch earlier. I'm sending this again with a different tool and e-mail
> address.
> >
>
>
> >
> > # HG changeset patch
> > # User Matt DeVore
> > # Date 1539816481 25200
> > # Wed Oct 17 15:48:01 2018 -0700
> > # Node ID 13c2fe6d3b30a743daa1984404a70ed769779d11
> > # Parent a0e7fa019290d5348c4a83f6b287f2612d645025
> > blackbox: add configitem for format of log timestamps
> >
> > Sometimes blackbox logs are used to report performance problems, but the
> > timestamps are only at second granularity, so often the timings have to
> > stated separately by the reporter. This is inconvenient and error-prone,
> > so I would like to include %f in the date format. This patch makes that
> > possible.
> >
> > diff --git a/hgext/blackbox.py b/hgext/blackbox.py
> > --- a/hgext/blackbox.py
> > +++ b/hgext/blackbox.py
> > @@ -33,6 +33,11 @@ Examples::
> > # rotate up to N log files when the current one gets too big
> > maxfiles = 3
> >
> > + [blackbox]
> > + # Include nanoseconds in log entries with %f (see Python function
> > + # datetime.datetime.strftime)
> > + date-format = '%Y-%m-%d @ %H:%M:%S.%f'
> > +
> > """
> >
> > from __future__ import absolute_import
> > @@ -82,6 +87,9 @@ configitem('blackbox', 'maxfiles',
> > configitem('blackbox', 'track',
> > default=lambda: ['*'],
> > )
> > +configitem('blackbox', 'date-format',
> > + default='%Y/%m/%d %H:%M:%S',
> > +)
> >
> > lastui = None
> >
> > @@ -169,7 +177,9 @@ def wrapui(ui):
> > return
> > ui._bbinlog = True
> > default = self.configdate('devel', 'default-date')
> > - date = dateutil.datestr(default, '%Y/%m/%d %H:%M:%S')
> > + format = ui.config('blackbox', 'date-format')
> > + date = dateutil.datestr(default,
> > + ui.config('blackbox', 'date-format'))
> > user = procutil.getuser()
> > pid = '%d' % procutil.getpid()
> > formattedmsg = msg[0] % msg[1:]
> > diff --git a/tests/test-blackbox.t b/tests/test-blackbox.t
> > --- a/tests/test-blackbox.t
> > +++ b/tests/test-blackbox.t
> > @@ -82,6 +82,16 @@ recursive aliases work correctly
> > 1970/01/01 00:00:00 bob @
> (5000)> so-confusing exited 0 after * seconds (glob)
> > 1970/01/01 00:00:00 bob @
> (5000)> blackbox
> >
> > +custom date format
> > + $ rm ./.hg/blackbox.log
> > + $ hg --config blackbox.date-format='%Y-%m-%d @ %H:%M:%S' \
> > + > --config devel.default-date='1334347993 0' --traceback status
> > + A a
> > + $ hg blackbox
> > + 2012-04-13 @ 20:13:13 bob @
> (5000)> --config 'blackbox.date-format=%Y-%m-%d @ %H:%M:%S' --config
> 'devel.default-date=1334347993 0' --traceback status
> > + 2012-04-13 @ 20:13:13 bob @
> (5000)> --config 'blackbox.date-format=%Y-%m-%d @ %H:%M:%S' --config
> 'devel.default-date=1334347993 0' --traceback status exited 0 after *
> seconds (glob)
> > + 1970/01/01 00:00:00 bob @
> (5000)> blackbox
> > +
> > incoming change tracking
> >
> > create two heads to verify that we only see one change in the log later
> >
> >
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] blackbox: add configitem for format of log timestamps
On Wed, Nov 21, 2018 at 7:32 PM MATTHEW DEVORE wrote: > Ping. Did you have any trouble applying this updated patch? This one > should not have any mangled spaces. > Your patch is already pushed. https://www.mercurial-scm.org/repo/hg/rev/25f1c7bd649d ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] blackbox: add configitem for format of log timestamps
I see now Yuya told me the change was queued in a prior e-mail. This thread is actually a nexus of 5 or 6 e-mail screw-ups on my part, so I didn't see it. Thank you all for helping me and I'll try Phabricator next time :) On Wed, Nov 21, 2018 at 8:37 AM Pulkit Goyal <7895pul...@gmail.com> wrote: > > > > On Wed, Nov 21, 2018 at 7:32 PM MATTHEW DEVORE wrote: >> >> Ping. Did you have any trouble applying this updated patch? This one should >> not have any mangled spaces. > > > Your patch is already pushed. > https://www.mercurial-scm.org/repo/hg/rev/25f1c7bd649d ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5294: tests: make test-check-module-imports more robust
valentin.gatienbaron created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY It fails for me with (among other things): tests/test-commandserver.t:19: relative import of stdlib module tests/test-lfs-serve.t:108: relative import of stdlib module tests/test-lfs-serve.t:255: relative import of stdlib module tests/test-lfs-serve.t:362: relative import of stdlib module tests/test-lfs-serve.t:406: relative import of stdlib module This is because every module is considered a stdlib module, because the stdlib_prefixes is /usr, and my repo is in /usr/local/home (so sys.path[0] is /usr/local/home/../hg/contrib since import-checker.py lives there). So remove the import-checker directory. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5294 AFFECTED FILES contrib/import-checker.py CHANGE DETAILS diff --git a/contrib/import-checker.py b/contrib/import-checker.py --- a/contrib/import-checker.py +++ b/contrib/import-checker.py @@ -260,9 +260,9 @@ break else: stdlib_prefixes.add(dirname) -for libpath in sys.path: -# We want to walk everything in sys.path that starts with -# something in stdlib_prefixes. +for libpath in sys.path[1:]: +# We want to walk everything in sys.path (other than the directory of +# the current script) that starts with something in stdlib_prefixes. if not any(libpath.startswith(p) for p in stdlib_prefixes): continue for top, dirs, files in os.walk(libpath): To: valentin.gatienbaron, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5294: tests: make test-check-module-imports more robust
valentin.gatienbaron updated this revision to Diff 12580. valentin.gatienbaron edited the summary of this revision. REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5294?vs=12579&id=12580 REVISION DETAIL https://phab.mercurial-scm.org/D5294 AFFECTED FILES contrib/import-checker.py CHANGE DETAILS diff --git a/contrib/import-checker.py b/contrib/import-checker.py --- a/contrib/import-checker.py +++ b/contrib/import-checker.py @@ -260,10 +260,12 @@ break else: stdlib_prefixes.add(dirname) +sourceroot = os.path.dirname(sys.path[0]) for libpath in sys.path: -# We want to walk everything in sys.path that starts with -# something in stdlib_prefixes. -if not any(libpath.startswith(p) for p in stdlib_prefixes): +# We want to walk everything in sys.paththat starts with something in +# stdlib_prefixes, but not directories from the hg sources. +if (libpath.startswith(sourceroot) +or not any(libpath.startswith(p) for p in stdlib_prefixes)): continue for top, dirs, files in os.walk(libpath): for i, d in reversed(list(enumerate(dirs))): To: valentin.gatienbaron, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 8] perf: explicitly pass title as a keyword argument in `perfdiffwd`
# HG changeset patch
# User Boris Feld
# Date 1542668549 0
# Mon Nov 19 23:02:29 2018 +
# Node ID 954159d40a5f4bf43465c50cc97261ea139f0032
# Parent dba590f27c7abacbd7e9b27f3e06822bb0b339cb
# EXP-Topic perf-ignore
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
954159d40a5f
perf: explicitly pass title as a keyword argument in `perfdiffwd`
This will help to update the timer function arguments in a later changeset.
diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -1404,7 +1404,7 @@ def perfdiffwd(ui, repo, **opts):
ui.popbuffer()
diffopt = diffopt.encode('ascii')
title = b'diffopts: %s' % (diffopt and (b'-' + diffopt) or b'none')
-timer(d, title)
+timer(d, title=title)
fm.end()
@command(b'perfrevlogindex', revlogopts + formatteropts,
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 8] perf: add a `setup` argument to run code outside of the timed section
# HG changeset patch # User Boris Feld # Date 1542668107 0 # Mon Nov 19 22:55:07 2018 + # Node ID 04a1a32aaaedc6a75117994dbf2823e39204fd7d # Parent 954159d40a5f4bf43465c50cc97261ea139f0032 # EXP-Topic perf-ignore # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 04a1a32aaaed perf: add a `setup` argument to run code outside of the timed section With this new argument, it is possible to perform special setup and cleanup outside of code actually timed. This is useful to provide more accurate benchmark. diff --git a/contrib/perf.py b/contrib/perf.py --- a/contrib/perf.py +++ b/contrib/perf.py @@ -275,7 +275,7 @@ def gettimer(ui, opts=None): displayall = ui.configbool(b"perf", b"all-timing", False) return functools.partial(_timer, fm, displayall=displayall), fm -def stub_timer(fm, func, title=None): +def stub_timer(fm, func, setup=None, title=None): func() @contextlib.contextmanager @@ -289,12 +289,14 @@ def timeone(): a, b = ostart, ostop r.append((cstop - cstart, b[0] - a[0], b[1]-a[1])) -def _timer(fm, func, title=None, displayall=False): +def _timer(fm, func, setup=None, title=None, displayall=False): gc.collect() results = [] begin = util.timer() count = 0 while True: +if setup is not None: +setup() with timeone() as item: r = func() count += 1 ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 8] perf: use the new setup function in "perfbookmarks"
# HG changeset patch # User Boris Feld # Date 1542669286 0 # Mon Nov 19 23:14:46 2018 + # Node ID 73f19cded5865968e672a0e1d1b6aaaef2b77e18 # Parent 04a1a32aaaedc6a75117994dbf2823e39204fd7d # EXP-Topic perf-ignore # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 73f19cded586 perf: use the new setup function in "perfbookmarks" This command was picked arbitrarily to display the usefulness of the new feature. In my Mercurial repository (with very few bookmarks), moving cache cleanup in the dedicated setup function has a visible and stable effect on the benchmark number. before: ! wall 0.61 comb 0.00 user 0.00 sys 0.00 (median of 40837) after: ! wall 0.58 comb 0.00 user 0.00 sys 0.00 (median of 40500) diff --git a/contrib/perf.py b/contrib/perf.py --- a/contrib/perf.py +++ b/contrib/perf.py @@ -576,10 +576,12 @@ def perfbookmarks(ui, repo, **opts): """benchmark parsing bookmarks from disk to memory""" opts = _byteskwargs(opts) timer, fm = gettimer(ui, opts) -def d(): + +def s(): clearfilecache(repo, b'_bookmarks') +def d(): repo._bookmarks -timer(d) +timer(d, setup=s) fm.end() @command(b'perfbundleread', formatteropts, b'BUNDLE') ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 8] perf: make `clearfilecache` helper work with any object
# HG changeset patch # User Boris Feld # Date 1542646221 0 # Mon Nov 19 16:50:21 2018 + # Node ID ea3a9f11bceef210118fc368908f4cfcd5a814fa # Parent 73f19cded5865968e672a0e1d1b6aaaef2b77e18 # EXP-Topic perf-ignore # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r ea3a9f11bcee perf: make `clearfilecache` helper work with any object If the object is not a localrepo, it won't have an `unfiltered` method (and won't need one). diff --git a/contrib/perf.py b/contrib/perf.py --- a/contrib/perf.py +++ b/contrib/perf.py @@ -457,11 +457,13 @@ def repocleartagscachefunc(repo): # utilities to clear cache -def clearfilecache(repo, attrname): -unfi = repo.unfiltered() -if attrname in vars(unfi): -delattr(unfi, attrname) -unfi._filecache.pop(attrname, None) +def clearfilecache(obj, attrname): +unfiltered = getattr(obj, 'unfiltered', None) +if unfiltered is not None: +obj = obj.unfiltered() +if attrname in vars(obj): +delattr(obj, attrname) +obj._filecache.pop(attrname, None) # perf commands ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 8] match: avoid translating glob to matcher multiple times for large sets
# HG changeset patch # User Boris Feld # Date 1542651672 0 # Mon Nov 19 18:21:12 2018 + # Node ID 03b60ccca50c77a552de85ac3402c5174539f150 # Parent aeea6de7f1a88e2e710ee01299fe4fe9e1f0d336 # EXP-Topic perf-ignore # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 03b60ccca50c match: avoid translating glob to matcher multiple times for large sets For hgignore with many globs, the resulting regexp might not fit under the 20K length limit. So the patterns need to be broken up in smaller pieces. Before this change, the logic was re-starting the full process from scratch for each smaller pieces, including the translation of globs into regexp. Effectively doing the work over and over. If the 20K limit is reached, we are likely in a case where there is many such glob, so exporting them is especially expensive and we should be careful not to do that work more than once. To work around this, we now translate glob to regexp once and for all. Then, we assemble the resulting individual regexp into valid blocks. This raises a very significant performance win for large `.hgignore file`: Before: ! wall 0.153153 comb 0.15 user 0.15 sys 0.00 (median of 66) After: ! wall 0.059793 comb 0.06 user 0.06 sys 0.00 (median of 100) diff --git a/mercurial/match.py b/mercurial/match.py --- a/mercurial/match.py +++ b/mercurial/match.py @@ -1184,25 +1184,40 @@ def _buildmatch(kindpats, globsuffix, li else: return regex, lambda f: any(mf(f) for mf in matchfuncs) +MAXRESIZE = 2 + +def _groupregexps(regexps): +return '(?:%s)' % '|'.join(regexps) + def _buildregexmatch(kindpats, globsuffix): """Build a match function from a list of kinds and kindpats, return regexp string and a matcher function.""" try: -regex = '(?:%s)' % '|'.join([_regex(k, p, globsuffix) - for (k, p, s) in kindpats]) -if len(regex) > 2: -raise OverflowError -return regex, _rematcher(regex) -except OverflowError: -# We're using a Python with a tiny regex engine and we -# made it explode, so we'll divide the pattern list in two -# until it works -l = len(kindpats) -if l < 2: -raise -regexa, a = _buildregexmatch(kindpats[:l//2], globsuffix) -regexb, b = _buildregexmatch(kindpats[l//2:], globsuffix) -return regex, lambda s: a(s) or b(s) +allgroups = [] +regexps = [_regex(k, p, globsuffix) for (k, p, s) in kindpats] +fullregexp = _groupregexps(regexps) + +startidx = 0 +groupsize = 3 +for idx, r in enumerate(regexps): +piecesize = len(r) +if piecesize > MAXRESIZE: +raise OverflowError +elif (groupsize + 1 + piecesize) > MAXRESIZE: +group = regexps[startidx:idx] +allgroups.append(_groupregexps(group)) +startidx = idx +groupsize = 3 +groupsize += piecesize + +if startidx == 0: +func = _rematcher(fullregexp) +else: +group = regexps[startidx:idx] +allgroups.append(_groupregexps(group)) +allmatchers = [_rematcher(g) for g in allgroups] +func = lambda s: any(m(s) for m in allmatchers) +return fullregexp, func except re.error: for k, p, s in kindpats: try: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 8] perf: add a perfignore command
# HG changeset patch # User Boris Feld # Date 1542669732 0 # Mon Nov 19 23:22:12 2018 + # Node ID aeea6de7f1a88e2e710ee01299fe4fe9e1f0d336 # Parent ea3a9f11bceef210118fc368908f4cfcd5a814fa # EXP-Topic perf-ignore # Available At https://bitbucket.org/octobus/mercurial-devel/ # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r aeea6de7f1a8 perf: add a perfignore command The command is meant to benchmark operations related to hgignore. Right now the command is benchmarking the loading time of the hgignore rules. diff --git a/contrib/perf.py b/contrib/perf.py --- a/contrib/perf.py +++ b/contrib/perf.py @@ -960,6 +960,23 @@ def perfchangeset(ui, repo, rev, **opts) timer(d) fm.end() +@command(b'perfignore', formatteropts) +def perfignore(ui, repo, **opts): +"""benchmark operation related to computing ignore""" +opts = _byteskwargs(opts) +timer, fm = gettimer(ui, opts) +dirstate = repo.dirstate + +def setupone(): +dirstate.invalidate() +clearfilecache(dirstate, b'_ignore') + +def runone(): +dirstate._ignore + +timer(runone, setup=setupone, title="load") +fm.end() + @command(b'perfindex', formatteropts) def perfindex(ui, repo, **opts): import mercurial.revlog diff --git a/tests/test-contrib-perf.t b/tests/test-contrib-perf.t --- a/tests/test-contrib-perf.t +++ b/tests/test-contrib-perf.t @@ -83,6 +83,7 @@ perfstatus perffncachewrite (no help text available) perfheads (no help text available) + perfignorebenchmark operation related to computing ignore perfindex (no help text available) perflinelogedits (no help text available) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 8 of 8] match: reformat `syntaxes` dictionary for better maintainability
# HG changeset patch
# User Boris Feld
# Date 1542647994 0
# Mon Nov 19 17:19:54 2018 +
# Node ID 865075265c79c576c53c1bd2b1a6d4b9f5719e2d
# Parent 8992601f1d1a54a42c984518097a246b49c06c12
# EXP-Topic perf-ignore
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
865075265c79
match: reformat `syntaxes` dictionary for better maintainability
diff --git a/mercurial/match.py b/mercurial/match.py
--- a/mercurial/match.py
+++ b/mercurial/match.py
@@ -1346,8 +1346,13 @@ def readpatternfile(filepath, warn, sour
(pattern, lineno, originalline). This is useful to debug ignore patterns.
'''
-syntaxes = {'re': 'relre:', 'regexp': 'relre:', 'glob': 'relglob:',
-'include': 'include', 'subinclude': 'subinclude'}
+syntaxes = {
+'re': 'relre:',
+'regexp': 'relre:',
+'glob': 'relglob:',
+'include': 'include',
+'subinclude': 'subinclude',
+}
syntax = 'relre:'
patterns = []
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 7 of 8] match: provide and use a quick way to escape a single byte
# HG changeset patch
# User Boris Feld
# Date 1542653684 0
# Mon Nov 19 18:54:44 2018 +
# Node ID 8992601f1d1a54a42c984518097a246b49c06c12
# Parent 03b60ccca50c77a552de85ac3402c5174539f150
# EXP-Topic perf-ignore
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
8992601f1d1a
match: provide and use a quick way to escape a single byte
The previous function has a lot of overhead (including being a function). In
the `_globre` case, we always escape a single byte. So we provide a dictionary
dedicated to this use case. We directly use the dictionary to avoid a function
call, these are expensive in Python.
Again, this raise a very significant performance gain:
Before: ! wall 0.059793 comb 0.06 user 0.06 sys 0.00 (median of 100)
After: ! wall 0.020390 comb 0.02 user 0.02 sys 0.00 (median of 146)
Total improvement for the full series:
Before: ! wall 0.153153 comb 0.15 user 0.15 sys 0.00 (median of 66)
After: ! wall 0.020390 comb 0.02 user 0.02 sys 0.00 (median of 146)
diff --git a/mercurial/match.py b/mercurial/match.py
--- a/mercurial/match.py
+++ b/mercurial/match.py
@@ -1057,14 +1057,14 @@ def _globre(pat):
i, n = 0, len(pat)
res = ''
group = 0
-escape = util.stringutil.reescape
+escape = util.stringutil.regexbytesescapemap.get
def peek():
return i < n and pat[i:i + 1]
while i < n:
c = pat[i:i + 1]
i += 1
if c not in '*?[{},\\':
-res += escape(c)
+res += escape(c, c)
elif c == '*':
if peek() == '*':
i += 1
@@ -1105,11 +1105,11 @@ def _globre(pat):
p = peek()
if p:
i += 1
-res += escape(p)
+res += escape(p, p)
else:
-res += escape(c)
+res += escape(c, c)
else:
-res += escape(c)
+res += escape(c, c)
return res
def _regex(kind, pat, globsuffix):
diff --git a/mercurial/utils/stringutil.py b/mercurial/utils/stringutil.py
--- a/mercurial/utils/stringutil.py
+++ b/mercurial/utils/stringutil.py
@@ -28,6 +28,7 @@ from .. import (
# which was part of Python 3.7.
_respecial = pycompat.bytestr(b'()[]{}?*+-|^$\\.&~# \t\n\r\v\f')
_regexescapemap = {ord(i): (b'\\' + i).decode('latin1') for i in _respecial}
+regexbytesescapemap = {i: (b'\\' + i) for i in _respecial}
def reescape(pat):
"""Drop-in replacement for re.escape."""
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5294: tests: make test-check-module-imports more robust
valentin.gatienbaron updated this revision to Diff 12581. REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D5294?vs=12580&id=12581 REVISION DETAIL https://phab.mercurial-scm.org/D5294 AFFECTED FILES contrib/import-checker.py CHANGE DETAILS diff --git a/contrib/import-checker.py b/contrib/import-checker.py --- a/contrib/import-checker.py +++ b/contrib/import-checker.py @@ -260,10 +260,12 @@ break else: stdlib_prefixes.add(dirname) +sourceroot = os.path.abspath(os.path.dirname(os.path.dirname(__file__))) for libpath in sys.path: -# We want to walk everything in sys.path that starts with -# something in stdlib_prefixes. -if not any(libpath.startswith(p) for p in stdlib_prefixes): +# We want to walk everything in sys.path that starts with something in +# stdlib_prefixes, but not directories from the hg sources. +if (os.path.abspath(libpath).startswith(sourceroot) +or not any(libpath.startswith(p) for p in stdlib_prefixes)): continue for top, dirs, files in os.walk(libpath): for i, d in reversed(list(enumerate(dirs))): To: valentin.gatienbaron, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Extra files in changelog "changed files" and bundles
Hi I see some odd and potentially related behaviour: The files list in changelog entries for merges sometimes contain files that didn't change, but where the manifest use the same filelog entry as one of the parents. It seems to often be related to merge of file moves. Also, sometimes there are different filelog entries, but debugrevlogindex shows rename entries (p1=null) with length 0 and where debugdata confirms that the content is the same as their p2. And because of these extra "changedfiles" entries, bundles sometimes contains too many files. Linkrev is used for pruning, but is too fragile to handle it correctly. (I have a PoC for pruning all filelog entries that are reused from the parents "common" and outside the bundled set.) (It also doesn't seem to use an optimal delta parent when bundling - that could perhaps mitigate it.) Sorry for the vague description. I have not been fully able to reproduce the problems - perhaps because bugs have been fixed. Are you aware of changes/fixes that can explain this? Or hints about where to look or related problems? /Mads ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH v2] graft: introduce --base option for using custom base revision while merging
On 10/23/18 4:59 PM, Augie Fackler wrote: On Oct 14, 2018, at 11:15, Mads Kiilerich wrote: # HG changeset patch # User Mads Kiilerich # Date 1539529698 -7200 # Sun Oct 14 17:08:18 2018 +0200 # Node ID 258029c642d97ef663396476c63ce34dbef89b13 # Parent 38ac525b44c93fcadb3680d4ded56f1e5a0029b2 graft: introduce --base option for using custom base revision while merging queued Thanks. At the sprint you had mentioned maybe being able to use this to help backing out a merge - if you know the incantation for that maybe send that as a follow-up for the verbose help? IIRC, graft has other safety mechanisms that prevent using a descendant as base. The primary use of this is to "graft merges". Backout of merges should be possible with existing backout functionality. The change help hinted in that direction: + + For using non-ancestors as the base to backout changes, see the backout + command and the hidden --parent option. (But ok, backout could use more helpful help too.) /Mads ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 7 V4] sparse-revlog: add a `index_get_start` function in C
On Wed, 21 Nov 2018 16:50:35 +0100, Boris FELD wrote:
> > Sigh, PyLong API is so inconsistent on Python 2. Let's switch to
> > PyLong_AsLongLong(), which can handle PyInt as well. I think it's
> > okay to raise OverflowError if offset >= 1 << (63 - 16).
>
> Okay, so dropping the special case and directly using:
>
> offset = PyLong_AsLongLong(pylong);
> if (offset < 0){ … }
>
> Right?
Yep, something similar to the one I proposed for the PATCH 2.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 7 V4] sparse-revlog: add a `index_segment_span` function in C
On Wed, 21 Nov 2018 16:52:22 +0100, Boris FELD wrote:
>
> On 21/11/2018 13:27, Yuya Nishihara wrote:
> > On Tue, 20 Nov 2018 21:44:34 +0100, Boris Feld wrote:
> >> # HG changeset patch
> >> # User Boris Feld
> >> # Date 1541785396 -3600
> >> # Fri Nov 09 18:43:16 2018 +0100
> >> # Node ID 5bb00d137c14d311c91c7a4f32f58378cd9195ec
> >> # Parent 8863dc1f7d078827708d178bbbfee5519b4c9b0f
> >> # EXP-Topic sparse-perf
> >> # Available At https://bitbucket.org/octobus/mercurial-devel/
> >> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> >> 5bb00d137c14
> >> sparse-revlog: add a `index_segment_span` function in C
> >>
> >> We are about to implement a native version of `slicechunktodensity`. For
> >> clarity, we introduce the helper functions first. This new function
> >> provides
> >> an efficient way to retrieve some of the information needed by
> >> `slicechunktodensity`.
> >>
> >> diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
> >> --- a/mercurial/cext/revlog.c
> >> +++ b/mercurial/cext/revlog.c
> >> @@ -1068,6 +1068,27 @@ bail:
> >>return NULL;
> >> }
> >>
> >> +static inline int64_t
> >> +index_segment_span(indexObject *self, Py_ssize_t start_rev, Py_ssize_t
> >> end_rev)
> >> +{
> >> + int64_t start_offset;
> >> + int64_t end_offset;
> >> + int end_size;
> >> + start_offset = index_get_start(self, start_rev);
> >> + if (start_offset < 0) {
> >> + return -1;
> >> + }
> >> + end_offset = index_get_start(self, end_rev);
> >> + if (end_offset < 0) {
> >> + return -1;
> >> + }
> >> + end_size = index_get_length(self, end_rev);
> >> + if (end_size < 0) {
> >> + return -1;
> >> + }
> >> + return (end_offset - start_offset) + (int64_t)end_size;
> > Nit: maybe raise error if end_offset - start_offset) + (int64_t)end_size <
> > 0 ?
>
> Unless the revlog is corrupted, we have:
>
> start_rev <= end_rev
>
> and:
>
> start_offset <= end_offset
>
> So this value will be positive. If the revlog is corrupt I assume we'll
> have more troubles sooner.
>
> Or do you want extra checks on the above assertion?
I just want to make sure that the CPython function never return NULL without
setting an exception, even if index_segment_span() happens to return -1
due to revlog corruption.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
