D9567: copies: make calculating lazy for dir move detection's "addedfiles"
spectral created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
The information calculated here was only needed if (a) --debug was specified,
or
(b) a directory move was plausibly detected. With tree manifests (especially
in
my pathological repo and with our custom setup), pre-calculating the `u1` and
`u2` can be quite slow, and it's not even necessary in many cases. Let's delay
calculating it until we know it's actually necessary. This should have no
observable differences in output.
Performance
---
I ran a rebase command in my pathological repo, rebasing two nodes across
several public phase commits, but where no directory copies exist in any of
the
paths I'm tracking.
Before
--
Time (mean ± σ): 3.711 s ± 0.061 s[User: 0.3 ms, System: 1.5 ms]
Range (min … max):3.640 s … 3.827 s10 runs
After
-
Time (mean ± σ): 868.3 ms ± 10.1 ms[User: 0.5 ms, System: 1.2 ms]
Range (min … max): 856.6 ms … 883.6 ms10 runs
REPOSITORY
rHG Mercurial
BRANCH
default
REVISION DETAIL
https://phab.mercurial-scm.org/D9567
AFFECTED FILES
mercurial/copies.py
CHANGE DETAILS
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -896,18 +896,30 @@
)
# find interesting file sets from manifests
-addedinm1 = m1.filesnotin(mb, repo.narrowmatch())
-addedinm2 = m2.filesnotin(mb, repo.narrowmatch())
-u1 = sorted(addedinm1 - addedinm2)
-u2 = sorted(addedinm2 - addedinm1)
+cache = []
+def _get_addedfiles(idx):
+if not cache:
+addedinm1 = m1.filesnotin(mb, repo.narrowmatch())
+addedinm2 = m2.filesnotin(mb, repo.narrowmatch())
+u1 = sorted(addedinm1 - addedinm2)
+u2 = sorted(addedinm2 - addedinm1)
+cache.extend((u1, u2))
+return cache[idx]
-header = b" unmatched files in %s"
-if u1:
-repo.ui.debug(b"%s:\n %s\n" % (header % b'local', b"\n ".join(u1)))
-if u2:
-repo.ui.debug(b"%s:\n %s\n" % (header % b'other', b"\n ".join(u2)))
+u1fn = lambda: _get_addedfiles(0)
+u2fn = lambda: _get_addedfiles(1)
+if repo.ui.debugflag:
+u1 = u1fn()
+u2 = u2fn()
-if repo.ui.debugflag:
+header = b" unmatched files in %s"
+if u1:
+repo.ui.debug(b"%s:\n %s\n" %
+ (header % b'local', b"\n ".join(u1)))
+if u2:
+repo.ui.debug(b"%s:\n %s\n" %
+ (header % b'other', b"\n ".join(u2)))
+
renamedeleteset = set()
divergeset = set()
for dsts in diverge.values():
@@ -941,8 +953,8 @@
repo.ui.debug(b" checking for directory renames\n")
-dirmove1, movewithdir2 = _dir_renames(repo, c1, copy1, copies1, u2)
-dirmove2, movewithdir1 = _dir_renames(repo, c2, copy2, copies2, u1)
+dirmove1, movewithdir2 = _dir_renames(repo, c1, copy1, copies1, u2fn)
+dirmove2, movewithdir1 = _dir_renames(repo, c2, copy2, copies2, u1fn)
branch_copies1 = branch_copies(copy1, renamedelete1, dirmove1,
movewithdir1)
branch_copies2 = branch_copies(copy2, renamedelete2, dirmove2,
movewithdir2)
@@ -950,14 +962,15 @@
return branch_copies1, branch_copies2, diverge
-def _dir_renames(repo, ctx, copy, fullcopy, addedfiles):
+def _dir_renames(repo, ctx, copy, fullcopy, addedfilesfn):
"""Finds moved directories and files that should move with them.
ctx: the context for one of the sides
copy: files copied on the same side (as ctx)
fullcopy: files copied on the same side (as ctx), including those that
merge.manifestmerge() won't care about
-addedfiles: added files on the other side (compared to ctx)
+addedfilesfn: function returning added files on the other side (compared to
+ ctx)
"""
# generate a directory move map
invalid = set()
@@ -997,7 +1010,7 @@
movewithdir = {}
# check unaccounted nonoverlapping files against directory moves
-for f in addedfiles:
+for f in addedfilesfn():
if f not in fullcopy:
for d in dirmove:
if f.startswith(d):
To: spectral, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@46102: 21 new changesets
21 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/c80f9e3daec3 changeset: 46082:c80f9e3daec3 parent: 46078:c581b9ee22b1 user:Martin von Zweigbergk date:Wed Dec 09 09:54:49 2020 -0800 summary: share: remove unexpected heading from "verbose" container in help test https://www.mercurial-scm.org/repo/hg/rev/81c1f5d1801f changeset: 46083:81c1f5d1801f user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 02 13:55:17 2020 +0530 summary: procutils: don't try to get `.buffer` if sys.stdin is None https://www.mercurial-scm.org/repo/hg/rev/7e1b4154cdca changeset: 46084:7e1b4154cdca user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 02 14:19:09 2020 +0530 summary: dispatch: disable line ending normalization on sys.stdin if its None https://www.mercurial-scm.org/repo/hg/rev/e0866c047e64 changeset: 46085:e0866c047e64 user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 02 14:27:45 2020 +0530 summary: tests: conditionalize output in test-ssh.t with chg+py3 https://www.mercurial-scm.org/repo/hg/rev/ac9de799d390 changeset: 46086:ac9de799d390 user:Pulkit Goyal <7895pul...@gmail.com> date:Thu Dec 03 17:18:49 2020 +0530 summary: commandserver: handle IOError related to flushing of streams https://www.mercurial-scm.org/repo/hg/rev/64292addbe67 changeset: 46087:64292addbe67 user:Martin von Zweigbergk date:Wed Dec 09 18:31:19 2020 -0800 summary: diff: add --from and --to flags as clearer alternative to -r -r https://www.mercurial-scm.org/repo/hg/rev/8837498ae6e0 changeset: 46088:8837498ae6e0 user:Martin von Zweigbergk date:Wed Dec 09 18:51:52 2020 -0800 summary: docs: prefer `hg diff --from/--to` over `-r` https://www.mercurial-scm.org/repo/hg/rev/8ff2d8359d0f changeset: 46089:8ff2d8359d0f user:Simon Sapin date:Mon Dec 07 18:06:53 2020 +0100 summary: persistent-nodemap: properly ignore non-existent `.nd` data file https://www.mercurial-scm.org/repo/hg/rev/9eb07ab3f2d4 changeset: 46090:9eb07ab3f2d4 user:Simon Sapin date:Fri Dec 04 17:27:10 2020 +0100 summary: rhg: use persistent nodemap when available https://www.mercurial-scm.org/repo/hg/rev/af3a6900f893 changeset: 46091:af3a6900f893 user:Matt Harbison date:Wed Dec 09 12:57:40 2020 -0500 summary: run-tests: fix `HGTESTEXTRAEXTENSIONS` with py3 https://www.mercurial-scm.org/repo/hg/rev/08fd76a553c9 changeset: 46092:08fd76a553c9 user:Matt Harbison date:Wed Dec 09 15:50:59 2020 -0500 summary: run-tests: configure the environment to expand `~` properly with Windows py38+ https://www.mercurial-scm.org/repo/hg/rev/224af78021de changeset: 46093:224af78021de user:Matt Harbison date:Wed Dec 09 18:21:16 2020 -0500 summary: windows: continue looking at `%HOME%` for user config files with py3.8+ https://www.mercurial-scm.org/repo/hg/rev/1ced08423d59 changeset: 46094:1ced08423d59 user:Matt Harbison date:Tue Dec 08 10:51:05 2020 -0500 summary: extensions: avoid including `__index__` in the disabled extension list https://www.mercurial-scm.org/repo/hg/rev/93e09d370003 changeset: 46095:93e09d370003 user:Kyle Lippincott date:Thu Dec 03 14:39:39 2020 -0800 summary: treemanifest: stop storing full path for each item in manifest._lazydirs https://www.mercurial-scm.org/repo/hg/rev/4d5e2fd53707 changeset: 46096:4d5e2fd53707 user:Joerg Sonnenberger date:Sat Dec 05 23:35:55 2020 +0100 summary: singlehead: introduce option to restrict to public changes https://www.mercurial-scm.org/repo/hg/rev/1b5e0d0bdb05 changeset: 46097:1b5e0d0bdb05 user:Matt Harbison date:Tue Dec 08 12:43:18 2020 -0500 summary: hghave: update the check for virtualenv https://www.mercurial-scm.org/repo/hg/rev/5510e2ac213f changeset: 46098:5510e2ac213f user:Martin von Zweigbergk date:Tue Dec 08 22:59:17 2020 -0800 summary: simplemerge: work with opts as native strings instead of bytes https://www.mercurial-scm.org/repo/hg/rev/fd75e5c53ec3 changeset: 46099:fd75e5c53ec3 user:Martin von Zweigbergk date:Tue Dec 08 23:05:53 2020 -0800 summary: simplemerge: avoid quadratic concatenation when building output text https://www.mercurial-scm.org/repo/hg/rev/a771ffc378a8 changeset: 46100:a771ffc378a8 user:Martin von Zweigbergk date:Wed Dec 09 00:00:19 2020 -0800 summary: simplemerge: write output only once it's complete https://www.mercurial-scm.org/repo/hg/rev/49b6910217f9 changeset: 46101:49b6910217f9 user:Pulkit Goyal <7895pul...@gmail.com> date:Thu Dec 10 13:51:56 2020 +0530 summary: dispatch: move IOError handling and flushing of streams to `dispatch()` https://www.mercurial-scm.org/repo/hg/rev/7ce24d3761e8 changeset:
D9566: debugdiscovery: fix swapped heads and roots
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY Patch provided without comment… REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9566 AFFECTED FILES mercurial/debugcommands.py CHANGE DETAILS diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py --- a/mercurial/debugcommands.py +++ b/mercurial/debugcommands.py @@ -1058,8 +1058,8 @@ data[b'nb-revs'] = len(all) data[b'nb-revs-common'] = len(common) data[b'nb-revs-missing'] = len(missing) -data[b'nb-missing-heads'] = len(roots_missing) -data[b'nb-missing-roots'] = len(heads_missing) +data[b'nb-missing-heads'] = len(heads_missing) +data[b'nb-missing-roots'] = len(roots_missing) data[b'nb-ini_und'] = len(initial_undecided) data[b'nb-ini_und-heads'] = len(heads_initial_undecided) data[b'nb-ini_und-roots'] = len(roots_initial_undecided) To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9565: debugdiscovery: display the number of roundtrip used
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY This is a good metric of the complexity of a discovery process. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9565 AFFECTED FILES mercurial/debugcommands.py mercurial/setdiscovery.py mercurial/treediscovery.py tests/test-setdiscovery.t CHANGE DETAILS diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t --- a/tests/test-setdiscovery.t +++ b/tests/test-setdiscovery.t @@ -44,6 +44,7 @@ searching for changes unpruned common: 01241442b3c2 66f7d451a68b b5714e113bc0 elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 2 also local heads: 2 @@ -75,6 +76,7 @@ searching for changes all local changesets known remotely elapsed time: * seconds (glob) + round-trips: 1 heads summary: total common heads: 2 also local heads: 2 @@ -106,6 +108,7 @@ searching for changes all local changesets known remotely elapsed time: * seconds (glob) + round-trips: 1 heads summary: total common heads: 1 also local heads: 1 @@ -136,6 +139,7 @@ searching for changes unpruned common: 01241442b3c2 b5714e113bc0 elapsed time: * seconds (glob) + round-trips: 1 heads summary: total common heads: 2 also local heads: 1 @@ -167,6 +171,7 @@ searching for changes all remote heads known locally elapsed time: * seconds (glob) + round-trips: 1 heads summary: total common heads: 2 also local heads: 1 @@ -198,6 +203,7 @@ searching for changes all remote heads known locally elapsed time: * seconds (glob) + round-trips: 1 heads summary: total common heads: 2 also local heads: 1 @@ -235,6 +241,7 @@ searching for changes unpruned common: bebd167eb94d elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 1 also local heads: 1 @@ -269,6 +276,7 @@ query 2; still undecided: 29, sample size is: 29 2 total queries in *.s (glob) elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 1 also local heads: 1 @@ -303,6 +311,7 @@ query 2; still undecided: 31, sample size is: 31 2 total queries in *.s (glob) elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 1 also local heads: 0 @@ -333,6 +342,7 @@ searching for changes unpruned common: 66f7d451a68b bebd167eb94d elapsed time: * seconds (glob) + round-trips: 4 heads summary: total common heads: 1 also local heads: 0 @@ -367,6 +377,7 @@ query 2; still undecided: 2, sample size is: 2 2 total queries in *.s (glob) elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 1 also local heads: 0 @@ -401,6 +412,7 @@ query 2; still undecided: 2, sample size is: 2 2 total queries in *.s (glob) elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 1 also local heads: 0 @@ -437,6 +449,7 @@ searching for changes unpruned common: 2dc09a01254d elapsed time: * seconds (glob) + round-trips: 4 heads summary: total common heads: 1 also local heads: 1 @@ -471,6 +484,7 @@ query 2; still undecided: 29, sample size is: 29 2 total queries in *.s (glob) elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 1 also local heads: 1 @@ -505,6 +519,7 @@ query 2; still undecided: 31, sample size is: 31 2 total queries in *.s (glob) elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 1 also local heads: 0 @@ -535,6 +550,7 @@ searching for changes unpruned common: 2dc09a01254d 66f7d451a68b elapsed time: * seconds (glob) + round-trips: 4 heads summary: total common heads: 1 also local heads: 0 @@ -569,6 +585,7 @@ query 2; still undecided: 29, sample size is: 29 2 total queries in *.s (glob) elapsed time: * seconds (glob) + round-trips: 2 heads summary: total common heads: 1 also local heads: 0 @@ -603,6 +620,7 @@ query 2; still undecided: 29, sample size
