D9802: copies: explicitly filter out existing file in graftcopies
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY The `graftcopies` function does something very strange (maybe even wrong), it calls `_filter` with a pair of changeset that does not match the one used to compute the copies informations. We are about to do some rework of `_filter` to make it closer to its documented intent and fix a couple of bug. This means some of the logic that only make sense for graft need to go somewhere else. We add the extra filtering with proper documentation to `graftcopies`. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9802 AFFECTED FILES mercurial/copies.py CHANGE DETAILS diff --git a/mercurial/copies.py b/mercurial/copies.py --- a/mercurial/copies.py +++ b/mercurial/copies.py @@ -1220,6 +1220,12 @@ by merge.update(). """ new_copies = pathcopies(base, ctx) -_filter(wctx.p1(), wctx, new_copies) +parent = wctx.p1() +_filter(parent, wctx, new_copies) +# extra filtering to drop copy information for files that existed before +# the graft (otherwise we would create merge filelog for non-merge commit +for dest, __ in list(new_copies.items()): +if dest in parent: +del new_copies[dest] for dst, src in pycompat.iteritems(new_copies): wctx[dst].markcopied(src) To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9801: discovery: add a devel.discovery.exchange-heads
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
Currently all discovery start with testing local and remote heads. For
analysis
purpose we make it possible to disable that initial "handshake" and start
discovery with the whole repository as undecided.
REPOSITORY
rHG Mercurial
BRANCH
default
REVISION DETAIL
https://phab.mercurial-scm.org/D9801
AFFECTED FILES
mercurial/configitems.py
mercurial/setdiscovery.py
tests/test-setdiscovery.t
CHANGE DETAILS
diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t
--- a/tests/test-setdiscovery.t
+++ b/tests/test-setdiscovery.t
@@ -1412,23 +1412,22 @@
missing:1040
common heads: 3ee37d65064a
- $ hg -R a debugdiscovery b --debug --config devel.discovery.randomize=false
--config devel.discovery.grow-sample.rate=1.01
+ $ hg -R a debugdiscovery b --debug --config
devel.discovery.exchange-heads=false --config devel.discovery.randomize=false
--config devel.discovery.grow-sample.rate=1.01
comparing with b
- query 1; heads
searching for changes
- taking quick initial sample
- query 2; still undecided: 1080, sample size is: 100
sampling from both directions
- query 3; still undecided: 980, sample size is: 200
+ query 1; still undecided: 1340, sample size is: 200
+ sampling from both directions
+ query 2; still undecided: 795, sample size is: 202
sampling from both directions
- query 4; still undecided: 497, sample size is: 202
+ query 3; still undecided: 525, sample size is: 204
sampling from both directions
- query 5; still undecided: 294, sample size is: 204
+ query 4; still undecided: 252, sample size is: 206
sampling from both directions
- query 6; still undecided: 90, sample size is: 90
- 6 total queries in *s (glob)
+ query 5; still undecided: 44, sample size is: 44
+ 5 total queries in *s (glob)
elapsed time: * seconds (glob)
- round-trips: 6
+ round-trips: 5
heads summary:
total common heads: 1
also local heads: 0
diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py
--- a/mercurial/setdiscovery.py
+++ b/mercurial/setdiscovery.py
@@ -314,6 +314,8 @@
else:
ownheads = [rev for rev in cl.headrevs() if rev != nullrev]
+initial_head_exchange = ui.configbool(b'devel',
b'discovery.exchange-heads')
+
# We also ask remote about all the local heads. That set can be arbitrarily
# large, so we used to limit it size to `initialsamplesize`. We no longer
# do as it proved counter productive. The skipped heads could lead to a
@@ -365,33 +367,39 @@
# graph (with many heads) attached to, but very independant to a the
# "simple" graph on the server. This is a fairly usual case and have
# not been met in the wild so far.
-if remote.limitedarguments:
-sample = _limitsample(ownheads, initialsamplesize)
-# indices between sample and externalized version must match
-sample = list(sample)
-else:
-sample = ownheads
+if initial_head_exchange:
+if remote.limitedarguments:
+sample = _limitsample(ownheads, initialsamplesize)
+# indices between sample and externalized version must match
+sample = list(sample)
+else:
+sample = ownheads
-ui.debug(b"query 1; heads\n")
-roundtrips += 1
-with remote.commandexecutor() as e:
-fheads = e.callcommand(b'heads', {})
-fknown = e.callcommand(
-b'known',
-{
-b'nodes': [clnode(r) for r in sample],
-},
-)
+ui.debug(b"query 1; heads\n")
+roundtrips += 1
+with remote.commandexecutor() as e:
+fheads = e.callcommand(b'heads', {})
+fknown = e.callcommand(
+b'known',
+{
+b'nodes': [clnode(r) for r in sample],
+},
+)
+
+srvheadhashes, yesno = fheads.result(), fknown.result()
-srvheadhashes, yesno = fheads.result(), fknown.result()
-
-if audit is not None:
-audit[b'total-roundtrips'] = 1
+if audit is not None:
+audit[b'total-roundtrips'] = 1
-if cl.tip() == nullid:
-if srvheadhashes != [nullid]:
-return [nullid], True, srvheadhashes
-return [nullid], False, []
+if cl.tip() == nullid:
+if srvheadhashes != [nullid]:
+return [nullid], True, srvheadhashes
+return [nullid], False, []
+else:
+# we still need the remote head for the function return
+with remote.commandexecutor() as e:
+fheads = e.callcommand(b'heads', {})
+srvheadhashes = fheads.result()
# start actual discovery (we note this before the next "if" for
# compatibility reasons)
@@ -408,15
D9799: discovery: add a discovery.grow-sample.rate
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY This allow to control the effect of the growth rate on the discovery process while doing analysis. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9799 AFFECTED FILES mercurial/configitems.py mercurial/setdiscovery.py tests/test-setdiscovery.t CHANGE DETAILS diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t --- a/tests/test-setdiscovery.t +++ b/tests/test-setdiscovery.t @@ -1412,6 +1412,48 @@ missing:1040 common heads: 3ee37d65064a + $ hg -R a debugdiscovery b --debug --config devel.discovery.randomize=false --config devel.discovery.grow-sample.rate=1.01 + comparing with b + query 1; heads + searching for changes + taking quick initial sample + query 2; still undecided: 1080, sample size is: 100 + sampling from both directions + query 3; still undecided: 980, sample size is: 200 + sampling from both directions + query 4; still undecided: 497, sample size is: 202 + sampling from both directions + query 5; still undecided: 294, sample size is: 204 + sampling from both directions + query 6; still undecided: 90, sample size is: 90 + 6 total queries in *s (glob) + elapsed time: * seconds (glob) + round-trips: 6 + heads summary: +total common heads: 1 + also local heads: 0 + also remote heads: 0 + both: 0 +local heads: 260 + common:0 + missing: 260 +remote heads:1 + common:0 + unknown: 1 + local changesets: 1340 +common:300 + heads: 1 + roots: 1 +missing: 1040 + heads: 260 + roots: 260 +first undecided set: 1340 + heads: 260 + roots: 1 + common: 300 + missing:1040 + common heads: 3ee37d65064a + Test actual protocol when pulling one new head in addition to common heads $ hg clone -U b c diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py --- a/mercurial/setdiscovery.py +++ b/mercurial/setdiscovery.py @@ -290,7 +290,6 @@ fullsamplesize=200, abortwhenunrelated=True, ancestorsof=None, -samplegrowth=1.05, audit=None, ): """Return a tuple (common, anyincoming, remoteheads) used to identify @@ -300,6 +299,9 @@ will be updated with extra data about the discovery, this is useful for debug. """ + +samplegrowth = float(ui.config(b'devel', b'discovery.grow-sample.rate')) + start = util.timer() roundtrips = 0 diff --git a/mercurial/configitems.py b/mercurial/configitems.py --- a/mercurial/configitems.py +++ b/mercurial/configitems.py @@ -707,6 +707,12 @@ b'discovery.grow-sample', default=True, ) +# discovery.grow-sample.rate control the rate at which the sample grow +coreconfigitem( +b'devel', +b'discovery.grow-sample.rate', +default=1.05, +) # If discovery.randomize is False, random sampling during discovery are # deterministic. It is meant for integration tests. coreconfigitem( To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9798: discovery: add a `devel', b'discovery.grow-sample`
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY That option make it possible to disable the "sample growing" behavior when doing analysis and comparison. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9798 AFFECTED FILES mercurial/configitems.py mercurial/setdiscovery.py CHANGE DETAILS diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py --- a/mercurial/setdiscovery.py +++ b/mercurial/setdiscovery.py @@ -418,9 +418,14 @@ # full blown discovery +# if the server has a limit to its arguments size, we can't grow the sample. +hard_limit_sample = remote.limitedarguments +grow_sample = local.ui.configbool(b'devel', b'discovery.grow-sample') +hard_limit_sample = hard_limit_sample and grow_sample + randomize = ui.configbool(b'devel', b'discovery.randomize') disco = partialdiscovery( -local, ownheads, remote.limitedarguments, randomize=randomize +local, ownheads, hard_limit_sample, randomize=randomize ) # treat remote heads (and maybe own heads) as a first implicit sample # response @@ -438,7 +443,7 @@ ui.debug(b"taking initial sample\n") samplefunc = disco.takefullsample targetsize = fullsamplesize -if not remote.limitedarguments: +if not hard_limit_sample: fullsamplesize = int(fullsamplesize * samplegrowth) else: # use even cheaper initial sample diff --git a/mercurial/configitems.py b/mercurial/configitems.py --- a/mercurial/configitems.py +++ b/mercurial/configitems.py @@ -700,6 +700,13 @@ b'debug.peer-request', default=False, ) +# If discovery.grow-sample is False, the sample size used in set discovery will +# not be increased through the process +coreconfigitem( +b'devel', +b'discovery.grow-sample', +default=True, +) # If discovery.randomize is False, random sampling during discovery are # deterministic. It is meant for integration tests. coreconfigitem( To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9800: discovery: move some debug output closer to were it belong
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
I assume these debug output, increment and comment drifted over time.
REPOSITORY
rHG Mercurial
BRANCH
default
REVISION DETAIL
https://phab.mercurial-scm.org/D9800
AFFECTED FILES
mercurial/setdiscovery.py
CHANGE DETAILS
diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py
--- a/mercurial/setdiscovery.py
+++ b/mercurial/setdiscovery.py
@@ -314,9 +314,6 @@
else:
ownheads = [rev for rev in cl.headrevs() if rev != nullrev]
-# early exit if we know all the specified remote heads already
-ui.debug(b"query 1; heads\n")
-roundtrips += 1
# We also ask remote about all the local heads. That set can be arbitrarily
# large, so we used to limit it size to `initialsamplesize`. We no longer
# do as it proved counter productive. The skipped heads could lead to a
@@ -375,6 +372,8 @@
else:
sample = ownheads
+ui.debug(b"query 1; heads\n")
+roundtrips += 1
with remote.commandexecutor() as e:
fheads = e.callcommand(b'heads', {})
fknown = e.callcommand(
@@ -409,6 +408,7 @@
except error.LookupError:
continue
+# early exit if we know all the specified remote heads already
if len(knownsrvheads) == len(srvheadhashes):
ui.debug(b"all remote heads known locally\n")
return srvheadhashes, False, srvheadhashes
To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9797: discovery: document the `devel.discovery.randomize` option
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY Gratuitous improvement as I was passing by this config section to add more. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9797 AFFECTED FILES mercurial/configitems.py CHANGE DETAILS diff --git a/mercurial/configitems.py b/mercurial/configitems.py --- a/mercurial/configitems.py +++ b/mercurial/configitems.py @@ -700,6 +700,8 @@ b'debug.peer-request', default=False, ) +# If discovery.randomize is False, random sampling during discovery are +# deterministic. It is meant for integration tests. coreconfigitem( b'devel', b'discovery.randomize', To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9796: copies: add an devel option to trace all files
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY Filelog based copy tracing only trace copy for file that have been added. This is a trade off between skipping some rare copy case in exchange for avoiding atrocious-to-the-point-of-unusable performance. The changeset centric copy tracing does not need this trade off and naturally trace all copy, include the one involving non-new files. In order to ease the comparison from both algorithm, we add a small devel option to trace copy for all files in the target revisions. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9796 AFFECTED FILES mercurial/configitems.py mercurial/copies.py tests/test-copies.t tests/test-copy.t CHANGE DETAILS diff --git a/tests/test-copy.t b/tests/test-copy.t --- a/tests/test-copy.t +++ b/tests/test-copy.t @@ -228,6 +228,17 @@ should show no copies $ hg st -C +note: since filelog based copy tracing only trace copy for new file, the copy information here is not displayed. + + $ hg status --copies --change . + M bar + +They are a devel option to walk all file and fine this information anyway. + + $ hg status --copies --change . --config devel.copy-tracing.trace-all-files=yes + M bar +foo + copy --after on an added file $ cp bar baz $ hg add baz diff --git a/tests/test-copies.t b/tests/test-copies.t --- a/tests/test-copies.t +++ b/tests/test-copies.t @@ -95,6 +95,8 @@ x -> y $ hg debugpathcopies 0 1 x -> y (no-filelog !) + $ hg debugpathcopies 0 1 --config devel.copy-tracing.trace-all-files=yes + x -> y Copy a file onto another file with same content. If metadata is stored in changeset, this does not produce a new filelog entry. The changeset's "files" entry should still list the file. @@ -113,6 +115,8 @@ x -> x2 $ hg debugpathcopies 0 1 x -> x2 (no-filelog !) + $ hg debugpathcopies 0 1 --config devel.copy-tracing.trace-all-files=yes + x -> x2 Rename file in a loop: x->y->z->x $ newrepo diff --git a/mercurial/copies.py b/mercurial/copies.py --- a/mercurial/copies.py +++ b/mercurial/copies.py @@ -152,13 +152,21 @@ if b.p1() == a and b.p2().node() == nullid: filesmatcher = matchmod.exact(b.files()) forwardmissingmatch = matchmod.intersectmatchers(match, filesmatcher) -missing = _computeforwardmissing(a, b, match=forwardmissingmatch) +if repo.ui.configbool(b'devel', b'copy-tracing.trace-all-files'): +missing = list(b.walk(match)) +# _computeforwardmissing(a, b, match=forwardmissingmatch) +if debug: +dbg(b'debug.copies: searching all files: %d\n' % len(missing)) +else: +missing = _computeforwardmissing(a, b, match=forwardmissingmatch) +if debug: +dbg( +b'debug.copies: missing files to search: %d\n' +% len(missing) +) ancestrycontext = a._repo.changelog.ancestors([b.rev()], inclusive=True) -if debug: -dbg(b'debug.copies: missing files to search: %d\n' % len(missing)) - for f in sorted(missing): if debug: dbg(b'debug.copies:tracing file: %s\n' % f) diff --git a/mercurial/configitems.py b/mercurial/configitems.py --- a/mercurial/configitems.py +++ b/mercurial/configitems.py @@ -610,6 +610,12 @@ b'check-relroot', default=False, ) +# Track copy information for all file, not just "added" one (very slow) +coreconfigitem( +b'devel', +b'copy-tracing.trace-all-files', +default=False, +) coreconfigitem( b'devel', b'default-date', To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9795: copies: simplify the conditional for _filter's case 3
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY The conditional is much simpler and the test are actually happier. This clarification of the conditional will also be necessary to properly support tracing more renames in a coming changeset. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9795 AFFECTED FILES mercurial/copies.py tests/test-copies.t CHANGE DETAILS diff --git a/tests/test-copies.t b/tests/test-copies.t --- a/tests/test-copies.t +++ b/tests/test-copies.t @@ -93,8 +93,8 @@ x y $ hg debugp1copies -r 1 x -> y -Incorrectly doesn't show the rename $ hg debugpathcopies 0 1 + x -> y (no-filelog !) Copy a file onto another file with same content. If metadata is stored in changeset, this does not produce a new filelog entry. The changeset's "files" entry should still list the file. @@ -111,8 +111,8 @@ x x2 $ hg debugp1copies -r 1 x -> x2 -Incorrectly doesn't show the rename $ hg debugpathcopies 0 1 + x -> x2 (no-filelog !) Rename file in a loop: x->y->z->x $ newrepo diff --git a/mercurial/copies.py b/mercurial/copies.py --- a/mercurial/copies.py +++ b/mercurial/copies.py @@ -59,14 +59,13 @@ # Cases 1, 3, and 5 are then removed by _filter(). for k, v in list(t.items()): -# remove copies from files that didn't exist -if v not in src: # case 5 +if k == v: # case 3 del t[k] -# remove criss-crossed copies -elif k in src and v in dst: +elif v not in src: # case 5 +# remove copies from files that didn't exist del t[k] -# remove copies to files that were then removed elif k not in dst: # case 1 +# remove copies to files that were then removed del t[k] To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9794: copies: clarify which case some conditional are handling
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY This make the function a bit clearer. The middle conditional get no label because we about about to remove it. See next changeset for details. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9794 AFFECTED FILES mercurial/copies.py CHANGE DETAILS diff --git a/mercurial/copies.py b/mercurial/copies.py --- a/mercurial/copies.py +++ b/mercurial/copies.py @@ -60,13 +60,13 @@ for k, v in list(t.items()): # remove copies from files that didn't exist -if v not in src: +if v not in src: # case 5 del t[k] # remove criss-crossed copies elif k in src and v in dst: del t[k] # remove copies to files that were then removed -elif k not in dst: +elif k not in dst: # case 1 del t[k] To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9793: copies: fix some comment in _filter
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY The scenario the comment describes match case 6, not case 2. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9793 AFFECTED FILES mercurial/copies.py CHANGE DETAILS diff --git a/mercurial/copies.py b/mercurial/copies.py --- a/mercurial/copies.py +++ b/mercurial/copies.py @@ -41,7 +41,7 @@ # When _chain()'ing copies in 'a' (from 'src' via some other commit 'mid') # with copies in 'b' (from 'mid' to 'dst'), we can get the different cases -# in the following table (not including trivial cases). For example, case 2 +# in the following table (not including trivial cases). For example, case 6 # is where a file existed in 'src' and remained under that name in 'mid' and # then was renamed between 'mid' and 'dst'. # To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9792: heptapod-ci: allow testing with docker image other than :latest
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY The project have a default config of latest for this variable. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9792 AFFECTED FILES contrib/heptapod-ci.yml CHANGE DETAILS diff --git a/contrib/heptapod-ci.yml b/contrib/heptapod-ci.yml --- a/contrib/heptapod-ci.yml +++ b/contrib/heptapod-ci.yml @@ -2,7 +2,7 @@ - tests - phabricator -image: registry.heptapod.net/mercurial/ci-images/mercurial-core +image: registry.heptapod.net/mercurial/ci-images/mercurial-core:$HG_CI_IMAGE_TAG variables: PYTHON: python To: marmoute, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial-devel | Pipeline #16520 has failed for branch/default | 707dec54
Your pipeline has failed. Project: mercurial-devel ( https://foss.heptapod.net/octobus/mercurial-devel ) Branch: branch/default ( https://foss.heptapod.net/octobus/mercurial-devel/-/commits/branch/default ) Commit: 707dec54 ( https://foss.heptapod.net/octobus/mercurial-devel/-/commit/707dec54394c5a6d616345f69d9d266a17fa815f ) Commit Message: revlog: migrate from PyEval_CallObject to PyObj... Commit Author: Augie Fackler Pipeline #16520 ( https://foss.heptapod.net/octobus/mercurial-devel/-/pipelines/16520 ) triggered by Administrator ( https://foss.heptapod.net/root ) had 8 failed builds. Job #152141 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/152141/raw ) Stage: tests Name: test-py2 Job #152145 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/152145/raw ) Stage: tests Name: test-py2-rust Job #152147 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/152147/raw ) Stage: tests Name: test-py2-chg Job #152143 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/152143/raw ) Stage: tests Name: test-py2-pure Job #152146 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/152146/raw ) Stage: tests Name: test-py3-rust Job #152144 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/152144/raw ) Stage: tests Name: test-py3-pure Job #152142 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/152142/raw ) Stage: tests Name: test-py3 Job #152148 ( https://foss.heptapod.net/octobus/mercurial-devel/-/jobs/152148/raw ) Stage: tests Name: test-py3-chg -- You're receiving this email because of your account on foss.heptapod.net. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9791: revlog: migrate from PyEval_CallObject to PyObject_Call
durin42 created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
The former was deprecated in 3.9.0.
REPOSITORY
rHG Mercurial
BRANCH
default
REVISION DETAIL
https://phab.mercurial-scm.org/D9791
AFFECTED FILES
mercurial/cext/revlog.c
CHANGE DETAILS
diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
--- a/mercurial/cext/revlog.c
+++ b/mercurial/cext/revlog.c
@@ -498,7 +498,7 @@
return -1;
}
- result = PyEval_CallObject(filter, arglist);
+ result = PyObject_Call(filter, arglist, NULL);
Py_DECREF(arglist);
if (!result) {
return -1;
To: durin42, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9790: fuzz: fix Makefile default PYTHON_CONFIG_FLAGS to be modern
durin42 created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY This is actually what we already do on oss-fuzz, so it's more correct as tests go. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9790 AFFECTED FILES contrib/fuzz/Makefile CHANGE DETAILS diff --git a/contrib/fuzz/Makefile b/contrib/fuzz/Makefile --- a/contrib/fuzz/Makefile +++ b/contrib/fuzz/Makefile @@ -11,7 +11,7 @@ LIB_FUZZING_ENGINE ?= standalone_fuzz_target_runner.o PYTHON_CONFIG ?= $$OUT/sanpy/bin/python-config -PYTHON_CONFIG_FLAGS ?= --ldflags +PYTHON_CONFIG_FLAGS ?= --ldflags --embed CXXFLAGS += -Wno-deprecated-register To: durin42, #hg-reviewers Cc: mercurial-patches, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D9789: clone: make sure we warm the cache after a clone
marmoute created this revision. Herald added a reviewer: hg-reviewers. Herald added a subscriber: mercurial-patches. REVISION SUMMARY This work around any deviciency/limitation of the clone process. In our case this ensure the persistent nodemap exist with valid content. Ideally, the cloning process would also do "the right thing". However since older server will never be able to do "the right thing". The local workaround will be necessary anyway. I am not worried by the performance impact of this as `hg clone` is non-instant on large repositories where is could matters. Warming the cache if they are already correct is very fast. And if they are not already warm, this seems like a good time to do so. This impact various test as more cache are now warmed sooner, all the change should be harmless. REPOSITORY rHG Mercurial BRANCH default REVISION DETAIL https://phab.mercurial-scm.org/D9789 AFFECTED FILES mercurial/hg.py tests/test-acl.t tests/test-blackbox.t tests/test-clone-uncompressed.t tests/test-clone.t tests/test-clonebundles.t tests/test-empty.t tests/test-eol-clone.t tests/test-hardlinks.t tests/test-lfs-serve-access.t tests/test-persistent-nodemap.t tests/test-share.t tests/test-ssh.t tests/test-stream-bundle-v2.t tests/test-tags.t tests/test-treemanifest.t tests/test-wireproto-exchangev2-shallow.t tests/test-wireproto-exchangev2.t CHANGE DETAILS diff --git a/tests/test-wireproto-exchangev2.t b/tests/test-wireproto-exchangev2.t --- a/tests/test-wireproto-exchangev2.t +++ b/tests/test-wireproto-exchangev2.t @@ -111,6 +111,7 @@ } updating the branch cache new changesets 3390ef850073:caa2a465451d (3 drafts) + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ cat clone-output | grep "received frame" @@ -239,6 +240,7 @@ } updating the branch cache new changesets 3390ef850073:4432d83626e8 + updating the branch cache (sent 6 HTTP requests and * bytes; received * bytes in responses) (glob) $ cat clone-output | grep "received frame" @@ -557,6 +559,7 @@ } updating the branch cache new changesets 3390ef850073:caa2a465451d (1 drafts) + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ cat clone-output | grep "received frame" @@ -758,6 +761,7 @@ } updating the branch cache new changesets 3390ef850073:97765fc3cd62 + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ cat clone-output | grep "received frame" @@ -872,6 +876,7 @@ } updating the branch cache new changesets 3390ef850073:97765fc3cd62 + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ cat clone-output | grep "received frame" @@ -988,6 +993,7 @@ } updating the branch cache new changesets 3390ef850073:97765fc3cd62 + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ cat clone-output | grep "received frame" @@ -1087,6 +1093,7 @@ } ] } + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ cat clone-output | grep "received frame" @@ -1183,6 +1190,7 @@ } ] } + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ cat clone-output | grep "received frame" diff --git a/tests/test-wireproto-exchangev2-shallow.t b/tests/test-wireproto-exchangev2-shallow.t --- a/tests/test-wireproto-exchangev2-shallow.t +++ b/tests/test-wireproto-exchangev2-shallow.t @@ -168,6 +168,7 @@ dir1/f: remote created -> g getting dir1/f 6 files updated, 0 files merged, 0 files removed, 0 files unresolved + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ sqlite3 -line client-shallow-1/.hg/store/db.sqlite << EOF @@ -333,6 +334,7 @@ dir0/d: remote created -> g getting dir0/d 2 files updated, 0 files merged, 0 files removed, 0 files unresolved + updating the branch cache (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob) $ sqlite3 -line client-shallow-narrow-1/.hg/store/db.sqlite << EOF @@ -469,6 +471,7 @@ dir1/f: remote created -> g getting dir1/f 6 files updated, 0 files merged, 0 files removed, 0 files unresolved + updating the branch cache (sent 6 HTTP requests and * bytes; received * bytes in responses) (glob) Incremental pull of shallow clone fetches new changesets diff --git a/tests/test-treemanifest.t b/tests/test-treemanifest.t --- a/tests/test-treemanifest.t +++ b/tests/test-treemanifest.t @@ -792,7 +792,7 @@ $ hg clone --config experimental.changegroup3=True --stream -U \ > http://localhost:$HGPORT1 stream-clone-basicstore streaming all changes - 21 files to
D9788: rust: use the bytes-cast crate to parse persistent nodemaps
SimonSapin created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
This crate casts pointers to custom structs, with compile-time safety checks,
for easy and efficient binary data parsing.
See https://crates.io/crates/bytes-cast and
https://docs.rs/bytes-cast/0.1.0/bytes_cast/
REPOSITORY
rHG Mercurial
BRANCH
default
REVISION DETAIL
https://phab.mercurial-scm.org/D9788
AFFECTED FILES
rust/Cargo.lock
rust/hg-core/Cargo.toml
rust/hg-core/src/revlog/nodemap.rs
rust/hg-core/src/revlog/nodemap_docket.rs
rust/hg-core/src/revlog/revlog.rs
CHANGE DETAILS
diff --git a/rust/hg-core/src/revlog/revlog.rs
b/rust/hg-core/src/revlog/revlog.rs
--- a/rust/hg-core/src/revlog/revlog.rs
+++ b/rust/hg-core/src/revlog/revlog.rs
@@ -29,6 +29,12 @@
UnknowDataFormat(u8),
}
+impl From for RevlogError {
+fn from(_: bytes_cast::FromBytesError) -> Self {
+RevlogError::Corrupted
+}
+}
+
/// Read only implementation of revlog.
pub struct Revlog {
/// When index and data are not interleaved: bytes of the revlog index.
diff --git a/rust/hg-core/src/revlog/nodemap_docket.rs
b/rust/hg-core/src/revlog/nodemap_docket.rs
--- a/rust/hg-core/src/revlog/nodemap_docket.rs
+++ b/rust/hg-core/src/revlog/nodemap_docket.rs
@@ -1,5 +1,5 @@
+use bytes_cast::{unaligned, BytesCast};
use memmap::Mmap;
-use std::convert::TryInto;
use std::path::{Path, PathBuf};
use super::revlog::RevlogError;
@@ -13,6 +13,16 @@
// TODO: keep here more of the data from `parse()` when we need it
}
+#[derive(BytesCast)]
+#[repr(C)]
+struct DocketHeader {
+uid_size: u8,
+_tip_rev: unaligned::U64Be,
+data_length: unaligned::U64Be,
+_data_unused: unaligned::U64Be,
+tip_node_size: unaligned::U64Be,
+}
+
impl NodeMapDocket {
/// Return `Ok(None)` when the caller should proceed without a persistent
/// nodemap:
@@ -36,25 +46,22 @@
Ok(bytes) => bytes,
};
-let mut input = if let Some((_VERSION, rest)) =
+let input = if let Some((_VERSION, rest)) =
docket_bytes.split_first()
{
rest
} else {
return Ok(None);
};
-let input = input;
-let uid_size = read_u8(input)? as usize;
-let _tip_rev = read_be_u64(input)?;
+let (header, rest) = DocketHeader::from_bytes(input)?;
+let uid_size = header.uid_size as usize;
// TODO: do we care about overflow for 4 GB+ nodemap files on 32-bit
// systems?
-let data_length = read_be_u64(input)? as usize;
-let _data_unused = read_be_u64(input)?;
-let tip_node_size = read_be_u64(input)? as usize;
-let uid = read_bytes(input, uid_size)?;
-let _tip_node = read_bytes(input, tip_node_size)?;
-
+let tip_node_size = header.tip_node_size.get() as usize;
+let data_length = header.data_length.get() as usize;
+let (uid, rest) = u8::slice_from_bytes(rest, uid_size)?;
+let (_tip_node, _rest) = u8::slice_from_bytes(rest, tip_node_size)?;
let uid =
std::str::from_utf8(uid).map_err(|_| RevlogError::Corrupted)?;
let docket = NodeMapDocket { data_length };
@@ -81,29 +88,6 @@
}
}
-fn read_bytes<'a>(
-input: &'a [u8],
-count: usize,
-) -> Result<&'a [u8], RevlogError> {
-if let Some(start) = input.get(..count) {
-*input = [count..];
-Ok(start)
-} else {
-Err(RevlogError::Corrupted)
-}
-}
-
-fn read_u8<'a>(input: &[u8]) -> Result {
-Ok(read_bytes(input, 1)?[0])
-}
-
-fn read_be_u64<'a>(input: &[u8]) -> Result {
-let array = read_bytes(input, std::mem::size_of::())?
-.try_into()
-.unwrap();
-Ok(u64::from_be_bytes(array))
-}
-
fn rawdata_path(docket_path: , uid: ) -> PathBuf {
let docket_name = docket_path
.file_name()
diff --git a/rust/hg-core/src/revlog/nodemap.rs
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -17,12 +17,13 @@
RevlogIndex, NULL_REVISION,
};
+use bytes_cast::{unaligned, BytesCast};
+use mem::size_of;
use std::cmp::max;
use std::fmt;
use std::mem;
use std::ops::Deref;
use std::ops::Index;
-use std::slice;
#[derive(Debug, PartialEq)]
pub enum NodeMapError {
@@ -149,7 +150,7 @@
/// Low level NodeTree [`Blocks`] elements
///
/// These are exactly as for instance on persistent storage.
-type RawElement = i32;
+type RawElement = unaligned::I32Be;
/// High level representation of values in NodeTree
/// [`Blocks`](struct.Block.html)
@@ -168,23 +169,24 @@
///
/// See [`Block`](struct.Block.html) for explanation about the encoding.
fn from(raw: RawElement) -> Element {
-if raw >= 0 {
-Element::Block(raw as usize)
-} else if raw == -1 {
+let int = raw.get();
+if int >= 0 {
+
D9787: hghave: clarify `sqlite` requirements
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.
REVISION SUMMARY
We need more than the python module, we also need the sqlite3 command line.
REPOSITORY
rHG Mercurial
BRANCH
default
REVISION DETAIL
https://phab.mercurial-scm.org/D9787
AFFECTED FILES
tests/hghave.py
CHANGE DETAILS
diff --git a/tests/hghave.py b/tests/hghave.py
--- a/tests/hghave.py
+++ b/tests/hghave.py
@@ -1006,7 +1006,7 @@
return 'fncache' in getrepofeatures()
-@check('sqlite', 'sqlite3 module is available')
+@check('sqlite', 'sqlite3 module and cli is available')
def has_sqlite():
try:
import sqlite3
To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@46223: 20 new changesets
20 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/36c05ab02232 changeset: 46204:36c05ab02232 user:msuo...@google.com date:Sat Jan 02 01:48:12 2021 -0500 summary: beautifygraph: change the current commit symbol https://www.mercurial-scm.org/repo/hg/rev/53d083fa1f83 changeset: 46205:53d083fa1f83 user:Pulkit Goyal <7895pul...@gmail.com> date:Mon Dec 14 16:03:15 2020 +0530 summary: upgrade: rename finddeficiences() to find_format_upgrades() https://www.mercurial-scm.org/repo/hg/rev/9540945e51fd changeset: 46206:9540945e51fd user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 30 14:56:34 2020 +0530 summary: actions: rename DEFICIENCY constant to FORMAT_VARIANT https://www.mercurial-scm.org/repo/hg/rev/e2139e071b5c changeset: 46207:e2139e071b5c user:Pulkit Goyal <7895pul...@gmail.com> date:Mon Dec 14 16:15:01 2020 +0530 summary: upgrade: add a missing space in status message https://www.mercurial-scm.org/repo/hg/rev/083438d6f403 changeset: 46208:083438d6f403 user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 16 12:39:15 2020 +0530 summary: upgrade: drop support for old style optimization names https://www.mercurial-scm.org/repo/hg/rev/a51d345f1404 changeset: 46209:a51d345f1404 user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 16 14:06:24 2020 +0530 summary: upgrade: move optimization addition to determineactions() https://www.mercurial-scm.org/repo/hg/rev/6b40aac4da8e changeset: 46210:6b40aac4da8e user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 16 14:16:10 2020 +0530 summary: upgrade: rename actions to upgrade_actions https://www.mercurial-scm.org/repo/hg/rev/2dbe6053d49a changeset: 46211:2dbe6053d49a user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 16 14:55:27 2020 +0530 summary: debugupgraderepo: minor documentation fix https://www.mercurial-scm.org/repo/hg/rev/c97d8e0406a6 changeset: 46212:c97d8e0406a6 user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 16 15:04:17 2020 +0530 summary: actions: introduce function to calculate downgrades https://www.mercurial-scm.org/repo/hg/rev/30310886d423 changeset: 46213:30310886d423 user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 16 14:00:41 2020 +0530 summary: upgrade: introduce post upgrade and downgrade message for improvements https://www.mercurial-scm.org/repo/hg/rev/5dfa837d933e changeset: 46214:5dfa837d933e user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 30 16:11:24 2020 +0530 summary: engine: refactor how total dstsize is calculated https://www.mercurial-scm.org/repo/hg/rev/82f3ee1a505f changeset: 46215:82f3ee1a505f user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 30 16:20:25 2020 +0530 summary: actions: store deltareuse mode of whole operation in UpgradeOperation https://www.mercurial-scm.org/repo/hg/rev/34efa84a43a1 changeset: 46216:34efa84a43a1 user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 30 16:29:33 2020 +0530 summary: engine: pass upgrade operation inside _clonerevlogs() https://www.mercurial-scm.org/repo/hg/rev/02df91e895bd changeset: 46217:02df91e895bd user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 30 16:33:49 2020 +0530 summary: engine: pass upgrade operation inside `_perform_clone()` https://www.mercurial-scm.org/repo/hg/rev/3f92a9bb80f0 changeset: 46218:3f92a9bb80f0 user:Pulkit Goyal <7895pul...@gmail.com> date:Wed Dec 30 16:39:35 2020 +0530 summary: engine: prevent multiple checking of re-delta-multibase https://www.mercurial-scm.org/repo/hg/rev/481d9aed669c changeset: 46219:481d9aed669c user:Pulkit Goyal <7895pul...@gmail.com> date:Thu Dec 31 14:10:25 2020 +0530 summary: engine: make hook point for extension a public function https://www.mercurial-scm.org/repo/hg/rev/1ca7865c245d changeset: 46220:1ca7865c245d user:Pulkit Goyal <7895pul...@gmail.com> date:Thu Dec 31 14:45:16 2020 +0530 summary: engine: refactor code to replace stores in separate function https://www.mercurial-scm.org/repo/hg/rev/0ca98ed828f9 changeset: 46221:0ca98ed828f9 user:Pulkit Goyal <7895pul...@gmail.com> date:Thu Dec 31 19:24:23 2020 +0530 summary: upgrade: remove unnecessary `is None` check https://www.mercurial-scm.org/repo/hg/rev/e22aed089567 changeset: 46222:e22aed089567 user:Pulkit Goyal <7895pul...@gmail.com> date:Thu Dec 31 19:42:10 2020 +0530 summary: upgrade: migrated -> upgraded in ui messages https://www.mercurial-scm.org/repo/hg/rev/24bfd98978da changeset: 46223:24bfd98978da bookmark:@ tag: tip user:Pulkit Goyal <7895pul...@gmail.com> date:Fri Jan 08 22:38:33 2021 +0530 summary: upgrade: demonstrate that a no-op upgrade still
