[Bug 5567] New: Deleted local tag makes a global one of the same name invisible
https://bz.mercurial-scm.org/show_bug.cgi?id=5567 Bug ID: 5567 Summary: Deleted local tag makes a global one of the same name invisible Product: Mercurial Version: 4.2 Hardware: All OS: All Status: UNCONFIRMED Severity: bug Priority: normal Component: Mercurial Assignee: bugzi...@mercurial-scm.org Reporter: matt_harbi...@yahoo.com CC: mercurial-devel@mercurial-scm.org $ hg init r1 $ cd r1 $ echo c1 > f1 $ hg ci -Aqm0 $ hg tag --local 1.0 $ hg tag --local --remove 1.0 $ cat .hg/localtags 40d85e9847f29e3ba286b83893287fdec7ddf8e9 1.0 40d85e9847f29e3ba286b83893287fdec7ddf8e9 1.0 1.0 $ hg tag 1.0 $ hg log -r 1.0 abort: unknown revision '1.0'! [255] $ cat .hgtags 1.0 40d85e9847f29e3ba286b83893287fdec7ddf8e9 1.0 $ cat .hg/cache/tags2 0 40d85e9847f29e3ba286b83893287fdec7ddf8e9 I'm a bit surprised that the 00..00 line is added for a local tag removal, but issue4839 has some reasoning there. This bug looks like it was previously reported as issue3339, which got a WONTFIX with the justification that localtags would be deprecated. Not sure if that's still true 5 years later. It took awhile to figure out that a local tag was interfering, but if we are stuck with this behavior, it should probably be documented. Also, a warning when creating the global tag would be nice (though that won't help tags that are pulled in). -- You are receiving this mail because: You are on the CC list for the bug. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] perf: always pass node to revlog.revision()
> On May 15, 2017, at 22:37, Gregory Szorcwrote: > > # HG changeset patch > # User Gregory Szorc > # Date 1494902231 25200 > # Mon May 15 19:37:11 2017 -0700 > # Node ID 4215b67716ae4a9f40c42ffbbfe9db2c7c1333f1 > # Parent 49621512d133723fcf54d9c5cd865a40a81446e9 > perf: always pass node to revlog.revision() queued thanks ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] hg: backout optimizing for treemanifests
> On May 15, 2017, at 21:56, Durham Goodewrote: > > # HG changeset patch > # User Durham Goode > # Date 1494899758 25200 > # Mon May 15 18:55:58 2017 -0700 > # Node ID 26ecf5c249c1ade001d7aeb403da1f1897edc18e > # Parent df3cf9422e1bb0287f281fd169fb0a8673bd547b > hg: backout optimizing for treemanifests > > It turns out that the files list is not sufficient to identify with revlogs > have > changed. In a merge commit, no files could've changed but directories would > have. For now let's just backout this optimization. Shucks, I should have caught this earlier. Queued. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: Minisprint at PyCon US?
> On May 15, 2017, at 21:58, Siddharth Agarwalwrote: > > PyCon US is happening later this week in Portland, OR. I'm sure some people > from the Mercurial community would be attending the main event, and there's > some time reserved next week for development sprints, so let's get together! > > I've put up a wiki page at > https://www.mercurial-scm.org/wiki/201705MiniSprintPyCon -- if you're > interested, please sign up! I'll be there. > > Thanks, > Siddharth > > ___ > Mercurial-devel mailing list > Mercurial-devel@mercurial-scm.org > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] perf: always pass node to revlog.revision()
Loos good to me. Excerpts from Gregory Szorc's message of 2017-05-15 19:37:59 -0700: > # HG changeset patch > # User Gregory Szorc> # Date 1494902231 25200 > # Mon May 15 19:37:11 2017 -0700 > # Node ID 4215b67716ae4a9f40c42ffbbfe9db2c7c1333f1 > # Parent 49621512d133723fcf54d9c5cd865a40a81446e9 > perf: always pass node to revlog.revision() > > I removed this in 73c3e226d2fc thinking it wasn't necessary. In fact, > we need to always pass a node so the code is compatible with revisions > before d7d64b89a65c. > > The new code uses a variable to avoid check-style complaining > about "r.revision(r.node(" patterns. > > diff --git a/contrib/perf.py b/contrib/perf.py > --- a/contrib/perf.py > +++ b/contrib/perf.py > @@ -865,7 +865,9 @@ def perfrevlog(ui, repo, file_=None, sta > dist = -1 * dist > > for x in xrange(beginrev, endrev, dist): > -rl.revision(x) > +# Old revisions don't support passing int. > +n = rl.node(x) > +rl.revision(n) > > timer, fm = gettimer(ui, opts) > timer(d) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] perf: always pass node to revlog.revision()
# HG changeset patch # User Gregory Szorc# Date 1494902231 25200 # Mon May 15 19:37:11 2017 -0700 # Node ID 4215b67716ae4a9f40c42ffbbfe9db2c7c1333f1 # Parent 49621512d133723fcf54d9c5cd865a40a81446e9 perf: always pass node to revlog.revision() I removed this in 73c3e226d2fc thinking it wasn't necessary. In fact, we need to always pass a node so the code is compatible with revisions before d7d64b89a65c. The new code uses a variable to avoid check-style complaining about "r.revision(r.node(" patterns. diff --git a/contrib/perf.py b/contrib/perf.py --- a/contrib/perf.py +++ b/contrib/perf.py @@ -865,7 +865,9 @@ def perfrevlog(ui, repo, file_=None, sta dist = -1 * dist for x in xrange(beginrev, endrev, dist): -rl.revision(x) +# Old revisions don't support passing int. +n = rl.node(x) +rl.revision(n) timer, fm = gettimer(ui, opts) timer(d) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 3] killdaemons: use posixfile to avoid intermittent unlink errors on Windows
On Mon, 15 May 2017 10:19:19 -0400, Yuya Nishiharawrote: On Mon, 15 May 2017 00:08:31 -0400, Matt Harbison wrote: # HG changeset patch # User Matt Harbison # Date 1494808413 14400 # Sun May 14 20:33:33 2017 -0400 # Node ID 809709930080937b6b56e5cad285798f29a10280 # Parent 024271e90987e5794dab6eb00844467065fae7e4 killdaemons: use posixfile to avoid intermittent unlink errors on Windows This is the aforementioned fix for the occasional cleanup error with #serve enabled. There are a handful of tests that neglect to kill the daemons they spawned, and this code is doing a last ditch reap of them. The test that got flagged was non-deterministic, and I've seen up to 3 fail in the same run. The problem with trying to import the mercurial module is that while it is available for running the individual *.t files, it is not in sys.path for run-tests.py itself. I couldn't think of any other way to make this work, and not affect sys.path for the indiviual tests. (The main source tree _is_ in PYTHONPATH when this is imported from run-tests.py.) diff --git a/tests/killdaemons.py b/tests/killdaemons.py --- a/tests/killdaemons.py +++ b/tests/killdaemons.py @@ -7,6 +7,16 @@ import sys import time +# PYTHONPATH contains the hg source tree when invoked from ./run-tests, but +# sys.path does not, and importing mercurial fails. The first import works from +# the .t files without editing the path. +try: +from mercurial.util import posixfile +except ImportError: +srctree = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +sys.path.insert(1, srctree) +from mercurial.util import posixfile sys.path is global. I slightly prefer moving this hack to run-tests.py if it's okay for run-tests.py to depend on Mercurial modules. I'm not sure if the dependency is OK, but I wasn't sure what else to do. (util.posixfile is a wrapper around osutil.posixfile, so it's not like it can be copy/pasted.) When I was working on the egg problem last weekend, I was sure that $TESTTMP was in $PYTHONPATH, but I don't see that happening here. (Part of the trickiness there is that it is a test of the test-runner, not running from the source tree.) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 3] killdaemons: fix error handling on Windows
On Mon, 15 May 2017 10:21:13 -0400, Yuya Nishiharawrote: On Mon, 15 May 2017 00:08:30 -0400, Matt Harbison wrote: # HG changeset patch # User Matt Harbison # Date 1494779534 14400 # Sun May 14 12:32:14 2017 -0400 # Node ID 024271e90987e5794dab6eb00844467065fae7e4 # Parent db0b470547d5e21f042390a204f09ae7d7303757 killdaemons: fix error handling on Windows After taking Adrian's suggestion[1] to use util.posixfile to avoid os.unlink() errors, the error changed from: Errored test-hgweb-json.t: Traceback (most recent call last): File "./run-tests.py", line 724, in run self.tearDown() File "./run-tests.py", line 805, in tearDown killdaemons(entry) File "./run-tests.py", line 540, in killdaemons logfn=vlog) File "...\tests\killdaemons.py", line 94, in killdaemons os.unlink(pidfile) WindowsError: [Error 32] The process cannot access the file because it is being used by another process: '...\\hgtests.zmpqj3\\child80\\daemon.pids' to: Errored test-largefiles.t: Traceback (most recent call last): File "./run-tests.py", line 724, in run self.tearDown() File "./run-tests.py", line 805, in tearDown killdaemons(entry) File "./run-tests.py", line 540, in killdaemons logfn=vlog) File "...\tests\killdaemons.py", line 91, in killdaemons kill(pid, logfn, tryhard) File "...\tests\killdaemons.py", line 55, in kill _check(ctypes.windll.kernel32.CloseHandle(handle)) File "...\tests\killdaemons.py", line 18, in _check raise ctypes.WinError(winerrno) WindowsError: [Error 6] The handle is invalid. The handle in question is for the process, not the file. That made me wonder why WaitForSingleObject() didn't raise an error, and it's because it isn't a BOOL return. I think it's better to define _check() function per return-value type. Win32 API doesn't state that the TRUE value is 1, even though it's practically 1. (For this method, a return of 0 is WAIT_OBJECT_0, and an error is signalled by returning (DWORD) 0x.) When I printed out that value (as returned by the function) with '0x%x', it printed '0x-1'. That's probably because we don't teach ctypes the function signature. A return value is taken as int by default. -if handle == 0: +if handle is None: _check(0, 87) # err 87 when process not found This could be _checkptr(handle, 87) for example. Since there are 3 different return types in play that would need specialized check functions (and one of those is special cased to WaitForSingleObject()), isn't that a bit over-engineered? I thought about just using _check(0) to handle WFSO() right as I was about to patchbomb it, and that would allow the previous 'ret == 0' test. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2] tests: fix up recent conditionalized output changes
# HG changeset patch # User Matt Harbison# Date 1494725402 14400 # Sat May 13 21:30:02 2017 -0400 # Node ID 7ab9a669a1ee06828d867c1f2cc04c4ea4b0a961 # Parent 4f390ad1639a769275cc94b4f449b89cdd457797 tests: fix up recent conditionalized output changes It looks like (!) can have surprising results matching back to the original output when adjacent lines change, probably because it uses the same code matching that allows (?) to skip missing output. 24f55686a63d ended up adding unconditionalized check*{exec,link} lines, duplicating the conditionalized lines. A Windows run wanted to delete the unconditionalized lines. This now runs on both Windows and Linux. diff --git a/tests/test-hardlinks.t b/tests/test-hardlinks.t --- a/tests/test-hardlinks.t +++ b/tests/test-hardlinks.t @@ -210,15 +210,12 @@ $ nlinksdir r4 2 r4/.hg/00changelog.i 2 r4/.hg/branch + 2 r4/.hg/cache/branch2-base + 2 r4/.hg/cache/branch2-served 2 r4/.hg/cache/checkisexec (execbit !) 3 r4/.hg/cache/checklink (?) ? r4/.hg/cache/checklink-target (glob) (symlink !) 2 r4/.hg/cache/checknoexec (execbit !) - 2 r4/.hg/cache/branch2-base - 2 r4/.hg/cache/branch2-served - 2 r4/.hg/cache/checkisexec - 2 r4/.hg/cache/checklink-target - 2 r4/.hg/cache/checknoexec 2 r4/.hg/cache/rbc-names-v1 2 r4/.hg/cache/rbc-revs-v1 2 r4/.hg/dirstate @@ -254,14 +251,11 @@ $ nlinksdir r4 2 r4/.hg/00changelog.i 1 r4/.hg/branch + 2 r4/.hg/cache/branch2-base + 2 r4/.hg/cache/branch2-served 2 r4/.hg/cache/checkisexec (execbit !) 2 r4/.hg/cache/checklink-target (symlink !) 2 r4/.hg/cache/checknoexec (execbit !) - 2 r4/.hg/cache/branch2-base - 2 r4/.hg/cache/branch2-served - 2 r4/.hg/cache/checkisexec - 2 r4/.hg/cache/checklink-target - 2 r4/.hg/cache/checknoexec 2 r4/.hg/cache/rbc-names-v1 2 r4/.hg/cache/rbc-revs-v1 1 r4/.hg/dirstate diff --git a/tests/test-tags.t b/tests/test-tags.t --- a/tests/test-tags.t +++ b/tests/test-tags.t @@ -671,13 +671,10 @@ Missing tags2* files means the cache wasn't written through the normal mechanism. $ ls tagsclient/.hg/cache + branch2-base checkisexec (execbit !) checklink (symlink !) checklink-target (symlink !) - branch2-base - checkisexec - checklink - checklink-target hgtagsfnodes1 Cache should contain the head only, even though other nodes have tags data @@ -699,13 +696,10 @@ 0.10:96ee1d7354c4 $ ls tagsclient/.hg/cache + branch2-base checkisexec (execbit !) checklink (symlink !) checklink-target (symlink !) - branch2-base - checkisexec - checklink - checklink-target hgtagsfnodes1 tags2-visible ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 2] test-casefolding: sync with latest code changes
# HG changeset patch # User Matt Harbison# Date 1494724556 14400 # Sat May 13 21:15:56 2017 -0400 # Node ID 4f390ad1639a769275cc94b4f449b89cdd457797 # Parent 78496ac300255e9996b3e282086661afc08af37c test-casefolding: sync with latest code changes This goes with c2380b448265. diff --git a/tests/test-casefolding.t b/tests/test-casefolding.t --- a/tests/test-casefolding.t +++ b/tests/test-casefolding.t @@ -28,6 +28,7 @@ a committing manifest committing changelog + updating the branch cache committed changeset 0:07f4944404050f47db2e5c5071e0e84e7a27bba9 Case-changing renames should work: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] style: ban [ foo == bar] bashism in tests
Jun Wuwrites: > Seems a good exercise for greg and smf. Sure, queued again! signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] cleanup: use set literals
Augie Facklerwrites: > On Mon, May 15, 2017 at 09:33:16AM -0700, Martin von Zweigbergk via > Mercurial-devel wrote: >> # HG changeset patch >> # User Martin von Zweigbergk >> # Date 1486774589 28800 >> # Fri Feb 10 16:56:29 2017 -0800 >> # Node ID 4c861a7d3a2fc1a18bafdcf931a4aade46398c2b >> # Parent 539cbe0f8fa38b771b9056d8e7ed51e999073f02 >> cleanup: use set literals > > +1, but I'll leave this for Sean or Greg to take so we can verify that > things are working correctly. Well, here goes nothing ... queued! https://www.youtube.com/watch?v=2PtvLTZS4Ik signature.asc Description: PGP signature ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: Unifying sparse and narrow "profiles"
> On May 9, 2017, at 15:17, Gregory Szorcwrote: > > How about this for a concrete proposal (with intent on landing some bits of > sparse in 4.3): > > 1) Drop a copy of Facebook's sparse.py into hgext/ along with > minimally-modified tests > 2) Mark extension as experimental and not subject to any BC guarantees > 3) Rename "sparse" command to "debugsparse" until we have a proper UX bikeshed > 4) Start moving code from extension into core where appropriate. This will > help flush out what considerations core needs to make for maintaining sparse > checkouts > 5) Attempt to stabilize the sparse profiles file format so daring consumers > can use sparse checkouts in 4.3 without major BC concerns from file formats > 6) Bikeshed and paint the shed > 7) Repeat #6 > 8) (a few releases later) drop experimental labeling and/or move sparse > checkout into core as a feature that is enabled by default (presumably this > is gated on a "sign-off" from narrow clone) This works for me, assuming it's okay for the Facebook end of things. I don't want to hurt them while we're getting this sorted. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2] strip: make tree stripping O(changes) instead of O(repo)
On Mon, May 15, 2017 at 1:39 PM, Durham Goodewrote: > > > On 5/15/17 1:32 PM, Martin von Zweigbergk wrote: >> >> On Mon, May 8, 2017 at 11:40 AM, Durham Goode wrote: >>> >>> # HG changeset patch >>> # User Durham Goode >>> # Date 1494268523 25200 >>> # Mon May 08 11:35:23 2017 -0700 >>> # Node ID 74881b9a39b2bab273d09009385e3c9ca717a13a >>> # Parent 5dec5907fe49a488d3ade272d4a5cf090914e59c >>> strip: make tree stripping O(changes) instead of O(repo) >>> >>> The old tree stripping logic iterated over every tree revlog in the repo >>> looking >>> for commits that had revs to be stripped. That's very inefficient in >>> large >>> repos. Instead, let's look at what files are touched by the strip and >>> only >>> inspect those revlogs. >> >> >> Sorry, I didn't notice this patch until today (because bisection of a >> test failure in narrowhg pointed me to it). This patch makes me a >> little worried that it'll have the same bugs as the initial version of >> changegroup generation for treemanifests had, i.e. that it forgets >> that merge commits can affect no files, but still affect directories >> (fixed in commit 1ac8ce13). I haven't tried to see if I can make it >> fail with this patch applied (the narrowhg failures was unrelated). > > > Ug, good point. I think this patch will fail in that case. Iterating over > all the revlogs really isn't a scalable option though, so it sounds like > maybe we need to actually do a walk of the trees for manifests that are > being stripped. Like, for each tree being stripped, diff them with their > parents and return all directories that are new. Yep, that's exactly what I did for changegroup generation in 1ac8ce13. For now, could you send a patch to back this change out and I'll queue it? > > We could probably add an argument to walksubtrees that only yields subtrees > that are different from the argument tree. This type of logic is useful for > determining what trees need to be bundled too. We do something like this in > our native implementation. > > >>> >>> diff --git a/mercurial/repair.py b/mercurial/repair.py >>> --- a/mercurial/repair.py >>> +++ b/mercurial/repair.py >>> @@ -238,11 +238,12 @@ def strip(ui, repo, nodelist, backup=Tru >>> def striptrees(repo, tr, striprev, files): >>> if 'treemanifest' in repo.requirements: # safe but unnecessary >>> # otherwise >>> -for unencoded, encoded, size in repo.store.datafiles(): >>> -if (unencoded.startswith('meta/') and >>> -unencoded.endswith('00manifest.i')): >>> -dir = unencoded[5:-12] >>> -repo.manifestlog._revlog.dirlog(dir).strip(striprev, tr) >>> +treerevlog = repo.manifestlog._revlog >>> +for dir in util.dirs(files): >> >> >> Note that util.dirs() does not return the root directory (I've often >> wanted to change that, and I may send a patch soon), so do you need to >> manually include it here? > > > The root manifest is already handled by the normal strip mechanism, so no > need to handle it here. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2] test-hardlink: do not test .hg/cache/checklink
# HG changeset patch # User Jun Wu# Date 1494880158 25200 # Mon May 15 13:29:18 2017 -0700 # Node ID 0e53ae9a7e47e6596e79fa40fe6e6dc3e37db343 # Parent caf620c5d0a0aa05e0bc36d3a97edc0e0a9b2b9c # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r 0e53ae9a7e47 test-hardlink: do not test .hg/cache/checklink Linux and BSD have different behavior on "os.link(src, dst)" where "src" is a symlink. That causes test difference. According to POSIX [1]: If path1 names a symbolic link, it is implementation-defined whether link() follows the symbolic link, or creates a new link to the symbolic link itself. So both behaviors are correct. This patch removes the trouble maker "checklink" to make the test pass on both platforms. [1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/link.html diff --git a/tests/test-hardlinks.t b/tests/test-hardlinks.t --- a/tests/test-hardlinks.t +++ b/tests/test-hardlinks.t @@ -215,4 +215,10 @@ Create hardlinked copy r4 of r3 (on Linu $ linkcp r3 r4 +'checklink' is produced by hardlinking a symlink, which is undefined whether +the symlink should be followed or not. It does behave differently on Linux and +BSD. Just remove it so the test pass on both platforms. + + $ rm -f r4/.hg/cache/checklink + r4 has hardlinks in the working dir (not just inside .hg): @@ -221,5 +227,4 @@ r4 has hardlinks in the working dir (not 2 r4/.hg/branch 2 r4/.hg/cache/checkisexec (execbit !) - 3 r4/.hg/cache/checklink (?) ? r4/.hg/cache/checklink-target (glob) (symlink !) 2 r4/.hg/cache/checknoexec (execbit !) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 2] test-hardlinks: unify two test files into one
# HG changeset patch # User Jun Wu# Date 1494879959 25200 # Mon May 15 13:25:59 2017 -0700 # Node ID caf620c5d0a0aa05e0bc36d3a97edc0e0a9b2b9c # Parent 366db10cb40f7fb1b804bfc87c8767d79a876489 # Available At https://bitbucket.org/quark-zju/hg-draft # hg pull https://bitbucket.org/quark-zju/hg-draft -r caf620c5d0a0 test-hardlinks: unify two test files into one The "whitelisted" test could be unified using "#if"s. diff --git a/tests/test-hardlinks-whitelisted.t b/tests/test-hardlinks-whitelisted.t deleted file mode 100644 --- a/tests/test-hardlinks-whitelisted.t +++ /dev/null @@ -1,391 +0,0 @@ -#require hardlink -#require hardlink-whitelisted - -This test is similar to test-hardlinks.t, but will only run on some filesystems -that we are sure to have known good hardlink supports (see issue4546 for an -example where the filesystem claims hardlink support but is actually -problematic). - - $ cat > nlinks.py < import sys - > from mercurial import util - > for f in sorted(sys.stdin.readlines()): - > f = f[:-1] - > print util.nlinks(f), f - > EOF - - $ nlinksdir() - > { - > find $1 -type f | python $TESTTMP/nlinks.py - > } - -Some implementations of cp can't create hardlinks (replaces 'cp -al' on Linux): - - $ cat > linkcp.py < from mercurial import util - > import sys - > util.copyfiles(sys.argv[1], sys.argv[2], hardlink=True) - > EOF - - $ linkcp() - > { - > python $TESTTMP/linkcp.py $1 $2 - > } - -Prepare repo r1: - - $ hg init r1 - $ cd r1 - - $ echo c1 > f1 - $ hg add f1 - $ hg ci -m0 - - $ mkdir d1 - $ cd d1 - $ echo c2 > f2 - $ hg add f2 - $ hg ci -m1 - $ cd ../.. - - $ nlinksdir r1/.hg/store - 1 r1/.hg/store/00changelog.i - 1 r1/.hg/store/00manifest.i - 1 r1/.hg/store/data/d1/f2.i - 1 r1/.hg/store/data/f1.i - 1 r1/.hg/store/fncache - 1 r1/.hg/store/phaseroots - 1 r1/.hg/store/undo - 1 r1/.hg/store/undo.backup.fncache - 1 r1/.hg/store/undo.backupfiles - 1 r1/.hg/store/undo.phaseroots - - -Create hardlinked clone r2: - - $ hg clone -U --debug r1 r2 --config progress.debug=true - linking: 1 - linking: 2 - linking: 3 - linking: 4 - linking: 5 - linking: 6 - linking: 7 - linked 7 files - -Create non-hardlinked clone r3: - - $ hg clone --pull r1 r3 - requesting all changes - adding changesets - adding manifests - adding file changes - added 2 changesets with 2 changes to 2 files - updating to branch default - 2 files updated, 0 files merged, 0 files removed, 0 files unresolved - - -Repos r1 and r2 should now contain hardlinked files: - - $ nlinksdir r1/.hg/store - 2 r1/.hg/store/00changelog.i - 2 r1/.hg/store/00manifest.i - 2 r1/.hg/store/data/d1/f2.i - 2 r1/.hg/store/data/f1.i - 2 r1/.hg/store/fncache - 1 r1/.hg/store/phaseroots - 1 r1/.hg/store/undo - 1 r1/.hg/store/undo.backup.fncache - 1 r1/.hg/store/undo.backupfiles - 1 r1/.hg/store/undo.phaseroots - - $ nlinksdir r2/.hg/store - 2 r2/.hg/store/00changelog.i - 2 r2/.hg/store/00manifest.i - 2 r2/.hg/store/data/d1/f2.i - 2 r2/.hg/store/data/f1.i - 2 r2/.hg/store/fncache - -Repo r3 should not be hardlinked: - - $ nlinksdir r3/.hg/store - 1 r3/.hg/store/00changelog.i - 1 r3/.hg/store/00manifest.i - 1 r3/.hg/store/data/d1/f2.i - 1 r3/.hg/store/data/f1.i - 1 r3/.hg/store/fncache - 1 r3/.hg/store/phaseroots - 1 r3/.hg/store/undo - 1 r3/.hg/store/undo.backupfiles - 1 r3/.hg/store/undo.phaseroots - - -Create a non-inlined filelog in r3: - - $ cd r3/d1 - >>> f = open('data1', 'wb') - >>> for x in range(1): - ... f.write("%s\n" % str(x)) - >>> f.close() - $ for j in 0 1 2 3 4 5 6 7 8 9; do - > cat data1 >> f2 - > hg commit -m$j - > done - $ cd ../.. - - $ nlinksdir r3/.hg/store - 1 r3/.hg/store/00changelog.i - 1 r3/.hg/store/00manifest.i - 1 r3/.hg/store/data/d1/f2.d - 1 r3/.hg/store/data/d1/f2.i - 1 r3/.hg/store/data/f1.i - 1 r3/.hg/store/fncache - 1 r3/.hg/store/phaseroots - 1 r3/.hg/store/undo - 1 r3/.hg/store/undo.backup.fncache - 1 r3/.hg/store/undo.backup.phaseroots - 1 r3/.hg/store/undo.backupfiles - 1 r3/.hg/store/undo.phaseroots - -Push to repo r1 should break up most hardlinks in r2: - - $ hg -R r2 verify - checking changesets - checking manifests - crosschecking files in changesets and manifests - checking files - 2 files, 2 changesets, 2 total revisions - - $ cd r3 - $ hg push - pushing to $TESTTMP/r1 (glob) - searching for changes - adding changesets - adding manifests - adding file changes - added 10 changesets with 10 changes to 1 files - - $ cd .. - - $ nlinksdir r2/.hg/store - 1 r2/.hg/store/00changelog.i - 1 r2/.hg/store/00manifest.i - 1 r2/.hg/store/data/d1/f2.i - 2 r2/.hg/store/data/f1.i - 2 r2/.hg/store/fncache - - $ hg -R r2 verify - checking changesets - checking manifests - crosschecking files in changesets and manifests - checking files - 2 files, 2 changesets, 2 total revisions - - - $ cd r1 - $ hg up - 1 files updated, 0 files merged, 0 files
Re: [PATCH 2 of 2] strip: make tree stripping O(changes) instead of O(repo)
On 5/15/17 1:32 PM, Martin von Zweigbergk wrote: On Mon, May 8, 2017 at 11:40 AM, Durham Goodewrote: # HG changeset patch # User Durham Goode # Date 1494268523 25200 # Mon May 08 11:35:23 2017 -0700 # Node ID 74881b9a39b2bab273d09009385e3c9ca717a13a # Parent 5dec5907fe49a488d3ade272d4a5cf090914e59c strip: make tree stripping O(changes) instead of O(repo) The old tree stripping logic iterated over every tree revlog in the repo looking for commits that had revs to be stripped. That's very inefficient in large repos. Instead, let's look at what files are touched by the strip and only inspect those revlogs. Sorry, I didn't notice this patch until today (because bisection of a test failure in narrowhg pointed me to it). This patch makes me a little worried that it'll have the same bugs as the initial version of changegroup generation for treemanifests had, i.e. that it forgets that merge commits can affect no files, but still affect directories (fixed in commit 1ac8ce13). I haven't tried to see if I can make it fail with this patch applied (the narrowhg failures was unrelated). Ug, good point. I think this patch will fail in that case. Iterating over all the revlogs really isn't a scalable option though, so it sounds like maybe we need to actually do a walk of the trees for manifests that are being stripped. Like, for each tree being stripped, diff them with their parents and return all directories that are new. We could probably add an argument to walksubtrees that only yields subtrees that are different from the argument tree. This type of logic is useful for determining what trees need to be bundled too. We do something like this in our native implementation. diff --git a/mercurial/repair.py b/mercurial/repair.py --- a/mercurial/repair.py +++ b/mercurial/repair.py @@ -238,11 +238,12 @@ def strip(ui, repo, nodelist, backup=Tru def striptrees(repo, tr, striprev, files): if 'treemanifest' in repo.requirements: # safe but unnecessary # otherwise -for unencoded, encoded, size in repo.store.datafiles(): -if (unencoded.startswith('meta/') and -unencoded.endswith('00manifest.i')): -dir = unencoded[5:-12] -repo.manifestlog._revlog.dirlog(dir).strip(striprev, tr) +treerevlog = repo.manifestlog._revlog +for dir in util.dirs(files): Note that util.dirs() does not return the root directory (I've often wanted to change that, and I may send a patch soon), so do you need to manually include it here? The root manifest is already handled by the normal strip mechanism, so no need to handle it here. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2] strip: make tree stripping O(changes) instead of O(repo)
On Mon, May 8, 2017 at 11:40 AM, Durham Goodewrote: > # HG changeset patch > # User Durham Goode > # Date 1494268523 25200 > # Mon May 08 11:35:23 2017 -0700 > # Node ID 74881b9a39b2bab273d09009385e3c9ca717a13a > # Parent 5dec5907fe49a488d3ade272d4a5cf090914e59c > strip: make tree stripping O(changes) instead of O(repo) > > The old tree stripping logic iterated over every tree revlog in the repo > looking > for commits that had revs to be stripped. That's very inefficient in large > repos. Instead, let's look at what files are touched by the strip and only > inspect those revlogs. Sorry, I didn't notice this patch until today (because bisection of a test failure in narrowhg pointed me to it). This patch makes me a little worried that it'll have the same bugs as the initial version of changegroup generation for treemanifests had, i.e. that it forgets that merge commits can affect no files, but still affect directories (fixed in commit 1ac8ce13). I haven't tried to see if I can make it fail with this patch applied (the narrowhg failures was unrelated). > > I don't have actual perf numbers, since internally we don't use a true > treemanifest, but simply iterating over hundreds of thousands of revlogs takes > many, many seconds, so this should help tremendously when stripping only a few > commits. > > diff --git a/mercurial/repair.py b/mercurial/repair.py > --- a/mercurial/repair.py > +++ b/mercurial/repair.py > @@ -238,11 +238,12 @@ def strip(ui, repo, nodelist, backup=Tru > def striptrees(repo, tr, striprev, files): > if 'treemanifest' in repo.requirements: # safe but unnecessary > # otherwise > -for unencoded, encoded, size in repo.store.datafiles(): > -if (unencoded.startswith('meta/') and > -unencoded.endswith('00manifest.i')): > -dir = unencoded[5:-12] > -repo.manifestlog._revlog.dirlog(dir).strip(striprev, tr) > +treerevlog = repo.manifestlog._revlog > +for dir in util.dirs(files): Note that util.dirs() does not return the root directory (I've often wanted to change that, and I may send a patch soon), so do you need to manually include it here? > +# If the revlog doesn't exist, this returns an empty revlog and > is a > +# no-op. > +rl = treerevlog.dirlog(dir) > +rl.strip(striprev, tr) > > def rebuildfncache(ui, repo): > """Rebuilds the fncache file from repo history. > ___ > Mercurial-devel mailing list > Mercurial-devel@mercurial-scm.org > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] repair: compress bundles with gzip instead of bzip2 (BC)
On 5/13/17 23:33, Gregory Szorc wrote: I*would* like to change the default to zstd (if available). However, that's a significant BC break and I don't feel comfortable changing it just yet. When we have official zstd revlog support and the repo is using it, I think it makes sense to use zstd for backup bundles. But for zlib-based revlogs, it's safer to avoid zstd for the time being. Even if the backup bundle uses gzip, could the temp bundle use zstd? IIRC the temp bundle can potentially be much larger than the backup bundle because it contains all of the commits with a rev number higher than the lowest one being stripped. - Siddharth ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] style: ban [ foo == bar] bashism in tests
Seems a good exercise for greg and smf. Excerpts from Augie Fackler's message of 2017-05-15 14:09:14 -0400: > # HG changeset patch > # User Augie Fackler> # Date 1494871682 14400 > # Mon May 15 14:08:02 2017 -0400 > # Node ID cce0fa53cc1401b5bb234acc3e846f0595febe4b > # Parent 366db10cb40f7fb1b804bfc87c8767d79a876489 > style: ban [ foo == bar] bashism in tests > > diff --git a/contrib/check-code.py b/contrib/check-code.py > --- a/contrib/check-code.py > +++ b/contrib/check-code.py > @@ -116,6 +116,7 @@ testpats = [ > (r'printf.*[^\\]\\x', "don't use printf \\x, use Python"), > (r'\$\(.*\)', "don't use $(expr), use `expr`"), > (r'rm -rf \*', "don't use naked rm -rf, target a directory"), > +(r'\[[^\]]+==', '[ foo == bar ] is a bashism, use [ foo = bar ] > instead'), > (r'(^|\|\s*)grep (-\w\s+)*[^|]*[(|]\w', > "use egrep for extended grep syntax"), > (r'/bin/', "don't use explicit paths for tools"), > diff --git a/tests/test-contrib-check-code.t b/tests/test-contrib-check-code.t > --- a/tests/test-contrib-check-code.t > +++ b/tests/test-contrib-check-code.t > @@ -152,6 +152,15 @@ > > $ function onwarn {} > warning: don't use 'function', use old style >[1] > + $ cat > error.t < + > $ [ foo == bar ] > + > EOF > + $ "$check_code" error.t > + error.t:1: > + > $ [ foo == bar ] > + [ foo == bar ] is a bashism, use [ foo = bar ] instead > + [1] > + $ rm error.t >$ cat > raise-format.py <> raise SomeException, message >> # this next line is okay ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] changelog: load pending file directly
On Sat, May 13, 2017 at 04:29:50PM -0700, Gregory Szorc wrote: > # HG changeset patch > # User Gregory Szorc> # Date 1494718003 25200 > # Sat May 13 16:26:43 2017 -0700 > # Node ID 53a93d88e75d42e3e71c821154ccc6b5ed4fd349 > # Parent 78496ac300255e9996b3e282086661afc08af37c > changelog: load pending file directly queued, thanks ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] style: ban [ foo == bar] bashism in tests
# HG changeset patch # User Augie Fackler# Date 1494871682 14400 # Mon May 15 14:08:02 2017 -0400 # Node ID cce0fa53cc1401b5bb234acc3e846f0595febe4b # Parent 366db10cb40f7fb1b804bfc87c8767d79a876489 style: ban [ foo == bar] bashism in tests diff --git a/contrib/check-code.py b/contrib/check-code.py --- a/contrib/check-code.py +++ b/contrib/check-code.py @@ -116,6 +116,7 @@ testpats = [ (r'printf.*[^\\]\\x', "don't use printf \\x, use Python"), (r'\$\(.*\)', "don't use $(expr), use `expr`"), (r'rm -rf \*', "don't use naked rm -rf, target a directory"), +(r'\[[^\]]+==', '[ foo == bar ] is a bashism, use [ foo = bar ] instead'), (r'(^|\|\s*)grep (-\w\s+)*[^|]*[(|]\w', "use egrep for extended grep syntax"), (r'/bin/', "don't use explicit paths for tools"), diff --git a/tests/test-contrib-check-code.t b/tests/test-contrib-check-code.t --- a/tests/test-contrib-check-code.t +++ b/tests/test-contrib-check-code.t @@ -152,6 +152,15 @@ > $ function onwarn {} warning: don't use 'function', use old style [1] + $ cat > error.t < $ [ foo == bar ] + > EOF + $ "$check_code" error.t + error.t:1: + > $ [ foo == bar ] + [ foo == bar ] is a bashism, use [ foo = bar ] instead + [1] + $ rm error.t $ cat > raise-format.py < raise SomeException, message > # this next line is okay ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 3 V2] py3: convert date and format arguments str before passing in time.strptime
On Mon, May 15, 2017 at 11:20:48PM +0530, Pulkit Goyal wrote: > # HG changeset patch > # User Pulkit Goyal <7895pul...@gmail.com> > # Date 1494026485 -19800 > # Sat May 06 04:51:25 2017 +0530 > # Node ID bffb17a42e8c185bb3d31f4489066448ce2e259a > # Parent 75245b989dd661164041794620770ad7726fe9cd > py3: convert date and format arguments str before passing in time.strptime queued patches 2 and 3, thanks ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 3 V2] py3: explicitly convert a list to bytes to pass in ui.debug
On Mon, May 15, 2017 at 11:20:46PM +0530, Pulkit Goyal wrote: > # HG changeset patch > # User Pulkit Goyal <7895pul...@gmail.com> > # Date 1493837589 -19800 > # Thu May 04 00:23:09 2017 +0530 > # Node ID d186d8361d0526413dd1399e92ccdb40f3ca2056 > # Parent 1ada3d18e7fbc9069910f2c036992d2f2b28e058 > py3: explicitly convert a list to bytes to pass in ui.debug > > Here pats is a list obviously. Since we can't pass unicodes to ui.debug, we > have to pass this as bytes. > > diff --git a/hgext/largefiles/overrides.py b/hgext/largefiles/overrides.py > --- a/hgext/largefiles/overrides.py > +++ b/hgext/largefiles/overrides.py > @@ -17,6 +17,7 @@ > from mercurial import ( > archival, > cmdutil, > +encoding, You add the encoding arg here, but never use it? I suspect there should be a .encode somewhere in here or similar? > error, > hg, > match as matchmod, > @@ -380,8 +381,8 @@ > r = origmatchfn(f) > return r > m.matchfn = lfmatchfn > - > -ui.debug('updated patterns: %s\n' % sorted(pats)) > +bytespats = '[' + ', '.join(sorted(pats)) + ']' > +ui.debug('updated patterns: %s\n' % bytespats) > return m, pats > > # For hg log --patch, the match object is used in two different senses: > ___ > Mercurial-devel mailing list > Mercurial-devel@mercurial-scm.org > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 3 V2] py3: convert date and format arguments str before passing in time.strptime
# HG changeset patch # User Pulkit Goyal <7895pul...@gmail.com> # Date 1494026485 -19800 # Sat May 06 04:51:25 2017 +0530 # Node ID bffb17a42e8c185bb3d31f4489066448ce2e259a # Parent 75245b989dd661164041794620770ad7726fe9cd py3: convert date and format arguments str before passing in time.strptime time.strptime() raises ValueError if the arguments are not str. Source Code: https://hg.python.org/cpython/file/3.5/Lib/_strptime.py#l307 diff --git a/mercurial/util.py b/mercurial/util.py --- a/mercurial/util.py +++ b/mercurial/util.py @@ -1937,7 +1937,8 @@ # elements are relative to today usenow = True -timetuple = time.strptime(date, format) +timetuple = time.strptime(encoding.strfromlocal(date), +encoding.strfromlocal(format)) localunixtime = int(calendar.timegm(timetuple)) if offset is None: # local timezone ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 3 V2] py3: convert kwargs' keys to str using pycompat.strkwargs
# HG changeset patch # User Pulkit Goyal <7895pul...@gmail.com> # Date 1493837661 -19800 # Thu May 04 00:24:21 2017 +0530 # Node ID 75245b989dd661164041794620770ad7726fe9cd # Parent d186d8361d0526413dd1399e92ccdb40f3ca2056 py3: convert kwargs' keys to str using pycompat.strkwargs diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py --- a/mercurial/cmdutil.py +++ b/mercurial/cmdutil.py @@ -1496,6 +1496,7 @@ props['index'] = next(self._counter) props['revcache'] = {'copies': copies} props['cache'] = self.cache +props = pycompat.strkwargs(props) # write header if self._parts['header']: @@ -1515,7 +1516,7 @@ if self._parts['footer']: if not self.footer: self.footer = templater.stringify( -self.t(self._parts['footer'], **props)) +self.t(self._parts['footer'], **props)) def gettemplate(ui, tmpl, style): """ ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 3 V2] py3: explicitly convert a list to bytes to pass in ui.debug
# HG changeset patch # User Pulkit Goyal <7895pul...@gmail.com> # Date 1493837589 -19800 # Thu May 04 00:23:09 2017 +0530 # Node ID d186d8361d0526413dd1399e92ccdb40f3ca2056 # Parent 1ada3d18e7fbc9069910f2c036992d2f2b28e058 py3: explicitly convert a list to bytes to pass in ui.debug Here pats is a list obviously. Since we can't pass unicodes to ui.debug, we have to pass this as bytes. diff --git a/hgext/largefiles/overrides.py b/hgext/largefiles/overrides.py --- a/hgext/largefiles/overrides.py +++ b/hgext/largefiles/overrides.py @@ -17,6 +17,7 @@ from mercurial import ( archival, cmdutil, +encoding, error, hg, match as matchmod, @@ -380,8 +381,8 @@ r = origmatchfn(f) return r m.matchfn = lfmatchfn - -ui.debug('updated patterns: %s\n' % sorted(pats)) +bytespats = '[' + ', '.join(sorted(pats)) + ']' +ui.debug('updated patterns: %s\n' % bytespats) return m, pats # For hg log --patch, the match object is used in two different senses: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 3] run-tests: support multiple cases in .t test
Excerpts from Augie Fackler's message of 2017-05-15 13:05:33 -0400: > > Nit '==' is Bashism. Use '='. > > should I add a check-code rule for something like r'^ ($|>) .*\[[^]]+=='? > > (regex untested, wrote it out in my email client) That is a good idea. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] cleanup: use set literals
On Mon, May 15, 2017 at 09:33:16AM -0700, Martin von Zweigbergk via Mercurial-devel wrote: > # HG changeset patch > # User Martin von Zweigbergk> # Date 1486774589 28800 > # Fri Feb 10 16:56:29 2017 -0800 > # Node ID 4c861a7d3a2fc1a18bafdcf931a4aade46398c2b > # Parent 539cbe0f8fa38b771b9056d8e7ed51e999073f02 > cleanup: use set literals +1, but I'll leave this for Sean or Greg to take so we can verify that things are working correctly. > > We no longer support Python 2.6, so we can now use set literals. > > diff --git a/contrib/import-checker.py b/contrib/import-checker.py > --- a/contrib/import-checker.py > +++ b/contrib/import-checker.py > @@ -213,7 +213,7 @@ > yield m > for m in ['cffi']: > yield m > -stdlib_prefixes = set([sys.prefix, sys.exec_prefix]) > +stdlib_prefixes = {sys.prefix, sys.exec_prefix} > # We need to supplement the list of prefixes for the search to work > # when run from within a virtualenv. > for mod in (BaseHTTPServer, zlib): > diff --git a/contrib/synthrepo.py b/contrib/synthrepo.py > --- a/contrib/synthrepo.py > +++ b/contrib/synthrepo.py > @@ -71,7 +71,7 @@ > cmdtable = {} > command = cmdutil.command(cmdtable) > > -newfile = set(('new fi', 'rename', 'copy f', 'copy t')) > +newfile = {'new fi', 'rename', 'copy f', 'copy t'} > > def zerodict(): > return collections.defaultdict(lambda: 0) > @@ -336,7 +336,7 @@ > wlock = repo.wlock() > lock = repo.lock() > > -nevertouch = set(('.hgsub', '.hgignore', '.hgtags')) > +nevertouch = {'.hgsub', '.hgignore', '.hgtags'} > > progress = ui.progress > _synthesizing = _('synthesizing') > diff --git a/hgext/convert/filemap.py b/hgext/convert/filemap.py > --- a/hgext/convert/filemap.py > +++ b/hgext/convert/filemap.py > @@ -310,7 +310,7 @@ > # map to any revision in the restricted graph. Put SKIPREV > # in the set of wanted ancestors to simplify code elsewhere > self.parentmap[rev] = SKIPREV > -self.wantedancestors[rev] = set((SKIPREV,)) > +self.wantedancestors[rev] = {SKIPREV} > return > > # Reuse the data from our parent. > diff --git a/hgext/convert/git.py b/hgext/convert/git.py > --- a/hgext/convert/git.py > +++ b/hgext/convert/git.py > @@ -32,7 +32,7 @@ > return "%s %s" % (self.node, self.path) > > # Keys in extra fields that should not be copied if the user requests. > -bannedextrakeys = set([ > +bannedextrakeys = { > # Git commit object built-ins. > 'tree', > 'parent', > @@ -41,7 +41,7 @@ > # Mercurial built-ins. > 'branch', > 'close', > -]) > +} > > class convert_git(common.converter_source, common.commandline): > # Windows does not support GIT_DIR= construct while other systems > @@ -455,9 +455,9 @@ > ('refs/heads/', '') > ] > > -exclude = set([ > +exclude = { > 'refs/remotes/origin/HEAD', > -]) > +} > > try: > output, status = self.gitrunlines('show-ref') > diff --git a/hgext/histedit.py b/hgext/histedit.py > --- a/hgext/histedit.py > +++ b/hgext/histedit.py > @@ -1641,8 +1641,8 @@ > if os.path.exists(os.path.join(repo.path, 'histedit-state')): > state = histeditstate(repo) > state.read() > -histedit_nodes = set([action.node for action > - in state.actions if action.node]) > +histedit_nodes = {action.node for action > + in state.actions if action.node} > common_nodes = histedit_nodes & set(nodelist) > if common_nodes: > raise error.Abort(_("histedit in progress, can't strip %s") > diff --git a/hgext/journal.py b/hgext/journal.py > --- a/hgext/journal.py > +++ b/hgext/journal.py > @@ -158,7 +158,7 @@ > util.safehasattr(repo, 'journal')): > sharedrepo = share._getsrcrepo(repo) > sharedfeatures = _readsharedfeatures(repo) > -if sharedrepo and sharedfeatures > set(['journal']): > +if sharedrepo and sharedfeatures > {'journal'}: > # there is a shared repository and there are shared journal > entries > # to copy. move shared date over from source to destination but > # move the local file first > diff --git a/hgext/largefiles/__init__.py b/hgext/largefiles/__init__.py > --- a/hgext/largefiles/__init__.py > +++ b/hgext/largefiles/__init__.py > @@ -129,7 +129,7 @@ > > def featuresetup(ui, supported): > # don't die on seeing a repo with the largefiles requirement > -supported |= set(['largefiles']) > +supported |= {'largefiles'} > > def uisetup(ui): > localrepo.localrepository.featuresetupfuncs.add(featuresetup) > diff --git a/hgext/rebase.py b/hgext/rebase.py > --- a/hgext/rebase.py > +++ b/hgext/rebase.py > @@ -242,7 +242,7 @@ > skipped = set() > # recompute the set of
Re: [PATCH] repair: compress bundles with gzip instead of bzip2 (BC)
On Sat, May 13, 2017 at 11:33:27PM -0700, Gregory Szorc wrote: > # HG changeset patch > # User Gregory Szorc> # Date 1494743508 25200 > # Sat May 13 23:31:48 2017 -0700 > # Node ID 03c14dca1ff875ed9f4650166ada89baa02c5843 > # Parent 78496ac300255e9996b3e282086661afc08af37c > repair: compress bundles with gzip instead of bzip2 (BC) Can you give me an idea of how much bigger the bundles are on disk? Most users never clean these up. > repair.strip() produces up to 2 bundles: a persisted "backup" > bundle (containing stripped data) and a temporary bundle > (containing data that will be re-applied post strip). The > temporary bundle is uncompressed and the backup bundle is > compressed with bzip2. Should that temporary bundle be hit with zstd if available? It's scoped to a single execution (more or less), so it ought to be safe and a nearly-free IO win for minimal CPU. > > I imagine bzip2 was chosen as the compression format for backup > bundles because it yields the best compression. That makes sense: > you don't want backup bundles wasting space. But at the same time, > you want operations to complete fast as well. > > bzip2 is likely 3-5x slower than zlib/gip. If you are compressing > a large amount of data, that speed difference can translate to > seconds. It can also translate to lost megabytes. However, > storage space gets cheaper every year and single core CPU > performance isn't scaling as well as it was 10 years ago when > this default was likely chosen. I think we should go with the > faster compression algorithm at the expense of larger bundles. > > I *would* like to change the default to zstd (if available). However, > that's a significant BC break and I don't feel comfortable changing > it just yet. When we have official zstd revlog support and the > repo is using it, I think it makes sense to use zstd for backup > bundles. But for zlib-based revlogs, it's safer to avoid zstd > for the time being. > > This change does make some `hg strip` operations faster if large > amounts of data are being compressed. However, execution time is > often dominated by creating and applying the temporary bundle, so > the benefit likely isn't night and day. > > diff --git a/mercurial/repair.py b/mercurial/repair.py > --- a/mercurial/repair.py > +++ b/mercurial/repair.py > @@ -43,9 +43,9 @@ def _bundle(repo, bases, heads, node, su > if cgversion != '01': > bundletype = "HG20" > if compress: > -comp = 'BZ' > +comp = 'GZ' > elif compress: > -bundletype = "HG10BZ" > +bundletype = "HG10GZ" > else: > bundletype = "HG10UN" > return bundle2.writebundle(repo.ui, cg, name, bundletype, vfs, > diff --git a/tests/test-generaldelta.t b/tests/test-generaldelta.t > --- a/tests/test-generaldelta.t > +++ b/tests/test-generaldelta.t > @@ -154,7 +154,7 @@ Test that strip bundle use bundle2 >0 files updated, 0 files merged, 5 files removed, 0 files unresolved >saved backup bundle to > $TESTTMP/aggressive/.hg/strip-backup/1c5d4dc9a8b8-6c68e60c-backup.hg (glob) >$ hg debugbundle .hg/strip-backup/* > - Stream params: sortdict([('Compression', 'BZ')]) > + Stream params: sortdict([('Compression', 'GZ')]) >changegroup -- "sortdict([('version', '02'), ('nbchanges', '1')])" >1c5d4dc9a8b8d6e1750966d343e94db665e7a1e9 > > diff --git a/tests/test-strip.t b/tests/test-strip.t > --- a/tests/test-strip.t > +++ b/tests/test-strip.t > @@ -210,7 +210,7 @@ >summary: b > >$ hg debugbundle .hg/strip-backup/* > - Stream params: sortdict([('Compression', 'BZ')]) > + Stream params: sortdict([('Compression', 'GZ')]) >changegroup -- "sortdict([('version', '02'), ('nbchanges', '1')])" >264128213d290d868c54642d13aeaa3675551a78 >$ hg pull .hg/strip-backup/* > ___ > Mercurial-devel mailing list > Mercurial-devel@mercurial-scm.org > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: Coding style change proposals
On Sat, May 13, 2017 at 04:27:21PM -0700, Phillip Cohen wrote: > +1 to #1, +2 to #2, +0.5 to #3. > > Excited about this progression of the codebase. Thanks for sending this out. > > I'd like to humbly suggest: classes in their own files, at least for > important classes? Or, if not that, perhaps some basic ordering rules > for files, like that classes must come first? Some of our lengthier > files feel like big soups of randomly ordered elements. Eh, I'd rather not impose specifics on the ordering of files unless we can machine check it. (I've also got misgivings about "classes at the top" - my strong belief is that source files should be constructed to ease the burden on maintainers, and introduce methods before they're used, and structure documentation for API consumers...) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 6 of 6 RFC] extensions: show deprecation warning for the use of cmdutil.command
On Sun, May 14, 2017 at 12:11:53PM +0900, Yuya Nishihara wrote: > On Sat, 13 May 2017 11:00:02 -0700, Gregory Szorc wrote: > > On Sat, May 13, 2017 at 2:57 AM, Yuya Nishiharawrote: > > > # HG changeset patch > > > # User Yuya Nishihara > > > # Date 1452349492 -32400 > > > # Sat Jan 09 23:24:52 2016 +0900 > > > # Node ID 1b5d23403828e246d89816c4826a25a7de586cee > > > # Parent 484b2808830a2a390d9e1128c6e88d9249097fa6 > > > extensions: show deprecation warning for the use of cmdutil.command > > > > > > This is RFC. Do we really want to remove cmdutil.command? > > > > > > > I'm in favor of this. But I think it should be deprecated for multiple > > releases to give extensions time to transition. The reason is this is one > > of the oldest and most used APIs in extensions. When we remove it, we'll > > break a lot of extensions. But simply marking it as deprecated has no > > impact unless developer warnings are on. So it shouldn't be too > > controversial to start the process. > > How about setting the target to 4.2 + 2years = 5.0 for now? I don't want > to add version=None support to ui.deprecwarn() API because it would be > so convenient for developers to abuse. 2 years is a stinking long time. Could we go with 4.6 at the latest? (Agree that this is pretty high impact churn that merits some special attention, but I'm skeptical that there will be a difference between a year and two years for the deprecation window.) > ___ > Mercurial-devel mailing list > Mercurial-devel@mercurial-scm.org > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 3] run-tests: support multiple cases in .t test
On Mon, May 15, 2017 at 10:42:02PM +0900, Yuya Nishihara wrote: > On Tue, 2 May 2017 19:01:26 -0700, Jun Wu wrote: > > + > #case C > > + > $ [ $V == C ] > > + > #case A > > + > $ [ $V == C ] > > + > [1] > > + > #case A B > > + > $ [ $V == C ] > > + > [1] > > + > #endcase > > + > $ [ $V == D ] > > Nit '==' is Bashism. Use '='. should I add a check-code rule for something like r'^ ($|>) .*\[[^]]+=='? (regex untested, wrote it out in my email client) > ___ > Mercurial-devel mailing list > Mercurial-devel@mercurial-scm.org > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] cleanup: use set literals
# HG changeset patch # User Martin von Zweigbergk# Date 1486774589 28800 # Fri Feb 10 16:56:29 2017 -0800 # Node ID 4c861a7d3a2fc1a18bafdcf931a4aade46398c2b # Parent 539cbe0f8fa38b771b9056d8e7ed51e999073f02 cleanup: use set literals We no longer support Python 2.6, so we can now use set literals. diff --git a/contrib/import-checker.py b/contrib/import-checker.py --- a/contrib/import-checker.py +++ b/contrib/import-checker.py @@ -213,7 +213,7 @@ yield m for m in ['cffi']: yield m -stdlib_prefixes = set([sys.prefix, sys.exec_prefix]) +stdlib_prefixes = {sys.prefix, sys.exec_prefix} # We need to supplement the list of prefixes for the search to work # when run from within a virtualenv. for mod in (BaseHTTPServer, zlib): diff --git a/contrib/synthrepo.py b/contrib/synthrepo.py --- a/contrib/synthrepo.py +++ b/contrib/synthrepo.py @@ -71,7 +71,7 @@ cmdtable = {} command = cmdutil.command(cmdtable) -newfile = set(('new fi', 'rename', 'copy f', 'copy t')) +newfile = {'new fi', 'rename', 'copy f', 'copy t'} def zerodict(): return collections.defaultdict(lambda: 0) @@ -336,7 +336,7 @@ wlock = repo.wlock() lock = repo.lock() -nevertouch = set(('.hgsub', '.hgignore', '.hgtags')) +nevertouch = {'.hgsub', '.hgignore', '.hgtags'} progress = ui.progress _synthesizing = _('synthesizing') diff --git a/hgext/convert/filemap.py b/hgext/convert/filemap.py --- a/hgext/convert/filemap.py +++ b/hgext/convert/filemap.py @@ -310,7 +310,7 @@ # map to any revision in the restricted graph. Put SKIPREV # in the set of wanted ancestors to simplify code elsewhere self.parentmap[rev] = SKIPREV -self.wantedancestors[rev] = set((SKIPREV,)) +self.wantedancestors[rev] = {SKIPREV} return # Reuse the data from our parent. diff --git a/hgext/convert/git.py b/hgext/convert/git.py --- a/hgext/convert/git.py +++ b/hgext/convert/git.py @@ -32,7 +32,7 @@ return "%s %s" % (self.node, self.path) # Keys in extra fields that should not be copied if the user requests. -bannedextrakeys = set([ +bannedextrakeys = { # Git commit object built-ins. 'tree', 'parent', @@ -41,7 +41,7 @@ # Mercurial built-ins. 'branch', 'close', -]) +} class convert_git(common.converter_source, common.commandline): # Windows does not support GIT_DIR= construct while other systems @@ -455,9 +455,9 @@ ('refs/heads/', '') ] -exclude = set([ +exclude = { 'refs/remotes/origin/HEAD', -]) +} try: output, status = self.gitrunlines('show-ref') diff --git a/hgext/histedit.py b/hgext/histedit.py --- a/hgext/histedit.py +++ b/hgext/histedit.py @@ -1641,8 +1641,8 @@ if os.path.exists(os.path.join(repo.path, 'histedit-state')): state = histeditstate(repo) state.read() -histedit_nodes = set([action.node for action - in state.actions if action.node]) +histedit_nodes = {action.node for action + in state.actions if action.node} common_nodes = histedit_nodes & set(nodelist) if common_nodes: raise error.Abort(_("histedit in progress, can't strip %s") diff --git a/hgext/journal.py b/hgext/journal.py --- a/hgext/journal.py +++ b/hgext/journal.py @@ -158,7 +158,7 @@ util.safehasattr(repo, 'journal')): sharedrepo = share._getsrcrepo(repo) sharedfeatures = _readsharedfeatures(repo) -if sharedrepo and sharedfeatures > set(['journal']): +if sharedrepo and sharedfeatures > {'journal'}: # there is a shared repository and there are shared journal entries # to copy. move shared date over from source to destination but # move the local file first diff --git a/hgext/largefiles/__init__.py b/hgext/largefiles/__init__.py --- a/hgext/largefiles/__init__.py +++ b/hgext/largefiles/__init__.py @@ -129,7 +129,7 @@ def featuresetup(ui, supported): # don't die on seeing a repo with the largefiles requirement -supported |= set(['largefiles']) +supported |= {'largefiles'} def uisetup(ui): localrepo.localrepository.featuresetupfuncs.add(featuresetup) diff --git a/hgext/rebase.py b/hgext/rebase.py --- a/hgext/rebase.py +++ b/hgext/rebase.py @@ -242,7 +242,7 @@ skipped = set() # recompute the set of skipped revs if not collapse: -seen = set([dest]) +seen = {dest} for old, new in sorted(state.items()): if new != revtodo and new in seen: skipped.add(old) @@ -250,7 +250,7 @@ repo.ui.debug('computed skipped revs: %s\n' % (' '.join(str(r) for r in sorted(skipped)) or None)) repo.ui.debug('rebase status
Re: [PATCH V2] changegroup: add bundlecaps back
On Mon, May 15, 2017 at 9:41 AM, Durham Goodewrote: > # HG changeset patch > # User Durham Goode > # Date 1494866127 25200 > # Mon May 15 09:35:27 2017 -0700 > # Node ID ab69af9456e02e3979b88597041ee3f1277925ea > # Parent d0d9a4fca59bcb0c0d5a53e7b3d2db2068844d1f > changegroup: add bundlecaps back Queued, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 3] killdaemons: fix error handling on Windows
On Mon, 15 May 2017 00:08:30 -0400, Matt Harbison wrote: > # HG changeset patch > # User Matt Harbison> # Date 1494779534 14400 > # Sun May 14 12:32:14 2017 -0400 > # Node ID 024271e90987e5794dab6eb00844467065fae7e4 > # Parent db0b470547d5e21f042390a204f09ae7d7303757 > killdaemons: fix error handling on Windows > > After taking Adrian's suggestion[1] to use util.posixfile to avoid os.unlink() > errors, the error changed from: > > Errored test-hgweb-json.t: Traceback (most recent call last): > File "./run-tests.py", line 724, in run > self.tearDown() > File "./run-tests.py", line 805, in tearDown > killdaemons(entry) > File "./run-tests.py", line 540, in killdaemons > logfn=vlog) > File "...\tests\killdaemons.py", line 94, in killdaemons > os.unlink(pidfile) > WindowsError: [Error 32] The process cannot access the file because it is > being used by another process: > '...\\hgtests.zmpqj3\\child80\\daemon.pids' > > to: > > Errored test-largefiles.t: Traceback (most recent call last): > File "./run-tests.py", line 724, in run > self.tearDown() > File "./run-tests.py", line 805, in tearDown > killdaemons(entry) > File "./run-tests.py", line 540, in killdaemons > logfn=vlog) > File "...\tests\killdaemons.py", line 91, in killdaemons > kill(pid, logfn, tryhard) > File "...\tests\killdaemons.py", line 55, in kill > _check(ctypes.windll.kernel32.CloseHandle(handle)) > File "...\tests\killdaemons.py", line 18, in _check > raise ctypes.WinError(winerrno) > WindowsError: [Error 6] The handle is invalid. > > The handle in question is for the process, not the file. That made me wonder > why WaitForSingleObject() didn't raise an error, and it's because it isn't a > BOOL return. I think it's better to define _check() function per return-value type. Win32 API doesn't state that the TRUE value is 1, even though it's practically 1. > (For this method, a return of 0 is WAIT_OBJECT_0, and an error is > signalled by returning (DWORD) 0x.) When I printed out that value (as > returned by the function) with '0x%x', it printed '0x-1'. That's probably because we don't teach ctypes the function signature. A return value is taken as int by default. > -if handle == 0: > +if handle is None: > _check(0, 87) # err 87 when process not found This could be _checkptr(handle, 87) for example. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 3] killdaemons: use posixfile to avoid intermittent unlink errors on Windows
On Mon, 15 May 2017 00:08:31 -0400, Matt Harbison wrote: > # HG changeset patch > # User Matt Harbison> # Date 1494808413 14400 > # Sun May 14 20:33:33 2017 -0400 > # Node ID 809709930080937b6b56e5cad285798f29a10280 > # Parent 024271e90987e5794dab6eb00844467065fae7e4 > killdaemons: use posixfile to avoid intermittent unlink errors on Windows > > This is the aforementioned fix for the occasional cleanup error with #serve > enabled. There are a handful of tests that neglect to kill the daemons they > spawned, and this code is doing a last ditch reap of them. The test that got > flagged was non-deterministic, and I've seen up to 3 fail in the same run. > > The problem with trying to import the mercurial module is that while it is > available for running the individual *.t files, it is not in sys.path for > run-tests.py itself. I couldn't think of any other way to make this work, and > not affect sys.path for the indiviual tests. (The main source tree _is_ in > PYTHONPATH when this is imported from run-tests.py.) > > diff --git a/tests/killdaemons.py b/tests/killdaemons.py > --- a/tests/killdaemons.py > +++ b/tests/killdaemons.py > @@ -7,6 +7,16 @@ > import sys > import time > > +# PYTHONPATH contains the hg source tree when invoked from ./run-tests, but > +# sys.path does not, and importing mercurial fails. The first import works > from > +# the .t files without editing the path. > +try: > +from mercurial.util import posixfile > +except ImportError: > +srctree = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) > +sys.path.insert(1, srctree) > +from mercurial.util import posixfile sys.path is global. I slightly prefer moving this hack to run-tests.py if it's okay for run-tests.py to depend on Mercurial modules. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 3] flagprocessor: add a fast path when flags is 0
On Wed, 10 May 2017 18:47:35 -0700, Jun Wu wrote: > # HG changeset patch > # User Jun Wu> # Date 1494458278 25200 > # Wed May 10 16:17:58 2017 -0700 > # Node ID ffac07761a9b7e5277948c5a6572de125b23d296 > # Parent a5a22d616f981e8c220f1c2fd4eae098e104a11c > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > ffac07761a9b > flagprocessor: add a fast path when flags is 0 Queued this, thanks. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 3] run-tests: support multiple cases in .t test
On Tue, 2 May 2017 19:01:26 -0700, Jun Wu wrote: > # HG changeset patch > # User Jun Wu> # Date 1493488388 25200 > # Sat Apr 29 10:53:08 2017 -0700 > # Node ID 6663e696ae0c2a1b8f6fd735dfcf9b99fb4dac73 > # Parent f32a5c7a590fd5d8a9d6c8195b6171331ec9f3a8 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > 6663e696ae0c > run-tests: support multiple cases in .t test The idea sounds nice, and the implementation generally looks good to me. Can you send V2? > For example, if a test should behave the same with or without an > experimental flag, we can add the following to the .t header: > > #case default > > #case experimental-a > $ cat >> $HGRCPATH << EOF > > [experimental] > > feature=a > > EOF > #endcase So this is a sort of parameterized tests? I thought "#case" would be just a syntactic sugar of "#if"s. Another idea is to define a set of "cases" beforehand, and use the existing "#if" syntax, but I'm not sure which is better. #variants default experimental-a #if default ... #endif > +tests += [{'path': t, 'case': c} for c in sorted(cases)] > +else: > +tests.append({'path': t}) Instead of extending the test key to a dict, maybe TestRunner._gettest() could return a list of all cases? If that works, the key handling will get slightly simpler. > + > #case C > + > $ [ $V == C ] > + > #case A > + > $ [ $V == C ] > + > [1] > + > #case A B > + > $ [ $V == C ] > + > [1] > + > #endcase > + > $ [ $V == D ] Nit '==' is Bashism. Use '='. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 2 shelve-ext v7] shelve: refactor shelvestate loading
On Sun, 14 May 2017 14:30:52 -0700, Kostia Balytskyi wrote: > # HG changeset patch > # User Kostia Balytskyi> # Date 1494796507 25200 > # Sun May 14 14:15:07 2017 -0700 > # Node ID 76478fd50b5a116a49fbf7c6ef8e0b0be0c4673f > # Parent 78496ac300255e9996b3e282086661afc08af37c > shelve: refactor shelvestate loading Queued these, thanks! ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel