D5412: test: fix test-http-bad-server with current python 2.7
jcristau created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY https://github.com/python/cpython/pull/2825 changed the exception message for empty http status line. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5412 AFFECTED FILES tests/test-http-bad-server.t CHANGE DETAILS diff --git a/tests/test-http-bad-server.t b/tests/test-http-bad-server.t --- a/tests/test-http-bad-server.t +++ b/tests/test-http-bad-server.t @@ -69,7 +69,7 @@ $ cat hg.pid > $DAEMON_PIDS $ hg clone http://localhost:$HGPORT/ clone - abort: error: bad HTTP status line: '' + abort: error: bad HTTP status line: * (glob) [255] $ killdaemons.py $DAEMON_PIDS @@ -85,7 +85,7 @@ $ hg serve --config badserver.closeafterrecvbytes=40 -p $HGPORT -d --pid-file=hg.pid -E error.log $ cat hg.pid > $DAEMON_PIDS $ hg clone http://localhost:$HGPORT/ clone - abort: error: bad HTTP status line: '' + abort: error: bad HTTP status line: * (glob) [255] $ killdaemons.py $DAEMON_PIDS @@ -102,7 +102,7 @@ $ hg serve --config badserver.closeafterrecvbytes=210,223 -p $HGPORT -d --pid-file=hg.pid -E error.log $ cat hg.pid > $DAEMON_PIDS $ hg clone http://localhost:$HGPORT/ clone - abort: error: bad HTTP status line: '' + abort: error: bad HTTP status line: * (glob) [255] $ killdaemons.py $DAEMON_PIDS @@ -141,7 +141,7 @@ $ cat hg.pid > $DAEMON_PIDS $ hg clone http://localhost:$HGPORT/ clone requesting all changes - abort: error: bad HTTP status line: '' + abort: error: bad HTTP status line: * (glob) [255] $ killdaemons.py $DAEMON_PIDS @@ -200,7 +200,7 @@ $ cat hg.pid > $DAEMON_PIDS $ hg clone http://localhost:$HGPORT/ clone - abort: error: bad HTTP status line: '' + abort: error: bad HTTP status line: * (glob) [255] $ killdaemons.py $DAEMON_PIDS To: jcristau, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 5 of 5] py3: quote several instances of $PYTHON for Windows
On Mon, 10 Dec 2018 06:49:11 -0500, Yuya Nishihara wrote: On Sun, 09 Dec 2018 22:44:37 -0500, Matt Harbison wrote: # HG changeset patch # User Matt Harbison # Date 1544408614 18000 # Sun Dec 09 21:23:34 2018 -0500 # Node ID 4b007df3bff23f32439e299d13ce1a808b857c4d # Parent e5b7d60068537baa1ffeeca4e1a81f7498d0d48e py3: quote several instances of $PYTHON for Windows Queued, thanks. Any idea how to handle test-hghave.t? Here are 3 failed attempts: $ hg diff diff -r 008f3491dc53 tests/test-hghave.t --- a/tests/test-hghave.t Mon Dec 10 20:01:07 2018 + +++ b/tests/test-hghave.t Tue Dec 11 23:12:11 2018 -0500 @@ -20,12 +20,16 @@ > $ echo foo > foo > EOF - $ ( \ - > testrepohgenv; \ - > "$PYTHON" $TESTDIR/run-tests.py -j 1 \ - >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t \ - > ) - running 1 tests using 1 parallel processes + + $ cat > addontest.sh < #!/bin/sh + > + > . "$TESTDIR/helpers-testrepo.sh" + > testrepohgenv; + > '"$PYTHON"' "$TESTDIR/run-tests.py" -j 1 \ + >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t + > EOF + $ sh addontest.sh . # Ran 1 tests, 0 skipped, 0 failed. $ py -3 run-tests.py --local test-hghave.t running 1 tests using 1 parallel processes --- c:/Users/Matt/hg/tests/test-hghave.t +++ c:/Users/Matt/hg/tests/test-hghave.t.err @@ -30,8 +30,8 @@ >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t > EOF $ sh addontest.sh - . - # Ran 1 tests, 0 skipped, 0 failed. + addontest.sh: line 5: "C:/Program Files/Python37/python.exe": $ENOENT$ + [127] (invocation via command line) ERROR: test-hghave.t output changed ! Failed test-hghave.t: output changed # Ran 1 tests, 0 skipped, 1 failed. python hash seed: 3453714405 == $ hg diff diff -r 008f3491dc53 tests/test-hghave.t --- a/tests/test-hghave.t Mon Dec 10 20:01:07 2018 + +++ b/tests/test-hghave.t Tue Dec 11 23:13:22 2018 -0500 @@ -20,12 +20,16 @@ > $ echo foo > foo > EOF - $ ( \ - > testrepohgenv; \ - > "$PYTHON" $TESTDIR/run-tests.py -j 1 \ - >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t \ - > ) - running 1 tests using 1 parallel processes + + $ cat > addontest.sh < #!/bin/sh + > + > . "$TESTDIR/helpers-testrepo.sh" + > testrepohgenv; + > "$PYTHON" "$TESTDIR/run-tests.py" -j 1 \ + >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t + > EOF + $ sh addontest.sh . # Ran 1 tests, 0 skipped, 0 failed. $ py -3 run-tests.py --local test-hghave.t running 1 tests using 1 parallel processes --- c:/Users/Matt/hg/tests/test-hghave.t +++ c:/Users/Matt/hg/tests/test-hghave.t.err @@ -30,8 +30,9 @@ >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t > EOF $ sh addontest.sh - . - # Ran 1 tests, 0 skipped, 0 failed. + 'c:\\Program' is not recognized as an internal or external command,\r (esc) + operable program or batch file.\r (esc) + [1] (invocation via command line) ERROR: test-hghave.t output changed ! Failed test-hghave.t: output changed # Ran 1 tests, 0 skipped, 1 failed. python hash seed: 300496195 === $ hg diff diff -r 008f3491dc53 tests/test-hghave.t --- a/tests/test-hghave.t Mon Dec 10 20:01:07 2018 + +++ b/tests/test-hghave.t Tue Dec 11 23:14:19 2018 -0500 @@ -20,12 +20,16 @@ > $ echo foo > foo > EOF - $ ( \ - > testrepohgenv; \ - > "$PYTHON" $TESTDIR/run-tests.py -j 1 \ - >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t \ - > ) - running 1 tests using 1 parallel processes + + $ cat > addontest.sh < #!/bin/sh + > + > . "$TESTDIR/helpers-testrepo.sh" + > testrepohgenv; + > \"$PYTHON\" "$TESTDIR/run-tests.py" -j 1 \ + >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t + > EOF + $ sh addontest.sh . # Ran 1 tests, 0 skipped, 0 failed. $ py -3 run-tests.py --local test-hghave.t running 1 tests using 1 parallel processes --- c:/Users/Matt/hg/tests/test-hghave.t +++ c:/Users/Matt/hg/tests/test-hghave.t.err @@ -30,8 +30,8 @@ >$HGTEST_RUN_TESTS_PURE test-hghaveaddon.t > EOF $ sh addontest.sh - . - # Ran 1 tests, 0 skipped, 0 failed. + addontest.sh: line 5: "C:/Program: $ENOENT$ + [127] (invocation via command line) ERROR: test-hghave.t output changed ! Failed test-hghave.t: output changed # Ran 1 tests, 0 skipped, 1 failed. python hash seed: 2437087002 ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] win32: close the handles associated with a spawned child process
# HG changeset patch # User Matt Harbison # Date 1544583474 18000 # Tue Dec 11 21:57:54 2018 -0500 # Node ID ec40f2ca8ac974ef657d4d52ac78019e75444911 # Parent 76d8b20139a3b8b5835c7262216b97275845b582 win32: close the handles associated with a spawned child process Probably not a big deal because at this point, the call is only used when spawning a daemonized server. In that case, the parent process goes away first, so it won't prevent the child from being cleaned up. diff --git a/mercurial/win32.py b/mercurial/win32.py --- a/mercurial/win32.py +++ b/mercurial/win32.py @@ -588,6 +588,9 @@ def spawndetached(args): if not res: raise ctypes.WinError() +_kernel32.CloseHandle(pi.hProcess) +_kernel32.CloseHandle(pi.hThread) + return pi.dwProcessId def unlink(f): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5411: sqlitestore: create new connections on new PIDs
indygreg abandoned this revision. indygreg added a comment. It turns out we were hitting a bug in an ancient version of SQLite. Upgrading from 3.11 to 3.22 fixed it. I don't think this patch is needed. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5411 To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5411: sqlitestore: create new connections on new PIDs
indygreg planned changes to this revision. indygreg added a comment. Hold off on reviewing this. I'm still debugging some issues at Mozilla and want to be sure this patch is correct... REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5411 To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5411: sqlitestore: create new connections on new PIDs
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
If the Mercurial process fork()s, the Python thread ID remains
unchanged. The previous code for returning a SQLite connection would
recycle the existing connection among all children.
Mercurial can fork() when performing working directory updates.
For reasons I don't fully understand, the recycling of even a
read-only SQLite connection was resulting in Python raising a
"DatabaseError: database disk image is malformed" exception. This
message comes from the bowels of SQLite. I suspect there is some
internal client state in the SQLite database somewhere and having
multiple clients race to update it results in badness. Who knows.
This commit teaches the "get a SQLite connection" logic to also
verify the PID matches before returning an existing connection.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5411
AFFECTED FILES
hgext/sqlitestore.py
CHANGE DETAILS
diff --git a/hgext/sqlitestore.py b/hgext/sqlitestore.py
--- a/hgext/sqlitestore.py
+++ b/hgext/sqlitestore.py
@@ -76,6 +76,7 @@
)
from mercurial.utils import (
interfaceutil,
+procutil,
storageutil,
)
@@ -1005,17 +1006,18 @@
@property
def _dbconn(self):
-# SQLite connections can only be used on the thread that created
-# them. In most cases, this "just works." However, hgweb uses
-# multiple threads.
-tid = threading.current_thread().ident
+# SQLite connections can only be used on the OS and Python thread that
+# created them. In most cases, this "just works." However, hgweb uses
+# multiple Python threads. And Mercurial may fork. So we need to check
+# global state before returning an existing connection.
+key = (procutil.getpid(), threading.current_thread().ident)
if self._db:
-if self._db[0] == tid:
+if self._db[0] == key:
return self._db[1]
db = makedb(self.svfs.join('db.sqlite'))
-self._db = (tid, db)
+self._db = (key, db)
return db
To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2 evolve-ext] resolves 6028: return return (False, ".") instead of return (False, '')
# HG changeset patch
# User James Reynolds
# Date 1543952045 18000
# Tue Dec 04 14:34:05 2018 -0500
# Branch stable
# Node ID 9bf7096a013cbbef97064b47fef069e8ac87356c
# Parent 710f32053ba340564ca9909b4dc7bef1fc1946fb
# EXP-Topic issue6028
resolves 6028: return return (False, ".") instead of return (False, '')
- repo[] no longer takes an empty string
diff -r 710f32053ba3 -r 9bf7096a013c hgext3rd/evolve/evolvecmd.py
--- a/hgext3rd/evolve/evolvecmd.py Tue Dec 04 14:56:19 2018 -0500
+++ b/hgext3rd/evolve/evolvecmd.py Tue Dec 04 14:34:05 2018 -0500
@@ -61,7 +61,7 @@
bool: a boolean value indicating whether the instability was solved
newnode: if bool is True, then the newnode of the resultant commit
formed. newnode can be node, when resolution led to no new
- commit. If bool is False, this is ''.
+ commit. If bool is False, this is ".".
"""
displayer = None
if stacktmplt:
@@ -101,7 +101,7 @@
bool: a boolean value indicating whether the instability was solved
newnode: if bool is True, then the newnode of the resultant commit
formed. newnode can be node, when resolution led to no new
- commit. If bool is False, this is ''.
+ commit. If bool is False, this is ".".
"""
pctx = orig.p1()
keepbranch = orig.p1().branch() != orig.branch()
@@ -125,7 +125,7 @@
if not pctx.obsolete():
ui.warn(_("cannot solve instability of %s, skipping\n") % orig)
-return (False, '')
+return (False, ".")
obs = pctx
newer = obsutil.successorssets(repo, obs.node())
# search of a parent which is not killed
@@ -139,7 +139,7 @@
msg = _("skipping %s: divergent rewriting. can't choose "
"destination\n") % obs
ui.write_err(msg)
-return (False, '')
+return (False, ".")
targets = newer[0]
assert targets
if len(targets) > 1:
@@ -157,7 +157,7 @@
"ambiguous destination: "
"parent split across two branches\n")
ui.write_err(msg)
-return (False, '')
+return (False, ".")
target = repo[selectedrev]
else:
target = repo[heads.first()]
@@ -177,7 +177,7 @@
todo = 'hg rebase -r %s -d %s\n' % (orig, target)
if dryrun:
repo.ui.write(todo)
-return (False, '')
+return (False, ".")
else:
repo.ui.note(todo)
if progresscb:
@@ -201,7 +201,7 @@
bool: a boolean value indicating whether the instability was solved
newnode: if bool is True, then the newnode of the resultant commit
formed. newnode can be node, when resolution led to no new
- commit. If bool is False, this is ''.
+ commit. If bool is False, this is ".".
"""
repo = repo.unfiltered()
bumped = repo[bumped.rev()]
@@ -209,14 +209,14 @@
if len(bumped.parents()) > 1:
msg = _('skipping %s : we do not handle merge yet\n') % bumped
ui.write_err(msg)
-return (False, '')
+return (False, ".")
prec = repo.set('last(allprecursors(%d) and public())',
bumped.rev()).next()
# For now we deny target merge
if len(prec.parents()) > 1:
msg = _('skipping: %s: public version is a merge, '
'this is not handled yet\n') % prec
ui.write_err(msg)
-return (False, '')
+return (False, ".")
if not ui.quiet or confirm:
repo.ui.write(_('recreate:'), label='evolve.operation')
@@ -232,7 +232,7 @@
repo.ui.write(('hg revert --all --rev %s;\n' % bumped))
repo.ui.write(('hg commit --msg "%s update to %s"\n' %
(TROUBLES['PHASEDIVERGENT'], bumped)))
-return (False, '')
+return (False, ".")
if progresscb:
progresscb()
tmpctx = bumped
@@ -343,7 +343,7 @@
bool: a boolean value indicating whether the instability was solved
newnode: if bool is True, then the newnode of the resultant commit
formed. newnode can be node, when resolution led to no new
- commit. If bool is False, this is ''.
+ commit. If bool is False, this is ".".
"""
repo = repo.unfiltered()
divergent = repo[divergent.rev()]
@@ -376,7 +376,7 @@
"| You should contact your local evolution Guru for help.\n"
) % (divergent, TROUBLES['CONTENTDIVERGENT'], othersstr)
ui.write_err(msg)
-return (False, '')
+return (False, ".")
other = others[0]
evolvestate['other-divergent'] = other.node()
evolvestate['base'] = base.node()
@@ -390,7 +390,7 @@
"| This probably means redoing the merge and using \n"
"| `hg prune` to kill older version.\n")
[PATCH 1 of 2 evolve-ext] issue-6028: failing test
# HG changeset patch # User James Reynolds # Date 1543953379 18000 # Tue Dec 04 14:56:19 2018 -0500 # Branch stable # Node ID 710f32053ba340564ca9909b4dc7bef1fc1946fb # Parent 48da02d7f8ebd7d274ace7bb6c74a856c4a62882 # EXP-Topic issue6028 issue-6028: failing test diff -r 48da02d7f8eb -r 710f32053ba3 tests/test-issue-6028.t --- /dev/null Thu Jan 01 00:00:00 1970 + +++ b/tests/test-issue-6028.t Tue Dec 04 14:56:19 2018 -0500 @@ -0,0 +1,123 @@ +This test file test the #6028 issue + +evolve fails with mercurial.error.ProgrammingError: unsupported changeid '' of type + +https://bz.mercurial-scm.org/show_bug.cgi?id=6028 + +Global setup + + + $ . $TESTDIR/testlib/common.sh + $ cat >> $HGRCPATH < [ui] + > interactive = true + > [phases] + > publish=False + > [extensions] + > evolve = + > topic = + > EOF + +Test + + + $ hg init $TESTTMP/issue-6028 + $ cd $TESTTMP/issue-6028 + +create initial commit + $ echo "0" > 0 + $ hg ci -Am 0 + adding 0 + + + $ hg up default + 0 files updated, 0 files merged, 0 files removed, 0 files unresolved + $ hg topics a + marked working directory as topic: a + $ echo "a" > a + $ hg ci -Am a + adding a + active topic 'a' grew its first changeset + (see 'hg help topics' for more information) + + + $ hg up default + 0 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ hg topics b + marked working directory as topic: b + $ echo "b" > b + $ hg ci -Am b + adding b + active topic 'b' grew its first changeset + (see 'hg help topics' for more information) + + $ hg up default + 0 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ hg branch integration + marked working directory as branch integration + + $ hg merge a + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged a" + + $ hg merge b + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged b" + + $ hg up a + switching to topic a + 0 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ echo "a bad commit" >> a_bad_commit + $ hg add a_bad_commit + $ hg ci -m "a bad commit" + $ hg up integration + 1 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ hg merge a + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged a bad commit" + + $ hg up a + switching to topic a + 0 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ echo "aa" >> a + $ hg ci -m "aa" + $ hg up integration + 2 files updated, 0 files merged, 0 files removed, 0 files unresolved + $ hg merge a + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged aa" + + $ hg up b + switching to topic b + 0 files updated, 0 files merged, 2 files removed, 0 files unresolved + $ echo "bb" >> b + $ hg ci -m "bb" + $ hg up integration + 3 files updated, 0 files merged, 0 files removed, 0 files unresolved + $ hg merge b + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged bb" + +create instability by pruning two changesets, one in a topic, one in a merge + $ hg prune -r 5:6 + 2 changesets pruned + 3 new orphan changesets + + $ hg up 4 + 2 files updated, 0 files merged, 1 files removed, 0 files unresolved + +start the evolve + $ hg evolve + move:[8] merged aa + atop:[4] merged b + working directory is now at c920dd828523 + +evolve creates an obsolete changeset above as 11 + $ hg evolve -r . + cannot solve instability of c920dd828523, skipping + cannot solve instability of c920dd828523, skipping ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2 evolve-ext] resolves 6028: return return (False, ".") instead of return (False, '')
# HG changeset patch
# User James Reynolds
# Date 1543952045 18000
# Tue Dec 04 14:34:05 2018 -0500
# Branch stable
# Node ID 4bbca2202a72adf5877e5d5a3d8850424a01cb6f
# Parent 710f32053ba340564ca9909b4dc7bef1fc1946fb
# EXP-Topic issue6028
resolves 6028: return return (False, ".") instead of return (False, '')
- repo[] no longer takes an empty string
diff -r 710f32053ba3 -r 4bbca2202a72 hgext3rd/evolve/evolvecmd.py
--- a/hgext3rd/evolve/evolvecmd.py Tue Dec 04 14:56:19 2018 -0500
+++ b/hgext3rd/evolve/evolvecmd.py Tue Dec 04 14:34:05 2018 -0500
@@ -61,7 +61,7 @@
bool: a boolean value indicating whether the instability was solved
newnode: if bool is True, then the newnode of the resultant commit
formed. newnode can be node, when resolution led to no new
- commit. If bool is False, this is ''.
+ commit. If bool is False, this is ".".
"""
displayer = None
if stacktmplt:
@@ -101,7 +101,7 @@
bool: a boolean value indicating whether the instability was solved
newnode: if bool is True, then the newnode of the resultant commit
formed. newnode can be node, when resolution led to no new
- commit. If bool is False, this is ''.
+ commit. If bool is False, this is ".".
"""
pctx = orig.p1()
keepbranch = orig.p1().branch() != orig.branch()
@@ -125,7 +125,7 @@
if not pctx.obsolete():
ui.warn(_("cannot solve instability of %s, skipping\n") % orig)
-return (False, '')
+return return (False, ".")
obs = pctx
newer = obsutil.successorssets(repo, obs.node())
# search of a parent which is not killed
@@ -139,7 +139,7 @@
msg = _("skipping %s: divergent rewriting. can't choose "
"destination\n") % obs
ui.write_err(msg)
-return (False, '')
+return return (False, ".")
targets = newer[0]
assert targets
if len(targets) > 1:
@@ -157,7 +157,7 @@
"ambiguous destination: "
"parent split across two branches\n")
ui.write_err(msg)
-return (False, '')
+return return (False, ".")
target = repo[selectedrev]
else:
target = repo[heads.first()]
@@ -177,7 +177,7 @@
todo = 'hg rebase -r %s -d %s\n' % (orig, target)
if dryrun:
repo.ui.write(todo)
-return (False, '')
+return return (False, ".")
else:
repo.ui.note(todo)
if progresscb:
@@ -201,7 +201,7 @@
bool: a boolean value indicating whether the instability was solved
newnode: if bool is True, then the newnode of the resultant commit
formed. newnode can be node, when resolution led to no new
- commit. If bool is False, this is ''.
+ commit. If bool is False, this is ".".
"""
repo = repo.unfiltered()
bumped = repo[bumped.rev()]
@@ -209,14 +209,14 @@
if len(bumped.parents()) > 1:
msg = _('skipping %s : we do not handle merge yet\n') % bumped
ui.write_err(msg)
-return (False, '')
+return return (False, ".")
prec = repo.set('last(allprecursors(%d) and public())',
bumped.rev()).next()
# For now we deny target merge
if len(prec.parents()) > 1:
msg = _('skipping: %s: public version is a merge, '
'this is not handled yet\n') % prec
ui.write_err(msg)
-return (False, '')
+return return (False, ".")
if not ui.quiet or confirm:
repo.ui.write(_('recreate:'), label='evolve.operation')
@@ -232,7 +232,7 @@
repo.ui.write(('hg revert --all --rev %s;\n' % bumped))
repo.ui.write(('hg commit --msg "%s update to %s"\n' %
(TROUBLES['PHASEDIVERGENT'], bumped)))
-return (False, '')
+return return (False, ".")
if progresscb:
progresscb()
tmpctx = bumped
@@ -343,7 +343,7 @@
bool: a boolean value indicating whether the instability was solved
newnode: if bool is True, then the newnode of the resultant commit
formed. newnode can be node, when resolution led to no new
- commit. If bool is False, this is ''.
+ commit. If bool is False, this is ".".
"""
repo = repo.unfiltered()
divergent = repo[divergent.rev()]
@@ -376,7 +376,7 @@
"| You should contact your local evolution Guru for help.\n"
) % (divergent, TROUBLES['CONTENTDIVERGENT'], othersstr)
ui.write_err(msg)
-return (False, '')
+return return (False, ".")
other = others[0]
evolvestate['other-divergent'] = other.node()
evolvestate['base'] = base.node()
@@ -390,7 +390,7 @@
"| This probably means redoing the merge and using \n"
"|
[PATCH 1 of 2 evolve-ext] issue-6028: failing test
# HG changeset patch # User James Reynolds # Date 1543953379 18000 # Tue Dec 04 14:56:19 2018 -0500 # Branch stable # Node ID 710f32053ba340564ca9909b4dc7bef1fc1946fb # Parent 48da02d7f8ebd7d274ace7bb6c74a856c4a62882 # EXP-Topic issue6028 issue-6028: failing test diff -r 48da02d7f8eb -r 710f32053ba3 tests/test-issue-6028.t --- /dev/null Thu Jan 01 00:00:00 1970 + +++ b/tests/test-issue-6028.t Tue Dec 04 14:56:19 2018 -0500 @@ -0,0 +1,123 @@ +This test file test the #6028 issue + +evolve fails with mercurial.error.ProgrammingError: unsupported changeid '' of type + +https://bz.mercurial-scm.org/show_bug.cgi?id=6028 + +Global setup + + + $ . $TESTDIR/testlib/common.sh + $ cat >> $HGRCPATH < [ui] + > interactive = true + > [phases] + > publish=False + > [extensions] + > evolve = + > topic = + > EOF + +Test + + + $ hg init $TESTTMP/issue-6028 + $ cd $TESTTMP/issue-6028 + +create initial commit + $ echo "0" > 0 + $ hg ci -Am 0 + adding 0 + + + $ hg up default + 0 files updated, 0 files merged, 0 files removed, 0 files unresolved + $ hg topics a + marked working directory as topic: a + $ echo "a" > a + $ hg ci -Am a + adding a + active topic 'a' grew its first changeset + (see 'hg help topics' for more information) + + + $ hg up default + 0 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ hg topics b + marked working directory as topic: b + $ echo "b" > b + $ hg ci -Am b + adding b + active topic 'b' grew its first changeset + (see 'hg help topics' for more information) + + $ hg up default + 0 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ hg branch integration + marked working directory as branch integration + + $ hg merge a + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged a" + + $ hg merge b + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged b" + + $ hg up a + switching to topic a + 0 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ echo "a bad commit" >> a_bad_commit + $ hg add a_bad_commit + $ hg ci -m "a bad commit" + $ hg up integration + 1 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ hg merge a + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged a bad commit" + + $ hg up a + switching to topic a + 0 files updated, 0 files merged, 1 files removed, 0 files unresolved + $ echo "aa" >> a + $ hg ci -m "aa" + $ hg up integration + 2 files updated, 0 files merged, 0 files removed, 0 files unresolved + $ hg merge a + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged aa" + + $ hg up b + switching to topic b + 0 files updated, 0 files merged, 2 files removed, 0 files unresolved + $ echo "bb" >> b + $ hg ci -m "bb" + $ hg up integration + 3 files updated, 0 files merged, 0 files removed, 0 files unresolved + $ hg merge b + 1 files updated, 0 files merged, 0 files removed, 0 files unresolved + (branch merge, don't forget to commit) + $ hg ci -m "merged bb" + +create instability by pruning two changesets, one in a topic, one in a merge + $ hg prune -r 5:6 + 2 changesets pruned + 3 new orphan changesets + + $ hg up 4 + 2 files updated, 0 files merged, 1 files removed, 0 files unresolved + +start the evolve + $ hg evolve + move:[8] merged aa + atop:[4] merged b + working directory is now at c920dd828523 + +evolve creates an obsolete changeset above as 11 + $ hg evolve -r . + cannot solve instability of c920dd828523, skipping + cannot solve instability of c920dd828523, skipping ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2 evolve-ext] resolves 6028: return return (False, None) instead of return (False, '')
On Tue, Dec 11, 2018 at 7:18 PM James Reynolds < james.glenn.reyno...@gmail.com> wrote: > > On Wed, Dec 5, 2018 at 5:15 PM Martin von Zweigbergk < > martinv...@google.com> wrote: > >> >> >> On Tue, Dec 4, 2018 at 1:19 PM James Reynolds < >> james.glenn.reyno...@gmail.com> wrote: >> >>> # HG changeset patch >>> # User James Reynolds >>> # Date 1543952045 18000 >>> # Tue Dec 04 14:34:05 2018 -0500 >>> # Branch stable >>> # Node ID 6ea3ff2517b9ebf12f0678f28c9942c45c5924ce >>> # Parent 91aec886fc8701398182ab83adcdc04c86735c1d >>> # EXP-Topic issue6028 >>> resolves 6028: return return (False, None) instead of return (False, '') >>> >> >> repo[''] used to mean repo['.']. Is the right return value still (False, >> None)? >> > > Makes sense to me. I think Pulkit Goyal said he was going to take a look > at this. > Sorry for missing to have a look before. I agree with @martinvonz suggestion and the default return value should be '.'. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2 evolve-ext] resolves 6028: return return (False, None) instead of return (False, '')
On Wed, Dec 5, 2018 at 5:15 PM Martin von Zweigbergk
wrote:
>
>
> On Tue, Dec 4, 2018 at 1:19 PM James Reynolds <
> james.glenn.reyno...@gmail.com> wrote:
>
>> # HG changeset patch
>> # User James Reynolds
>> # Date 1543952045 18000
>> # Tue Dec 04 14:34:05 2018 -0500
>> # Branch stable
>> # Node ID 6ea3ff2517b9ebf12f0678f28c9942c45c5924ce
>> # Parent 91aec886fc8701398182ab83adcdc04c86735c1d
>> # EXP-Topic issue6028
>> resolves 6028: return return (False, None) instead of return (False, '')
>>
>
> repo[''] used to mean repo['.']. Is the right return value still (False,
> None)?
>
Makes sense to me. I think Pulkit Goyal said he was going to take a look at
this.
>
>
>> - repo[] no longer takes an emptry string
>
>
>> diff -r 91aec886fc87 -r 6ea3ff2517b9 hgext3rd/evolve/evolvecmd.py
>> --- a/hgext3rd/evolve/evolvecmd.py Tue Dec 04 14:56:19 2018 -0500
>> +++ b/hgext3rd/evolve/evolvecmd.py Tue Dec 04 14:34:05 2018 -0500
>> @@ -61,7 +61,7 @@
>> bool: a boolean value indicating whether the instability was
>> solved
>> newnode: if bool is True, then the newnode of the resultant
>> commit
>> formed. newnode can be node, when resolution led to no
>> new
>> - commit. If bool is False, this is ''.
>> + commit. If bool is False, this is None.
>> """
>> displayer = None
>> if stacktmplt:
>> @@ -101,7 +101,7 @@
>> bool: a boolean value indicating whether the instability was
>> solved
>> newnode: if bool is True, then the newnode of the resultant
>> commit
>> formed. newnode can be node, when resolution led to no
>> new
>> - commit. If bool is False, this is ''.
>> + commit. If bool is False, this is None.
>> """
>> pctx = orig.p1()
>> keepbranch = orig.p1().branch() != orig.branch()
>> @@ -125,7 +125,7 @@
>>
>> if not pctx.obsolete():
>> ui.warn(_("cannot solve instability of %s, skipping\n") % orig)
>> -return (False, '')
>> +return (False, None)
>> obs = pctx
>> newer = obsutil.successorssets(repo, obs.node())
>> # search of a parent which is not killed
>> @@ -139,7 +139,7 @@
>> msg = _("skipping %s: divergent rewriting. can't choose "
>> "destination\n") % obs
>> ui.write_err(msg)
>> -return (False, '')
>> +return (False, None)
>> targets = newer[0]
>> assert targets
>> if len(targets) > 1:
>> @@ -157,7 +157,7 @@
>> "ambiguous destination: "
>> "parent split across two branches\n")
>> ui.write_err(msg)
>> -return (False, '')
>> +return (False, None)
>> target = repo[selectedrev]
>> else:
>> target = repo[heads.first()]
>> @@ -177,7 +177,7 @@
>> todo = 'hg rebase -r %s -d %s\n' % (orig, target)
>> if dryrun:
>> repo.ui.write(todo)
>> -return (False, '')
>> +return (False, None)
>> else:
>> repo.ui.note(todo)
>> if progresscb:
>> @@ -201,7 +201,7 @@
>> bool: a boolean value indicating whether the instability was
>> solved
>> newnode: if bool is True, then the newnode of the resultant
>> commit
>> formed. newnode can be node, when resolution led to no
>> new
>> - commit. If bool is False, this is ''.
>> + commit. If bool is False, this is None.
>> """
>> repo = repo.unfiltered()
>> bumped = repo[bumped.rev()]
>> @@ -209,14 +209,14 @@
>> if len(bumped.parents()) > 1:
>> msg = _('skipping %s : we do not handle merge yet\n') % bumped
>> ui.write_err(msg)
>> -return (False, '')
>> +return (False, None)
>> prec = repo.set('last(allprecursors(%d) and public())',
>> bumped.rev()).next()
>> # For now we deny target merge
>> if len(prec.parents()) > 1:
>> msg = _('skipping: %s: public version is a merge, '
>> 'this is not handled yet\n') % prec
>> ui.write_err(msg)
>> -return (False, '')
>> +return (False, None)
>>
>> if not ui.quiet or confirm:
>> repo.ui.write(_('recreate:'), label='evolve.operation')
>> @@ -232,7 +232,7 @@
>> repo.ui.write(('hg revert --all --rev %s;\n' % bumped))
>> repo.ui.write(('hg commit --msg "%s update to %s"\n' %
>> (TROUBLES['PHASEDIVERGENT'], bumped)))
>> -return (False, '')
>> +return (False, None)
>> if progresscb:
>> progresscb()
>> tmpctx = bumped
>> @@ -343,7 +343,7 @@
>> bool: a boolean value indicating whether the instability was
>> solved
>> newnode: if bool is True, then the newnode of the resultant
>> commit
>> formed. newnode can be node, when resolution led to no
>> new
>> -
D5410: merge: allow to merge non-conflicting changes outside narrowspec
martinvonz requested changes to this revision. martinvonz added a comment. This revision now requires changes to proceed. I'm pretty sure this doesn't actually perform the merge, it just drops the changes outside the narrowspec. On commit, we need to record that outside/ has the new nodeid that we got from the side we merged in. To do that, we need to remember what that nodeid is, from the time of `hg merge` to the time of `hg commit`. That probably means storing the nodeid in the dirstate (like git does), or maybe in the merge state. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5410 To: pulkit, durin42, #hg-reviewers, martinvonz Cc: martinvonz, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH STABLE] worker: do not swallow exception occurred in main process
# HG changeset patch # User Yuya Nishihara # Date 1544535247 -32400 # Tue Dec 11 22:34:07 2018 +0900 # Branch stable # Node ID da41e2e94cfdddce12778d460d53cc7867f69f10 # Parent 4265bfb53dd3c8961993a8af8a41691bcf03d6d7 worker: do not swallow exception occurred in main process Before, SystemExit(255) would be most likely raised since the worker processes were terminated by the main process and the status would be set to 255 in response. We should instead re-raise the exception occurred first. It's pretty hard to debug problems like the issue 6035 with no traceback. diff --git a/mercurial/worker.py b/mercurial/worker.py --- a/mercurial/worker.py +++ b/mercurial/worker.py @@ -213,11 +213,7 @@ def _posixworker(ui, func, staticargs, a waitforworkers() signal.signal(signal.SIGCHLD, oldchldhandler) selector.close() -status = problem[0] -if status: -if status < 0: -os.kill(os.getpid(), -status) -sys.exit(status) +return problem[0] try: openpipes = len(pipes) while openpipes > 0: @@ -236,7 +232,11 @@ def _posixworker(ui, func, staticargs, a killworkers() cleanup() raise -cleanup() +status = cleanup() +if status: +if status < 0: +os.kill(os.getpid(), -status) +sys.exit(status) def _posixexitstatus(code): '''convert a posix exit status into the same form returned by ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] rust: remove comment about error handling of AncestorsIterator
On Tue, Dec 11, 2018 at 4:27 PM Yuya Nishihara wrote: > # HG changeset patch > # User Yuya Nishihara > # Date 1544534619 -32400 > # Tue Dec 11 22:23:39 2018 +0900 > # Node ID 76d8b20139a3b8b5835c7262216b97275845b582 > # Parent 4e17679c336bc38709c32d9b8972f0e402e0057a > rust: remove comment about error handling of AncestorsIterator > > To be align with 443eb4bc41af "rust: propagate error of index_get_parents() > properly." > > Spotted by Georges Racinet. > Queued, many thanks! ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5408: progress: [RFC] avoid ui.configbool() lookup when progress bar is active
yuja added a comment. The idea sounds good to me. Going further, maybe we can get rid of the `ui` reference from the progbar. There's a reference cycle. > DO NOT LAND. test-check-config.t COMPLAINS FOR SOME REASON. Probably missing `# developer config: progress.debug`. The checker isn't aware of `self` being `ui`. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5408 To: indygreg, #hg-reviewers Cc: yuja, pulkit, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5408: progress: [RFC] avoid ui.configbool() lookup when progress bar is active
The idea sounds good to me. Going further, maybe we can get rid of the `ui` reference from the progbar. There's a reference cycle. > DO NOT LAND. test-check-config.t COMPLAINS FOR SOME REASON. Probably missing `# developer config: progress.debug`. The checker isn't aware of `self` being `ui`. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] rust: remove comment about error handling of AncestorsIterator
# HG changeset patch
# User Yuya Nishihara
# Date 1544534619 -32400
# Tue Dec 11 22:23:39 2018 +0900
# Node ID 76d8b20139a3b8b5835c7262216b97275845b582
# Parent 4e17679c336bc38709c32d9b8972f0e402e0057a
rust: remove comment about error handling of AncestorsIterator
To be align with 443eb4bc41af "rust: propagate error of index_get_parents()
properly."
Spotted by Georges Racinet.
diff --git a/rust/hg-core/src/ancestors.rs b/rust/hg-core/src/ancestors.rs
--- a/rust/hg-core/src/ancestors.rs
+++ b/rust/hg-core/src/ancestors.rs
@@ -105,18 +105,6 @@ impl AncestorsIterator {
/// (case where p1 == rev-1), because it amounts to update the first element
/// of the heap without sifting, which Rust's BinaryHeap doesn't let us do.
/// - we save a few pushes by comparing with `stoprev` before pushing
-///
-/// Error treatment:
-/// We swallow the possible GraphError of conditionally_push_parents() to
-/// respect the Iterator trait in a simple manner: never emitting parents
-/// for the returned revision. We finds this good enough for now, because:
-///
-/// - there's a good chance that invalid revisionss are fed from the start,
-/// and `new()` doesn't swallow the error result.
-/// - this is probably what the Python implementation produces anyway, due
-/// to filtering at each step, and Python code is currently the only
-/// concrete caller we target, so we shouldn't need a finer error treatment
-/// for the time being.
impl Iterator for AncestorsIterator {
type Item = Result;
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@40925: 4 new changesets
4 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/ca6372b7e566 changeset: 40922:ca6372b7e566 user:Gregory Szorc date:Mon Dec 10 17:26:12 2018 + summary: tests: add tests for server-side linknode adjustment with wireprotov2 https://www.mercurial-scm.org/repo/hg/rev/3ed77780f4a6 changeset: 40923:3ed77780f4a6 user:Gregory Szorc date:Mon Dec 10 18:04:12 2018 + summary: wireprotov2: send linknodes to emitfilerevisions() https://www.mercurial-scm.org/repo/hg/rev/08cfa77d7288 changeset: 40924:08cfa77d7288 user:Gregory Szorc date:Mon Dec 10 18:55:08 2018 + summary: wireprotov2: unify file revision collection and linknode derivation https://www.mercurial-scm.org/repo/hg/rev/008f3491dc53 changeset: 40925:008f3491dc53 bookmark:@ tag: tip user:Gregory Szorc date:Mon Dec 10 20:01:07 2018 + summary: perf: add perfprogress command -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2 RESEND] commandserver: preload repository in master server and reuse its file cache
# HG changeset patch
# User Yuya Nishihara
# Date 1540993388 -32400
# Wed Oct 31 22:43:08 2018 +0900
# Node ID 9258d4940c6cd372d2a5bf59454fad81ef0f3e62
# Parent 96ba2adf8adbc016f1d4db03ab61429ad9618569
commandserver: preload repository in master server and reuse its file cache
This greatly speeds up repository operation with lots of obsolete markers:
$ ls -lh .hg/store/obsstore
-rw-r--r-- 1 yuya yuya 21M Dec 2 17:55 .hg/store/obsstore
$ time hg log -G -l10 --pager no
(hg) 1.79s user 0.13s system 99% cpu 1.919 total
(chg uncached) 0.00s user 0.01s system 0% cpu 1.328 total
(chg cached) 0.00s user 0.00s system 3% cpu 0.180 total
As you can see, the implementation of the preloader function is highly
experimental. It works, but I'm yet to be sure how things can be organized.
So I don't want to formalize the API at this point.
diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
--- a/mercurial/commandserver.py
+++ b/mercurial/commandserver.py
@@ -28,6 +28,7 @@ from . import (
error,
loggingutil,
pycompat,
+repocache,
util,
vfs as vfsmod,
)
@@ -511,6 +512,11 @@ class unixforkingservice(object):
self._oldsigchldhandler = None
self._workerpids = set() # updated by signal handler; do not iterate
self._socketunlinked = None
+# experimental config: cmdserver.max-repo-cache
+maxlen = ui.configint(b'cmdserver', b'max-repo-cache')
+if maxlen < 0:
+raise error.Abort(_('negative max-repo-cache size not allowed'))
+self._repoloader = repocache.repoloader(ui, maxlen)
def init(self):
self._sock = socket.socket(socket.AF_UNIX)
@@ -525,6 +531,7 @@ class unixforkingservice(object):
o = signal.signal(signal.SIGCHLD, self._sigchldhandler)
self._oldsigchldhandler = o
self._socketunlinked = False
+self._repoloader.start()
def _unlinksocket(self):
if not self._socketunlinked:
@@ -537,6 +544,7 @@ class unixforkingservice(object):
self._mainipc.close()
self._workeripc.close()
self._unlinksocket()
+self._repoloader.stop()
# don't kill child processes as they have active clients, just wait
self._reapworkers(0)
@@ -590,6 +598,10 @@ class unixforkingservice(object):
return
raise
+# Future improvement: On Python 3.7, maybe gc.freeze() can be used
+# to prevent COW memory from being touched by GC.
+# https://instagram-engineering.com/
+# copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf
pid = os.fork()
if pid:
try:
@@ -622,8 +634,7 @@ class unixforkingservice(object):
if inst.args[0] == errno.EINTR:
return
raise
-
-self.ui.log(b'cmdserver', b'repository: %s\n', path)
+self._repoloader.load(path)
def _sigchldhandler(self, signal, frame):
self._reapworkers(os.WNOHANG)
@@ -671,3 +682,9 @@ class unixforkingservice(object):
repo.__class__ = unixcmdserverrepo
repo._cmdserveripc = self._workeripc
+
+cachedrepo = self._repoloader.get(repo.root)
+if cachedrepo is None:
+return
+repo.ui.log(b'repocache', b'repo from cache: %s\n', repo.root)
+repocache.copycache(cachedrepo, repo)
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -179,11 +179,14 @@ coreconfigitem('cmdserver', 'max-log-fil
coreconfigitem('cmdserver', 'max-log-size',
default='1 MB',
)
+coreconfigitem('cmdserver', 'max-repo-cache',
+default=0,
+)
coreconfigitem('cmdserver', 'message-encodings',
default=list,
)
coreconfigitem('cmdserver', 'track-log',
-default=lambda: ['chgserver', 'cmdserver'],
+default=lambda: ['chgserver', 'cmdserver', 'repocache'],
)
coreconfigitem('color', '.*',
default=None,
diff --git a/mercurial/repocache.py b/mercurial/repocache.py
new file mode 100644
--- /dev/null
+++ b/mercurial/repocache.py
@@ -0,0 +1,131 @@
+# repocache.py - in-memory repository cache for long-running services
+#
+# Copyright 2018 Yuya Nishihara
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import
+
+import collections
+import gc
+import threading
+
+from . import (
+error,
+hg,
+obsolete,
+scmutil,
+util,
+)
+
+class repoloader(object):
+"""Load repositories in background thread
+
+This is designed for a forking server. A cached repo cannot be obtained
+until the server fork()s a worker and the loader thread stops.
+"""
+
+def __init__(self, ui, maxlen):
+self._ui = ui.copy()
+self._cache = util.lrucachedict(max=maxlen)
+# use deque and Event instead of Queue since deque
[PATCH 1 of 2 RESEND] commandserver: add IPC channel to teach repository path on command finished
# HG changeset patch
# User Yuya Nishihara
# Date 1540991943 -32400
# Wed Oct 31 22:19:03 2018 +0900
# Node ID 96ba2adf8adbc016f1d4db03ab61429ad9618569
# Parent 4e17679c336bc38709c32d9b8972f0e402e0057a
commandserver: add IPC channel to teach repository path on command finished
The idea is to load recently-used repositories first in the master process,
and fork(). The forked worker can reuse a warm repository if it's preloaded.
There are a couple of ways of in-memory repository caching. They have pros
and cons:
a. "preload by master"
pros: can use a single cache dict, maximizing cache hit rate
cons: need to reload a repo in master process (because worker process
dies per command)
b. "prefork"
pros: can cache a repo without reloading (as worker processes persist)
cons: lower cache hit rate since each worker has to maintain its own cache
c. "shared memory" (or separate key-value store server)
pros: no need to reload a repo in master process, ideally
cons: need to serialize objects to sharable form
Since my primary goal is to get rid of the cost of loading obsstore without
massive rewrites, (c) doesn't work. (b) isn't ideal since it would require
much more SDRAMs than (a). So I take (a).
The idea credits to Jun Wu.
diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
--- a/mercurial/commandserver.py
+++ b/mercurial/commandserver.py
@@ -506,12 +506,19 @@ class unixforkingservice(object):
raise error.Abort(_('no socket path specified with --address'))
self._servicehandler = handler or unixservicehandler(ui)
self._sock = None
+self._mainipc = None
+self._workeripc = None
self._oldsigchldhandler = None
self._workerpids = set() # updated by signal handler; do not iterate
self._socketunlinked = None
def init(self):
self._sock = socket.socket(socket.AF_UNIX)
+# IPC channel from many workers to one main process; this is actually
+# a uni-directional pipe, but is backed by a DGRAM socket so each
+# message can be easily separated.
+o = socket.socketpair(socket.AF_UNIX, socket.SOCK_DGRAM)
+self._mainipc, self._workeripc = o
self._servicehandler.bindsocket(self._sock, self.address)
if util.safehasattr(procutil, 'unblocksignal'):
procutil.unblocksignal(signal.SIGCHLD)
@@ -527,6 +534,8 @@ class unixforkingservice(object):
def _cleanup(self):
signal.signal(signal.SIGCHLD, self._oldsigchldhandler)
self._sock.close()
+self._mainipc.close()
+self._workeripc.close()
self._unlinksocket()
# don't kill child processes as they have active clients, just wait
self._reapworkers(0)
@@ -543,6 +552,8 @@ class unixforkingservice(object):
selector = selectors.DefaultSelector()
selector.register(self._sock, selectors.EVENT_READ,
self._acceptnewconnection)
+selector.register(self._mainipc, selectors.EVENT_READ,
+ self._handlemainipc)
while True:
if not exiting and h.shouldexit():
# clients can no longer connect() to the domain socket, so
@@ -592,8 +603,10 @@ class unixforkingservice(object):
try:
selector.close()
sock.close()
+self._mainipc.close()
self._runworker(conn)
conn.close()
+self._workeripc.close()
os._exit(0)
except: # never return, hence no re-raises
try:
@@ -601,6 +614,17 @@ class unixforkingservice(object):
finally:
os._exit(255)
+def _handlemainipc(self, sock, selector):
+"""Process messages sent from a worker"""
+try:
+path = sock.recv(32768) # large enough to receive path
+except socket.error as inst:
+if inst.args[0] == errno.EINTR:
+return
+raise
+
+self.ui.log(b'cmdserver', b'repository: %s\n', path)
+
def _sigchldhandler(self, signal, frame):
self._reapworkers(os.WNOHANG)
@@ -628,6 +652,22 @@ class unixforkingservice(object):
h = self._servicehandler
try:
_serverequest(self.ui, self.repo, conn, h.createcmdserver,
- prereposetups=None) # TODO: pass in hook functions
+ prereposetups=[self._reposetup])
finally:
gc.collect() # trigger __del__ since worker process uses os._exit
+
+def _reposetup(self, ui, repo):
+if not repo.local():
+return
+
+class unixcmdserverrepo(repo.__class__):
+def close(self):
+super(unixcmdserverrepo, self).close()
+try:
+self._cmdserveripc.send(self.root)
+except socket.error:
+
Re: [PATCH 4 of 6] revlog: add public CPython function to get parent revisions
On Mon, 10 Dec 2018 18:54:56 +0100, Georges Racinet wrote: > On 12/9/18 6:02 AM, Yuya Nishihara wrote: > > On Wed, 05 Dec 2018 22:43:35 +0900, Yuya Nishihara wrote: > >> # HG changeset patch > >> # User Yuya Nishihara > >> # Date 1543756237 -32400 > >> # Sun Dec 02 22:10:37 2018 +0900 > >> # Node ID 716a73bab79be30c20c75e13324c44205d5e2120 > >> # Parent 3842abba948cd7f4bb3fad6805265a35fb94a083 > >> revlog: add public CPython function to get parent revisions > >> +/* > >> + * Get parents of the given rev. > >> + * > >> + * If the specified rev is out of range, IndexError will be raised. If the > >> + * revlog entry is corrupted, ValueError may be raised. > >> + * > >> + * Returns 0 on success or -1 on failure. > >> + */ > >> +int HgRevlogIndex_GetParents(PyObject *op, int rev, int *ps) > > This is based on the idea that the Rust module will be statically linked > > with > > the cext objects. I thought that would be easier for our use case, > > package-local > > visibility, but I'm not certain. If that makes things complicated, maybe we > > should choose dynamic linking and wrap the function with PyCapsule, as you > > did > > in the PoC code. > > Yes, it's true in the direct-ffi code, I passed the function pointer > around because I was wary of a loop in dependencies (that's a bit silly > since we can't avoid linking the Rust extension within the parsers > module anyway), but also I didn't want to touch the existing C code too > much for my first patch set. This version you're proposing feels simpler > to me. > > Using a capsule in that context wouldn't be much complicated either, all > we'd need in hg-direct-ffi would be to declare and call PyCapsule_Import > (it's obviously designed to use very few CPython API concepts, only > needs a char * for the name). The advantage it'd have then would be that > the same capsule could be used for all types of FFI, and we could do the > same for other functions we could need later on (maybe in other > packages). Adding a new capsule seems less risky for people like me, who > aren't as familiar with mercurial's C code as you are or, if you prefer > to see it that way, will require less review. > > To be more explicit for other mailing-list subscribers, here's the > change in revlog.c I'm doing in my PoC CPython code (this is inside the > module init function) : > > @@ -2846,6 +2846,12 @@ > if (nullentry) > PyObject_GC_UnTrack(nullentry); > > + void *caps = PyCapsule_New( > + index_get_parents, > "mercurial.cext.parsers.index_get_parents_CAPI", > + NULL); > + if (caps != NULL) > + PyModule_AddObject(mod, "index_get_parents_CAPI", caps); > + > #ifdef WITH_RUST > rustlazyancestorsType.tp_new = PyType_GenericNew; > if (PyType_Ready() < 0) > > So, to summarize, I think we should maybe promote capsules as the > preferred way to interface Rust code with inner C code. It wouldn't be > hard to document either. What do you think ? My only concern about using PyCapsule is, we would have to get around some "static mut" variables in Rust if the cost of the name resolution matters. That's what I noticed while reading your rust/hg-cpython code. To be clear, I'm not against using PyCapsule. It's documented as the "right" way to interface C-level APIs across modules. I wrote this patch before reading your upcoming series. That's the only reason I made the GetParents() function public. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 5 of 6] rust: look up HgRevlogIndex_GetParents() from symbol table
On Mon, 10 Dec 2018 19:00:43 +0100, Georges Racinet wrote:
> On 12/5/18 2:43 PM, Yuya Nishihara wrote:
> > # HG changeset patch
> > # User Yuya Nishihara
> > # Date 1543756838 -32400
> > # Sun Dec 02 22:20:38 2018 +0900
> > # Node ID f5cdfa49994e3943ba7c4ce2d66708142f0c7058
> > # Parent 716a73bab79be30c20c75e13324c44205d5e2120
> > rust: look up HgRevlogIndex_GetParents() from symbol table
> >
> > And removes the unused index_get_parents_checked() function.
> >
> > I expect the Index struct will be turned into a pyobject type, though I
> > haven't written any PoC-level patches yet.
>
> I'm not sure what you mean exactly with turning Index into a pyobject
> type. Would you care to elaborate ?
I mean we'll probably want a Rust type wrapping a Python Index object,
something like:
impl Index {
pub fn parents(, py: Python, rev: Revision)
-> PyResult<[Revision; 2]>
...
}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5409: remotefilelog: accepting a None node to cmp
yuja added a comment. > In context.py, basefilectx.cmp explicitly calls it with None, so it has to be > supported. Specifically, this breaks "hg absorb -i" currently. IIUC, `self._filenode` should never be `None` if the given `fctx._filenode` is `None`. If `None` were passed down to the filelog layer, exception would be raised. > returns True if text is different than what is stored. > """ > > > - if node == nullid: +if not node or node == nullid: return True Are we sure that the working-directory data is different from the given text? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5409 To: rdamazio, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: D5409: remotefilelog: accepting a None node to cmp
> In context.py, basefilectx.cmp explicitly calls it with None, so it has to > be > supported. Specifically, this breaks "hg absorb -i" currently. IIUC, `self._filenode` should never be `None` if the given `fctx._filenode` is `None`. If `None` were passed down to the filelog layer, exception would be raised. > returns True if text is different than what is stored. > """ > > -if node == nullid: > +if not node or node == nullid: > return True Are we sure that the working-directory data is different from the given text? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@40921: 2 new changesets
2 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/5014e93a5696 changeset: 40920:5014e93a5696 user:Gregory Szorc date:Mon Dec 10 19:41:43 2018 + summary: tests: add sparserevlog requirement to test-sqlitestore.t https://www.mercurial-scm.org/repo/hg/rev/afdbc9c6a333 changeset: 40921:afdbc9c6a333 bookmark:@ tag: tip user:Gregory Szorc date:Mon Dec 10 16:53:09 2018 + summary: tests: fix empty commit in test -- Repository URL: https://www.mercurial-scm.org/repo/hg ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5410: merge: allow to merge non-conflicting changes outside narrowspec
pulkit added a subscriber: martinvonz. pulkit added a comment. I am dubiuos that my fix is correct. I went through the history and didn't find any explanation why we don't allow merging non-conflicting changes outside narrowspec except TODO's. @martinvonz do you know why we don't allow merging of non-conflicting changes here? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5410 To: pulkit, durin42, #hg-reviewers Cc: martinvonz, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5410: merge: allow to merge non-conflicting changes outside narrowspec
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
This patch allows merging of non-conflicting changes outside narrowspec.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5410
AFFECTED FILES
mercurial/merge.py
tests/test-narrow-merge.t
CHANGE DETAILS
diff --git a/tests/test-narrow-merge.t b/tests/test-narrow-merge.t
--- a/tests/test-narrow-merge.t
+++ b/tests/test-narrow-merge.t
@@ -80,16 +80,14 @@
(no more unresolved files)
$ hg commit -m 'merge inside/f1'
-TODO: Can merge non-conflicting changes outside narrow spec
+Can merge non-conflicting changes outside narrow spec
$ hg update -q 'desc("modify inside/f1")'
$ hg merge 'desc("modify outside/f1")'
- abort: merge affects file 'outside/f1' outside narrow, which is not yet
supported (flat !)
- abort: merge affects file 'outside/' outside narrow, which is not yet
supported (tree !)
- (merging in the other direction may work)
- [255]
+ 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ (branch merge, don't forget to commit)
- $ hg update -q 'desc("modify outside/f1")'
+ $ hg update -q 'desc("modify outside/f1")' -C
$ hg merge 'desc("modify inside/f1")'
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
(branch merge, don't forget to commit)
diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -1109,10 +1109,7 @@
elif action[0] in nooptypes:
del actions[f] # merge does not affect file
elif action[0] in nonconflicttypes:
-raise error.Abort(_('merge affects file \'%s\' outside narrow, '
-'which is not yet supported') % f,
- hint=_('merging in the other direction '
- 'may work'))
+del actions[f]
else:
raise error.Abort(_('conflict in file \'%s\' is outside '
'narrow clone') % f)
To: pulkit, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D5408: progress: [RFC] avoid ui.configbool() lookup when progress bar is active
pulkit added a comment. To add on what commit message says, we have also internally seen progress bar taking a lot of time. I am +1 on getting this change in core. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D5408 To: indygreg, #hg-reviewers Cc: pulkit, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
