D2025: debugcommands: introduce debugpeer command
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY `hg debugpeer ` will establish a connection to a peer repository and print information about it. If you add --debug, it will log low-level protocol request info. This will be useful for upcoming tests around protocol handshaking. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2025 AFFECTED FILES mercurial/debugcommands.py tests/test-completion.t tests/test-debugcommands.t tests/test-help.t CHANGE DETAILS diff --git a/tests/test-help.t b/tests/test-help.t --- a/tests/test-help.t +++ b/tests/test-help.t @@ -948,6 +948,7 @@ debugoptEXP (no help text available) debugpathcomplete complete part or all of a tracked path + debugpeer establish a connection to a peer repository debugpickmergetool examine which merge tool is chosen for specified file debugpushkey access the pushkey key/value protocol diff --git a/tests/test-debugcommands.t b/tests/test-debugcommands.t --- a/tests/test-debugcommands.t +++ b/tests/test-debugcommands.t @@ -381,3 +381,24 @@ https stream v2 + +Test debugpeer + + $ hg --config ui.ssh="\"$PYTHON\" \"$TESTDIR/dummyssh\"" debugpeer ssh://user@dummy/debugrevlog + url: ssh://user@dummy/debugrevlog + local: no + pushable: yes + + $ hg --config ui.ssh="\"$PYTHON\" \"$TESTDIR/dummyssh\"" --debug debugpeer ssh://user@dummy/debugrevlog + running "*" "*/tests/dummyssh" 'user@dummy' 'hg -R debugrevlog serve --stdio' (glob) + devel-peer-request: hello + sending hello command + devel-peer-request: between + devel-peer-request: pairs: 81 bytes + sending between command + remote: 384 + remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN + remote: 1 + url: ssh://user@dummy/debugrevlog + local: no + pushable: yes diff --git a/tests/test-completion.t b/tests/test-completion.t --- a/tests/test-completion.t +++ b/tests/test-completion.t @@ -102,6 +102,7 @@ debugnamecomplete debugobsolete debugpathcomplete + debugpeer debugpickmergetool debugpushkey debugpvec @@ -281,6 +282,7 @@ debugnamecomplete: debugobsolete: flags, record-parents, rev, exclusive, index, delete, date, user, template debugpathcomplete: full, normal, added, removed + debugpeer: debugpickmergetool: rev, changedelete, include, exclude, tool debugpushkey: debugpvec: diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py --- a/mercurial/debugcommands.py +++ b/mercurial/debugcommands.py @@ -1693,6 +1693,25 @@ ui.write('\n'.join(repo.pathto(p, cwd) for p in sorted(files))) ui.write('\n') +@command('debugpeer', [], _('PATH'), norepo=True) +def debugpeer(ui, path): +"""establish a connection to a peer repository""" +# Always enable peer request logging. Requires --debug to display +# though. +overrides = { +('devel', 'debug.peer-request'): True, +} + +with ui.configoverride(overrides): +peer = hg.peer(ui, {}, path) + +local = peer.local() is not None +canpush = peer.canpush() + +ui.write(_('url: %s\n') % peer.url()) +ui.write(_('local: %s\n') % (_('yes') if local else _('no'))) +ui.write(_('pushable: %s\n') % (_('yes') if canpush else _('no'))) + @command('debugpickmergetool', [('r', 'rev', '', _('check for files in this revision'), _('REV')), ('', 'changedelete', None, _('emulate merging change and delete')), To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2036: sshpeer: remove support for connecting to <0.9.1 servers (BC)
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY https://phab.mercurial-scm.org/rHG197d10e157ce848129ff5e7a53cf81d4ca63a932 made this change for the HTTP peer. Let's do the same for the SSH peer. Test output changes as expected. A redundant test has been dropped. .. bc:: Support for connecting to Mercurial servers older than 0.9.1 has been removed. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2036 AFFECTED FILES mercurial/sshpeer.py tests/test-ssh-proto.t CHANGE DETAILS diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t --- a/tests/test-ssh-proto.t +++ b/tests/test-ssh-proto.t @@ -109,7 +109,9 @@ 1 -Connecting to a <0.9.1 server that doesn't support the hello command +Connecting to a <0.9.1 server that doesn't support the hello command. +The client should refuse, as we dropped support for connecting to such +servers. $ SSHSERVERMODE=no-hello hg --debug debugpeer ssh://user@dummy/server running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob) @@ -120,14 +122,8 @@ sending between command remote: 0 remote: 1 - url: ssh://user@dummy/server - local: no - pushable: yes - -The client should interpret this as no capabilities - - $ SSHSERVERMODE=no-hello hg debugcapabilities ssh://user@dummy/server - Main capabilities: + abort: no suitable response from remote hg! + [255] Sending an unknown command to the server results in an empty response to that command diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -243,6 +243,16 @@ caps.update(l[:-1].split(':')[1].split()) break +# Error if we couldn't find a response to ``hello``. This could +# mean: +# +# 1. Remote isn't a Mercurial server +# 2. Remote is a <0.9.1 Mercurial server +# 3. Remote is a future Mercurial server that dropped ``hello`` +#support. +if not caps: +badresponse() + return caps class sshpeer(wireproto.wirepeer): To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2026: tests: add low-level SSH protocol tests
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY We don't really have good low-level tests for the behavior of the SSH wire protocol. This commit attempts to establish some. The added tests consist of a mixture of starting a server with `hg serve --stdio` and sending bytes to it and using `hg debugpeer` to go through the official client code. Having insight into what raw bytes are exchanged as well as what the peer does is useful. We also introduce a test extension for modifying the behavior of the SSH server and peer. For example, we change the server to not recognize the "hello" command, simulating behavior of <0.9.1 servers. These tests are generally useful to have. But the impetus for creating them now is they will be needed for verifying behavior of old clients and servers when a new SSH protocol is introduced. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2026 AFFECTED FILES tests/sshprotoext.py tests/test-ssh-proto.t CHANGE DETAILS diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t new file mode 100644 --- /dev/null +++ b/tests/test-ssh-proto.t @@ -0,0 +1,325 @@ + $ cat >> $HGRCPATH << EOF + > [ui] + > ssh = $PYTHON "$TESTDIR/dummyssh" + > [devel] + > debug.peer-request = true + > [extensions] + > sshprotoext = $TESTDIR/sshprotoext.py + > EOF + + $ hg init server + $ cd server + $ echo 0 > foo + $ hg -q add foo + $ hg commit -m initial + $ cd .. + +Test a normal behaving server, for sanity + + $ hg --debug debugpeer ssh://user@dummy/server + running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob) + devel-peer-request: hello + sending hello command + devel-peer-request: between + devel-peer-request: pairs: 81 bytes + sending between command + remote: 384 + remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN + remote: 1 + url: ssh://user@dummy/server + local: no + pushable: yes + +Server should answer the "hello" command in isolation + + $ hg -R server serve --stdio << EOF + > hello + > EOF + 384 + capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN + +>=0.9.1 clients send a "hello" + "between" for the null range as part of handshake. +Server should reply with capabilities and should send "1\n\n" as a successful +reply with empty response to the "between". + + $ hg -R server serve --stdio << EOF + > hello + > between + > pairs 81 + > - + > EOF + 384 + capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN + 1 + + +SSH banner is not printed by default, ignored by clients + + $ SSHSERVERMODE=banner hg debugpeer ssh://user@dummy/server + url: ssh://user@dummy/server + local: no + pushable: yes + +--debug will print the banner + + $ SSHSERVERMODE=banner hg --debug debugpeer ssh://user@dummy/server + running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob) + devel-peer-request: hello + sending hello command + devel-peer-request: between + devel-peer-request: pairs: 81 bytes + sending between command + remote: banner: line 0 + remote: banner: line 1 + remote: banner: line 2 + remote: banner: line 3 + remote: banner: line 4 + remote: banner: line 5 + remote: banner: line 6 + remote: banner: line 7 + remote: banner: line 8 + remote: banner: line 9 + remote: 384 + remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN + remote: 1 + url: ssh://user@dummy/server + local: no + pushable: yes + +And test the banner with the raw protocol + + $ SSHSERVERMODE=banner hg -R server serve --stdio << EOF + > hello + > between + > pairs 81 + > - + > EOF + banner: line 0 + banner: line 1 + banner: line 2 + banner: line 3 + banner: line 4 + banner: line 5 + banner: line 6 + banner: line 7 + banner: line 8 + banner: line 9 + 384 + capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 $USUAL_BUNDLE2_CAPS_SERVER$ unbundle=HG10GZ,HG10BZ,HG10UN + 1 + + +Connecting to a <0.9.1 server that doesn't support the hello command + + $ SSHSERVERMODE=no-hello hg --debug debugpeer ssh://user@dummy/server + running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdi
D2028: sshpeer: move ssh command and repo creation logic out of __init__
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY It was easier to move both of these at once because repository creation requires various variables and I didn't want to add tons of arguments and code to __init__ that will soon be deleted anyway. We do add an extra argument so we can proxy values to the _validaterepo() call. But this is minimally invasive. Some callers of self._abort() were converted to just raise. Like before, the _abort() call wasn't necessary because self._pipe* aren't populated this early in the object's lifetime. As part of this, various private attributes derived from the parsed URL are no longer used. So we no longer set them. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2028 AFFECTED FILES mercurial/sshpeer.py tests/test-check-interfaces.py CHANGE DETAILS diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py --- a/tests/test-check-interfaces.py +++ b/tests/test-check-interfaces.py @@ -69,7 +69,7 @@ checkobject(badpeer()) checkobject(httppeer.httppeer(ui, 'http://localhost')) checkobject(localrepo.localpeer(dummyrepo())) -checkobject(testingsshpeer(ui, 'ssh://localhost/foo')) +checkobject(testingsshpeer(ui, 'ssh://localhost/foo', False, ())) checkobject(bundlerepo.bundlepeer(dummyrepo())) checkobject(statichttprepo.statichttppeer(dummyrepo())) checkobject(unionrepo.unionpeer(dummyrepo())) diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -115,35 +115,15 @@ return self._main.flush() class sshpeer(wireproto.wirepeer): -def __init__(self, ui, path, create=False): +def __init__(self, ui, path, create=False, sshstate=None): self._url = path self._ui = ui self._pipeo = self._pipei = self._pipee = None u = util.url(path, parsequery=False, parsefragment=False) - -self._user = u.user -self._host = u.host -self._port = u.port self._path = u.path or '.' -sshcmd = self.ui.config("ui", "ssh") -remotecmd = self.ui.config("ui", "remotecmd") -sshaddenv = dict(self.ui.configitems("sshenv")) -sshenv = util.shellenviron(sshaddenv) - -args = util.sshargs(sshcmd, self._host, self._user, self._port) - -if create: -cmd = '%s %s %s' % (sshcmd, args, -util.shellquote("%s init %s" % -(_serverquote(remotecmd), _serverquote(self._path -ui.debug('running %s\n' % cmd) -res = ui.system(cmd, blockedtag='sshpeer', environ=sshenv) -if res != 0: -self._abort(error.RepoError(_("could not create remote repo"))) - -self._validaterepo(sshcmd, args, remotecmd, sshenv) +self._validaterepo(*sshstate) # Begin of _basepeer interface. @@ -377,4 +357,23 @@ if u.passwd is not None: raise error.RepoError(_('password in URL not supported')) -return sshpeer(ui, path, create=create) +sshcmd = ui.config('ui', 'ssh') +remotecmd = ui.config('ui', 'remotecmd') +sshaddenv = dict(ui.configitems('sshenv')) +sshenv = util.shellenviron(sshaddenv) +remotepath = u.path or '.' + +args = util.sshargs(sshcmd, u.host, u.user, u.port) + +if create: +cmd = '%s %s %s' % (sshcmd, args, +util.shellquote('%s init %s' % +(_serverquote(remotecmd), _serverquote(remotepath +ui.debug('running %s\n' % cmd) +res = ui.system(cmd, blockedtag='sshpeer', environ=sshenv) +if res != 0: +raise error.RepoError(_('could not create remote repo')) + +sshstate = (sshcmd, args, remotecmd, sshenv) + +return sshpeer(ui, path, create=create, sshstate=sshstate) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2033: sshpeer: inline I/O into _validaterepo()
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY We want to move the handshake code out of the peer class so the peer factory function can perform the handshake and instantiate a proper class depending on the results. To make that refactor easier to read, we first inline I/O functionality into _validaterepo(). Test output for low-level protocol tests didn't change, thus hopefully demonstrating that this refactor didn't change any material behavior. Because we no longer call _callstream(), our test extension for monkeypatching the peer had to change its hook point. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2033 AFFECTED FILES mercurial/sshpeer.py tests/sshprotoext.py CHANGE DETAILS diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py --- a/tests/sshprotoext.py +++ b/tests/sshprotoext.py @@ -54,18 +54,11 @@ class extrahandshakecommandspeer(sshpeer.sshpeer): """An ssh peer that sends extra commands as part of initial handshake.""" -# There isn't a good hook point. So we wrap _callstream() and inject -# logic when the peer says "hello". -def _callstream(self, cmd, **args): -if cmd != b'hello': -return super(extrahandshakecommandspeer, self)._callstream(cmd, - **args) - +def _validaterepo(self): mode = self._ui.config(b'sshpeer', b'handshake-mode') if mode == b'pre-no-args': self._callstream(b'no-args') -return super(extrahandshakecommandspeer, self)._callstream( -cmd, **args) +return super(extrahandshakecommandspeer, self)._validaterepo() else: raise error.ProgrammingError(b'unknown HANDSHAKECOMMANDMODE: %s' % mode) diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -213,18 +213,37 @@ self._abort(error.RepoError(msg, hint=hint)) try: -# skip any noise generated by remote shell -self._callstream("hello") -r = self._callstream("between", pairs=("%s-%s" % ("0"*40, "0"*40))) +pairsarg = '%s-%s' % ('0' * 40, '0' * 40) + +handshake = [ +'hello\n', +'between\n', +'pairs %d\n' % len(pairsarg), +pairsarg, +] + +requestlog = self.ui.configbool('devel', 'debug.peer-request') + +if requestlog: +self.ui.debug('devel-peer-request: hello\n') +self.ui.debug('sending hello command\n') +if requestlog: +self.ui.debug('devel-peer-request: between\n') +self.ui.debug('devel-peer-request: pairs: %d bytes\n' % + len(pairsarg)) +self.ui.debug('sending between command\n') + +self._pipeo.write(''.join(handshake)) +self._pipeo.flush() except IOError: badresponse() lines = ["", "dummy"] max_noise = 500 while lines[-1] and max_noise: try: -l = r.readline() -self._readerr() +l = self._pipei.readline() +_forwardoutput(self.ui, self._pipee) if lines[-1] == "1\n" and l == "\n": break if l: To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2032: sshpeer: clean up API for sshpeer.__init__ (API)
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY Our refactoring left the state of sshpeer.__init__ in a poor state. "create" was no longer used. Process/pipe arguments were passed poorly. "name" was really a URL. This commit cleans all that up. .. api:: sshpeer.sshpeer.__init__ now receives arguments describing an existing connection instead of creating a connection itself. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2032 AFFECTED FILES mercurial/sshpeer.py tests/test-check-interfaces.py CHANGE DETAILS diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py --- a/tests/test-check-interfaces.py +++ b/tests/test-check-interfaces.py @@ -69,8 +69,8 @@ checkobject(badpeer()) checkobject(httppeer.httppeer(ui, 'http://localhost')) checkobject(localrepo.localpeer(dummyrepo())) -checkobject(testingsshpeer(ui, 'ssh://localhost/foo', False, - (None, None, None, None))) +checkobject(testingsshpeer(ui, 'ssh://localhost/foo', None, None, None, + None)) checkobject(bundlerepo.bundlepeer(dummyrepo())) checkobject(statichttprepo.statichttppeer(dummyrepo())) checkobject(unionrepo.unionpeer(dummyrepo())) diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -158,12 +158,21 @@ return proc, pipei, pipeo, pipee class sshpeer(wireproto.wirepeer): -def __init__(self, ui, path, create=False, sshstate=None): -self._url = path +def __init__(self, ui, url, proc, pipei, pipeo, pipee): +"""Create a peer from an existing SSH connection. + +``proc`` is a handle on the underlying SSH process. +``pipei``, ``pipeo``, and ``pipee`` are handles on the stdin, +stdout, and stderr file descriptors for that process. +""" +self._url = url self._ui = ui # self._subprocess is unused. Keeping a handle on the process # holds a reference and prevents it from being garbage collected. -self._subprocess, self._pipei, self._pipeo, self._pipee = sshstate +self._subprocess = proc +self._pipei = pipei +self._pipeo = pipeo +self._pipee = pipee self._validaterepo() @@ -387,6 +396,4 @@ proc, pipei, pipeo, pipee = _makeconnection(ui, sshcmd, args, remotecmd, remotepath, sshenv) -sshstate = (proc, pipei, pipeo, pipee) - -return sshpeer(ui, path, create=create, sshstate=sshstate) +return sshpeer(ui, path, proc, pipei, pipeo, pipee) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2029: sshpeer: extract pipe cleanup logic to own function
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY So it can be used outside of instantiated classes. This is needed to support pipe creation before __init__ is called. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2029 AFFECTED FILES mercurial/sshpeer.py CHANGE DETAILS diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -114,6 +114,23 @@ def flush(self): return self._main.flush() +def _cleanuppipes(ui, pipei, pipeo, pipee): +"""Clean up pipes used by an SSH connection.""" +if pipeo: +pipeo.close() +if pipei: +pipei.close() + +if pipee: +# Try to read from the err descriptor until EOF. +try: +for l in pipee: +ui.status(_('remote: '), l) +except (IOError, ValueError): +pass + +pipee.close() + class sshpeer(wireproto.wirepeer): def __init__(self, ui, path, create=False, sshstate=None): self._url = path @@ -221,17 +238,7 @@ raise exception def _cleanup(self): -if self._pipeo is None: -return -self._pipeo.close() -self._pipei.close() -try: -# read the error descriptor until EOF -for l in self._pipee: -self.ui.status(_("remote: "), l) -except (IOError, ValueError): -pass -self._pipee.close() +_cleanuppipes(self.ui, self._pipei, self._pipeo, self._pipee) __del__ = _cleanup To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2031: sshpeer: establish SSH connection before class instantiation
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY We want to move the handshake to before peers are created so we can instantiate a different peer class depending on the results of the handshake. This necessitates moving the SSH process invocation to outside the peer class. As part of this, the last use of self._path disappeared, so we stop setting that attribute and we can delete the redundant URL parsing necessary to set it. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2031 AFFECTED FILES mercurial/sshpeer.py tests/test-check-interfaces.py CHANGE DETAILS diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py --- a/tests/test-check-interfaces.py +++ b/tests/test-check-interfaces.py @@ -69,7 +69,8 @@ checkobject(badpeer()) checkobject(httppeer.httppeer(ui, 'http://localhost')) checkobject(localrepo.localpeer(dummyrepo())) -checkobject(testingsshpeer(ui, 'ssh://localhost/foo', False, ())) +checkobject(testingsshpeer(ui, 'ssh://localhost/foo', False, + (None, None, None, None))) checkobject(bundlerepo.bundlepeer(dummyrepo())) checkobject(statichttprepo.statichttppeer(dummyrepo())) checkobject(unionrepo.unionpeer(dummyrepo())) diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -131,16 +131,41 @@ pipee.close() +def _makeconnection(ui, sshcmd, args, remotecmd, path, sshenv=None): +"""Create an SSH connection to a server. + +Returns a tuple of (process, stdin, stdout, stderr) for the +spawned process. +""" +cmd = '%s %s %s' % ( +sshcmd, +args, +util.shellquote('%s -R %s serve --stdio' % ( +_serverquote(remotecmd), _serverquote(path + +ui.debug('running %s\n' % cmd) +cmd = util.quotecommand(cmd) + +# no buffer allow the use of 'select' +# feel free to remove buffering and select usage when we ultimately +# move to threading. +pipeo, pipei, pipee, proc = util.popen4(cmd, bufsize=0, env=sshenv) + +pipei = util.bufferedinputpipe(pipei) +pipei = doublepipe(ui, pipei, pipee) +pipeo = doublepipe(ui, pipeo, pipee) + +return proc, pipei, pipeo, pipee + class sshpeer(wireproto.wirepeer): def __init__(self, ui, path, create=False, sshstate=None): self._url = path self._ui = ui -self._pipeo = self._pipei = self._pipee = None +# self._subprocess is unused. Keeping a handle on the process +# holds a reference and prevents it from being garbage collected. +self._subprocess, self._pipei, self._pipeo, self._pipee = sshstate -u = util.url(path, parsequery=False, parsefragment=False) -self._path = u.path or '.' - -self._validaterepo(*sshstate) +self._validaterepo() # Begin of _basepeer interface. @@ -172,28 +197,7 @@ # End of _basewirecommands interface. -def _validaterepo(self, sshcmd, args, remotecmd, sshenv=None): -assert self._pipei is None - -cmd = '%s %s %s' % (sshcmd, args, -util.shellquote("%s -R %s serve --stdio" % -(_serverquote(remotecmd), _serverquote(self._path -self.ui.debug('running %s\n' % cmd) -cmd = util.quotecommand(cmd) - -# while self._subprocess isn't used, having it allows the subprocess to -# to clean up correctly later -# -# no buffer allow the use of 'select' -# feel free to remove buffering and select usage when we ultimately -# move to threading. -sub = util.popen4(cmd, bufsize=0, env=sshenv) -self._pipeo, self._pipei, self._pipee, self._subprocess = sub - -self._pipei = util.bufferedinputpipe(self._pipei) -self._pipei = doublepipe(self.ui, self._pipei, self._pipee) -self._pipeo = doublepipe(self.ui, self._pipeo, self._pipee) - +def _validaterepo(self): def badresponse(): msg = _("no suitable response from remote hg") hint = self.ui.config("ui", "ssherrorhint") @@ -380,6 +384,9 @@ if res != 0: raise error.RepoError(_('could not create remote repo')) -sshstate = (sshcmd, args, remotecmd, sshenv) +proc, pipei, pipeo, pipee = _makeconnection(ui, sshcmd, args, remotecmd, +remotepath, sshenv) + +sshstate = (proc, pipei, pipeo, pipee) return sshpeer(ui, path, create=create, sshstate=sshstate) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2035: sshpeer: document the handshake mechanism
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY The mechanism by which SSH peers establish connections with remotes is wonky and requires a bit of code archeology to understand. While it is already documented in `hg help internals.wireproto`, it helps to have documentation in the code as well. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2035 AFFECTED FILES mercurial/sshpeer.py CHANGE DETAILS diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -163,6 +163,37 @@ hint = ui.config('ui', 'ssherrorhint') raise error.RepoError(msg, hint=hint) +# The handshake consists of sending 2 wire protocol commands: +# ``hello`` and ``between``. +# +# The ``hello`` command (which was introduced in Mercurial 0.9.1) +# instructs the server to advertise its capabilities. +# +# The ``between`` command (which has existed in all Mercurial servers +# for as long as SSH support has existed), asks for the set of revisions +# between a pair of revisions. +# +# The ``between`` command is issued with a request for the null +# range. If the remote is a Mercurial server, this request will +# generate a specific response: ``1\n\n``. This represents the +# wire protocol encoded value for ``\n``. We look for ``1\n\n`` +# in the output stream and know this is the response to ``between`` +# and we're at the end of our handshake reply. +# +# The response to the ``hello`` command will be a line with the +# length of the value returned by that command followed by that +# value. If the server doesn't support ``hello`` (which should be +# rare), that line will be ``0\n``. Otherwise, the value will contain +# RFC 822 like lines. Of these, the ``capabilities:`` line contains +# the capabilities of the server. +# +# In addition to the responses to our command requests, the server +# may emit "banner" output on stdout. SSH servers are allowed to +# print messages to stdout on login. Issuing commands on connection +# allows us to flush this banner output from the server by scanning +# for output to our well-known ``between`` command. Of course, if +# the banner contains ``1\n\n``, this will throw off our detection. + try: pairsarg = '%s-%s' % ('0' * 40, '0' * 40) handshake = [ @@ -206,6 +237,8 @@ caps = set() for l in reversed(lines): +# Look for response to ``hello`` command. Scan from the back so +# we don't misinterpret banner output as the command reply. if l.startswith('capabilities:'): caps.update(l[:-1].split(':')[1].split()) break To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2030: sshpeer: remove frivolous call to _cleanup()
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY _validaterepo() is called once during __init__. _cleanup() no-ops if the self._pipe* attributes aren't set. These attributes are set during _validaterepo(). So the call to _cleanup() isn't necessary. But just to be on the safe side, we add an assertion. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2030 AFFECTED FILES mercurial/sshpeer.py CHANGE DETAILS diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -173,8 +173,7 @@ # End of _basewirecommands interface. def _validaterepo(self, sshcmd, args, remotecmd, sshenv=None): -# cleanup up previous run -self._cleanup() +assert self._pipei is None cmd = '%s %s %s' % (sshcmd, args, util.shellquote("%s -R %s serve --stdio" % To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2034: sshpeer: move handshake outside of sshpeer
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY With the handshake now performed before a peer class is instantiated, we can now instantiate a different peer class depending on the results of the handshake. Our test extension had to change to cope with the new API. Because we now issue the command via raw I/O calls and don't call _callstream(), we no longer have to register the fake command. (_callstream() uses the command registration to see what args to send). REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2034 AFFECTED FILES mercurial/sshpeer.py tests/sshprotoext.py tests/test-check-interfaces.py tests/test-ssh-proto.t CHANGE DETAILS diff --git a/tests/test-ssh-proto.t b/tests/test-ssh-proto.t --- a/tests/test-ssh-proto.t +++ b/tests/test-ssh-proto.t @@ -146,7 +146,6 @@ $ hg --config sshpeer.mode=extra-handshake-commands --config sshpeer.handshake-mode=pre-no-args --debug debugpeer ssh://user@dummy/server running * "*/tests/dummyssh" 'user@dummy' 'hg -R server serve --stdio' (glob) - devel-peer-request: no-args sending no-args command devel-peer-request: hello sending hello command diff --git a/tests/test-check-interfaces.py b/tests/test-check-interfaces.py --- a/tests/test-check-interfaces.py +++ b/tests/test-check-interfaces.py @@ -51,10 +51,6 @@ pass # Facilitates testing sshpeer without requiring an SSH server. -class testingsshpeer(sshpeer.sshpeer): -def _validaterepo(self, *args, **kwargs): -pass - class badpeer(httppeer.httppeer): def __init__(self): super(badpeer, self).__init__(uimod.ui(), 'http://localhost') @@ -69,8 +65,8 @@ checkobject(badpeer()) checkobject(httppeer.httppeer(ui, 'http://localhost')) checkobject(localrepo.localpeer(dummyrepo())) -checkobject(testingsshpeer(ui, 'ssh://localhost/foo', None, None, None, - None)) +checkobject(sshpeer.sshpeer(ui, 'ssh://localhost/foo', None, None, None, + None, None)) checkobject(bundlerepo.bundlepeer(dummyrepo())) checkobject(statichttprepo.statichttppeer(dummyrepo())) checkobject(unionrepo.unionpeer(dummyrepo())) diff --git a/tests/sshprotoext.py b/tests/sshprotoext.py --- a/tests/sshprotoext.py +++ b/tests/sshprotoext.py @@ -12,6 +12,7 @@ from mercurial import ( error, +extensions, registrar, sshpeer, wireproto, @@ -52,22 +53,17 @@ super(prehelloserver, self).serve_forever() -class extrahandshakecommandspeer(sshpeer.sshpeer): -"""An ssh peer that sends extra commands as part of initial handshake.""" -def _validaterepo(self): -mode = self._ui.config(b'sshpeer', b'handshake-mode') -if mode == b'pre-no-args': -self._callstream(b'no-args') -return super(extrahandshakecommandspeer, self)._validaterepo() -else: -raise error.ProgrammingError(b'unknown HANDSHAKECOMMANDMODE: %s' % - mode) - -def registercommands(): -def dummycommand(repo, proto): -raise error.ProgrammingError('this should never be called') - -wireproto.wireprotocommand(b'no-args', b'')(dummycommand) +def performhandshake(orig, ui, pipei, pipeo, pipee): +"""Wrapped version of sshpeer._performhandshake to send extra commands.""" +mode = ui.config(b'sshpeer', b'handshake-mode') +if mode == b'pre-no-args': +ui.debug('sending no-args command\n') +pipeo.write(b'no-args\n') +pipeo.flush() +return orig(ui, pipei, pipeo, pipee) +else: +raise error.ProgrammingError(b'unknown HANDSHAKECOMMANDMODE: %s' % + mode) def extsetup(ui): # It's easier for tests to define the server behavior via environment @@ -86,7 +82,6 @@ peermode = ui.config(b'sshpeer', b'mode') if peermode == b'extra-handshake-commands': -sshpeer.sshpeer = extrahandshakecommandspeer -registercommands() +extensions.wrapfunction(sshpeer, '_performhandshake', performhandshake) elif peermode: raise error.ProgrammingError(b'unknown peer mode: %s' % peermode) diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -157,13 +157,69 @@ return proc, pipei, pipeo, pipee +def _performhandshake(ui, pipei, pipeo, pipee): +def badresponse(): +msg = _('no suitable response from remote hg') +hint = ui.config('ui', 'ssherrorhint') +raise error.RepoError(msg, hint=hint) + +try: +pairsarg = '%s-%s' % ('0' * 40, '0' * 40) +handshake = [ +'hello\n', +'between\n', +'pairs %d\n' % len(pairsarg), +pairsarg, +] + +requestlog = ui.configbool('devel', 'debug.peer-request') + +
D2024: sshpeer: make "instance" a function
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY The API is that peer modules must provide an "instance" symbol that is callable to return a peer. Making "instance" a function instead of an alias to "sshpeer" makes it easier to monkeypatch the "sshpeer" type. It will also make it possible to turn instance() into a factory function of sorts that returns different types based on connection properties. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2024 AFFECTED FILES mercurial/sshpeer.py CHANGE DETAILS diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -370,4 +370,5 @@ self._pipeo.flush() self._readerr() -instance = sshpeer +def instance(ui, path, create): +return sshpeer(ui, path, create=create) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2027: sshpeer: move URL validation out of sshpeer.__init__
indygreg created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY We will soon have another SSH peer class to support the new version of the SSH protocol. However, we won't know which peer class to instantiate until we perform a handshake on an active connection. This means that we need to move connection establishment and handshake code out of sshpeer.__init__. This commit starts the process of migrating peer creation code out of sshpeer.__init__ into instance(), which is the API for creating peers. The moved code no longer calls _abort(). _abort() runs _cleanup() and raises. _cleanup() only performs actions on self._pipe*. These objects aren't instantiated until we actually connect to the peer. So _abort() was not necessary in the old code. To keep the API the same, __init__() now makes a redundant call to util.url(). This will be fixed in subsequent commits. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2027 AFFECTED FILES mercurial/sshpeer.py CHANGE DETAILS diff --git a/mercurial/sshpeer.py b/mercurial/sshpeer.py --- a/mercurial/sshpeer.py +++ b/mercurial/sshpeer.py @@ -121,13 +121,6 @@ self._pipeo = self._pipei = self._pipee = None u = util.url(path, parsequery=False, parsefragment=False) -if u.scheme != 'ssh' or not u.host or u.path is None: -self._abort(error.RepoError(_("couldn't parse location %s") % path)) - -util.checksafessh(path) - -if u.passwd is not None: -self._abort(error.RepoError(_("password in URL not supported"))) self._user = u.user self._host = u.host @@ -371,4 +364,17 @@ self._readerr() def instance(ui, path, create): +"""Create an SSH peer. + +The returned object conforms to the ``wireproto.wirepeer`` interface. +""" +u = util.url(path, parsequery=False, parsefragment=False) +if u.scheme != 'ssh' or not u.host or u.path is None: +raise error.RepoError(_("couldn't parse location %s") % path) + +util.checksafessh(path) + +if u.passwd is not None: +raise error.RepoError(_('password in URL not supported')) + return sshpeer(ui, path, create=create) To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1995: sshpeer: document the handshake mechanism
indygreg abandoned this revision. indygreg added a comment. I'll submit this as part of another series that targets `sshpeer`. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1995 To: indygreg, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D1856: wireproto: support for pullbundles
indygreg added a comment. @joerg.sonnenberger: if you are willing to wait just a week or two more, I think the next version of the wire protocol that I'm writing will make the "pullbundles" feature significantly better. Specifically, servers will be able to send multiple bundle2 bundles or parts with independent compression settings. In other words, a pullbundles aware server-side handler could "stream" pre-generated bundles from disk and then dynamically emit the data not found in any pre-generated bundles, applying compression for just the dynamic bit. I don't wish to create stop energy and discourage you from working on this feature. But at the same time, I'm reluctant to add new features to the existing wire protocol because the new mechanism will be much, much better and will make features like pullbundles even better. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D1856 To: joerg.sonnenberger, #hg-reviewers, indygreg Cc: indygreg, durin42, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2022: ui: improve ui.write performance when not coloring on Windows
joerg.sonnenberger updated this revision to Diff 5181. REPOSITORY rHG Mercurial CHANGES SINCE LAST UPDATE https://phab.mercurial-scm.org/D2022?vs=5177&id=5181 REVISION DETAIL https://phab.mercurial-scm.org/D2022 AFFECTED FILES mercurial/logcmdutil.py mercurial/ui.py CHANGE DETAILS diff --git a/mercurial/ui.py b/mercurial/ui.py --- a/mercurial/ui.py +++ b/mercurial/ui.py @@ -878,6 +878,18 @@ return "".join(self._buffers.pop()) +def writenolabels(self, **opts): +'''check if write actually uses the label''' +if self._buffers and not opts.get(r'prompt', False): +if not self._bufferapplylabels: +return True +return self._colormode is None + +def canbatchlabelwrites(self, **opts): +'''check if write calls with labels are batchable''' +# Windows color printing is special, see ``write``. +return self._colormode != 'win32' + def write(self, *args, **opts): '''write args to output diff --git a/mercurial/logcmdutil.py b/mercurial/logcmdutil.py --- a/mercurial/logcmdutil.py +++ b/mercurial/logcmdutil.py @@ -79,18 +79,31 @@ width = 80 if not ui.plain(): width = ui.termwidth() -chunks = patch.diff(repo, node1, node2, match, changes, opts=diffopts, -prefix=prefix, relroot=relroot, -hunksfilterfn=hunksfilterfn) -for chunk, label in patch.diffstatui(util.iterlines(chunks), - width=width): -write(chunk, label=label) + +chunks = patch.diff(repo, node1, node2, match, changes, opts=diffopts, +prefix=prefix, relroot=relroot, +hunksfilterfn=hunksfilterfn) + +if fp is not None or ui.writenolabels(): +if stat: +chunks = patch.diffstat(util.iterlines(chunks), width=width) +for chunk in util.filechunkiter(util.chunkbuffer(chunks)): +write(chunk) else: -for chunk, label in patch.diffui(repo, node1, node2, match, - changes, opts=diffopts, prefix=prefix, - relroot=relroot, - hunksfilterfn=hunksfilterfn): -write(chunk, label=label) +if stat: +chunks = patch.diffstatui(util.iterlines(chunks), width=width) +else: +chunks = patch.difflabel(lambda chunks, **kwargs: chunks, chunks, + opts=diffopts) +if ui.canbatchlabelwrites(): +def gen(): +for chunk, label in chunks: +yield ui.label(chunk, label=label) +for chunk in util.filechunkiter(util.chunkbuffer(gen())): +write(chunk) +else: +for chunk, label in chunks: +write(chunk, label=label) if listsubrepos: ctx1 = repo[node1] To: joerg.sonnenberger, #hg-reviewers, yuja Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2023: cmdutil: introduce deprecated aliases
lothiraldan created this revision. Herald added a subscriber: mercurial-devel. Herald added a reviewer: hg-reviewers. REVISION SUMMARY https://phab.mercurial-scm.org/rHGc8e2d6ed1f9ea4eee7b28250b5aff22433d538b6 moved some objects used by Evolve and hence broke the latest Evolve revision. Next Evolve version will use the new objects when available but introduce deprecated aliases so users using older version of Evolve won't have a broken Evolve extension. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2023 AFFECTED FILES mercurial/cmdutil.py CHANGE DETAILS diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py --- a/mercurial/cmdutil.py +++ b/mercurial/cmdutil.py @@ -3124,3 +3124,24 @@ if after[1]: hint = after[0] raise error.Abort(_('no %s in progress') % task, hint=hint) + +class changeset_printer(logcmdutil.changesetprinter): + +def __init__(self, ui, *args, **kwargs): +msg = ("'cmdutil.changeset_printer' is deprecated, " + "use 'logcmdutil.logcmdutil'") +ui.deprecwarn(msg, "4.6") +super(changeset_printer, self).__init__(ui, *args, **kwargs) + +def displaygraph(ui, *args, **kwargs): +msg = ("'cmdutil.displaygraph' is deprecated, " + "use 'logcmdutil.displaygraph'") +ui.deprecwarn(msg, "4.6") +return logcmdutil.displaygraph(ui, *args, **kwargs) + +def show_changeset(ui, *args, **kwargs): +msg = ("'cmdutil.show_changeset' is deprecated, " + "use 'logcmdutil.changesetdisplayer'") +ui.deprecwarn(msg, "4.6") +return logcmdutil.changesetdisplayer(ui, *args, **kwargs) + To: lothiraldan, #hg-reviewers Cc: mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 3] logcmdutil: mark changesetprinter.showpatch() as private
# HG changeset patch # User Yuya Nishihara # Date 1516510856 -32400 # Sun Jan 21 14:00:56 2018 +0900 # Node ID 4d7182357056c2672716d7caf849231d7b25691a # Parent f1a8a49af81a97618a4b1eb7e78c7372db776cdc logcmdutil: mark changesetprinter.showpatch() as private diff --git a/mercurial/logcmdutil.py b/mercurial/logcmdutil.py --- a/mercurial/logcmdutil.py +++ b/mercurial/logcmdutil.py @@ -251,7 +251,7 @@ class changesetprinter(object): label='log.summary') self.ui.write("\n") -self.showpatch(ctx, matchfn, hunksfilterfn=hunksfilterfn) +self._showpatch(ctx, matchfn, hunksfilterfn=hunksfilterfn) def _showobsfate(self, ctx): obsfate = templatekw.showobsfate(repo=self.repo, ctx=ctx, ui=self.ui) @@ -265,7 +265,7 @@ class changesetprinter(object): '''empty method used by extension as a hook point ''' -def showpatch(self, ctx, matchfn, hunksfilterfn=None): +def _showpatch(self, ctx, matchfn, hunksfilterfn=None): if not matchfn: matchfn = self.matchfn if matchfn: @@ -469,7 +469,7 @@ class changesettemplater(changesetprinte # write changeset metadata, then patch if requested key = self._parts[self._tref] self.ui.write(templater.stringify(self.t(key, **props))) -self.showpatch(ctx, matchfn, hunksfilterfn=hunksfilterfn) +self._showpatch(ctx, matchfn, hunksfilterfn=hunksfilterfn) if self._parts['footer']: if not self.footer: ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 3] logcmdutil: make default parameters of changesetprinters consistent
# HG changeset patch # User Yuya Nishihara # Date 1516510026 -32400 # Sun Jan 21 13:47:06 2018 +0900 # Node ID 6815de5ad04ee7cc94071b0806f8859bdc8e6435 # Parent 16b4cc4f0cd6f72f5b7be575a7498ed0017ccea5 logcmdutil: make default parameters of changesetprinters consistent diff --git a/mercurial/logcmdutil.py b/mercurial/logcmdutil.py --- a/mercurial/logcmdutil.py +++ b/mercurial/logcmdutil.py @@ -122,12 +122,12 @@ def changesetlabels(ctx): class changesetprinter(object): '''show changeset information when templating not requested.''' -def __init__(self, ui, repo, matchfn, diffopts, buffered): +def __init__(self, ui, repo, matchfn=None, diffopts=None, buffered=False): self.ui = ui self.repo = repo self.buffered = buffered self.matchfn = matchfn -self.diffopts = diffopts +self.diffopts = diffopts or {} self.header = {} self.hunk = {} self.lastheader = None @@ -290,7 +290,7 @@ class changesetprinter(object): class jsonchangeset(changesetprinter): '''format changeset information.''' -def __init__(self, ui, repo, matchfn, diffopts, buffered): +def __init__(self, ui, repo, matchfn=None, diffopts=None, buffered=False): changesetprinter.__init__(self, ui, repo, matchfn, diffopts, buffered) self.cache = {} self._first = True @@ -399,8 +399,6 @@ class changesettemplater(changesetprinte # adding/removing arguments before "buffered" to not break callers. def __init__(self, ui, repo, tmplspec, matchfn=None, diffopts=None, buffered=False): -diffopts = diffopts or {} - changesetprinter.__init__(self, ui, repo, matchfn, diffopts, buffered) tres = formatter.templateresources(ui, repo) self.t = formatter.loadtemplater(ui, tmplspec, ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 3] logcmdutil: drop default arguments from changesetdisplayer/templater() calls
# HG changeset patch # User Yuya Nishihara # Date 1516512483 -32400 # Sun Jan 21 14:28:03 2018 +0900 # Node ID f1a8a49af81a97618a4b1eb7e78c7372db776cdc # Parent 6815de5ad04ee7cc94071b0806f8859bdc8e6435 logcmdutil: drop default arguments from changesetdisplayer/templater() calls diff --git a/hgext/bugzilla.py b/hgext/bugzilla.py --- a/hgext/bugzilla.py +++ b/hgext/bugzilla.py @@ -1091,8 +1091,7 @@ class bugzilla(object): tmpl = _('changeset {node|short} in repo {root} refers ' 'to bug {bug}.\ndetails:\n\t{desc|tabindent}') spec = logcmdutil.templatespec(tmpl, mapfile) -t = logcmdutil.changesettemplater(self.ui, self.repo, spec, - False, None, False) +t = logcmdutil.changesettemplater(self.ui, self.repo, spec) self.ui.pushbuffer() t.show(ctx, changes=ctx.changeset(), bug=str(bugid), diff --git a/hgext/journal.py b/hgext/journal.py --- a/hgext/journal.py +++ b/hgext/journal.py @@ -503,8 +503,7 @@ def journal(ui, repo, *args, **opts): fm.write('command', ' %s\n', entry.command) if opts.get("commits"): -displayer = logcmdutil.changesetdisplayer(ui, repo, opts, - buffered=False) +displayer = logcmdutil.changesetdisplayer(ui, repo, opts) for hash in entry.newhashes: try: ctx = repo[hash] diff --git a/hgext/notify.py b/hgext/notify.py --- a/hgext/notify.py +++ b/hgext/notify.py @@ -258,8 +258,7 @@ class notifier(object): if not mapfile and not template: template = deftemplates.get(hooktype) or single_template spec = logcmdutil.templatespec(template, mapfile) -self.t = logcmdutil.changesettemplater(self.ui, self.repo, spec, - False, None, False) +self.t = logcmdutil.changesettemplater(self.ui, self.repo, spec) def strip(self, path): '''strip leading slashes from local path, turn into web-safe path.''' diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py --- a/mercurial/cmdutil.py +++ b/mercurial/cmdutil.py @@ -2497,7 +2497,7 @@ def commitforceeditor(repo, ctx, subs, f def buildcommittemplate(repo, ctx, subs, extramsg, ref): ui = repo.ui spec = formatter.templatespec(ref, None, None) -t = logcmdutil.changesettemplater(ui, repo, spec, None, {}, False) +t = logcmdutil.changesettemplater(ui, repo, spec) t.t.cache.update((k, templater.unquotestring(v)) for k, v in repo.ui.configitems('committemplate')) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 2] patch: unify check_binary and binary flags
# HG changeset patch # User Yuya Nishihara # Date 1517707683 -32400 # Sun Feb 04 10:28:03 2018 +0900 # Node ID d41b22b06360ceee3a9e6e66df5f57f267318314 # Parent a9802c9ecfb5aa20d89480763ae15b03f78f3a88 patch: unify check_binary and binary flags Follows up 079b27b5a869. If opts.text=True, check_binary is ignored, so we can just pass the binary flag to unidiff(). perfunidiff now takes any inputs as text files, which I think is a desired behavior. diff --git a/contrib/perf.py b/contrib/perf.py --- a/contrib/perf.py +++ b/contrib/perf.py @@ -1088,7 +1088,7 @@ def perfunidiff(ui, repo, file_, rev=Non for left, right in textpairs: # The date strings don't matter, so we pass empty strings. headerlines, hunks = mdiff.unidiff( -left, '', right, '', 'left', 'right') +left, '', right, '', 'left', 'right', binary=False) # consume iterators in roughly the way patch.py does b'\n'.join(headerlines) b''.join(sum((list(hlines) for hrange, hlines in hunks), [])) diff --git a/mercurial/mdiff.py b/mercurial/mdiff.py --- a/mercurial/mdiff.py +++ b/mercurial/mdiff.py @@ -236,7 +236,7 @@ def allblocks(text1, text2, opts=None, l yield s, type yield s1, '=' -def unidiff(a, ad, b, bd, fn1, fn2, opts=defaultopts, check_binary=True): +def unidiff(a, ad, b, bd, fn1, fn2, binary, opts=defaultopts): """Return a unified diff as a (headers, hunks) tuple. If the diff is not null, `headers` is a list with unified diff header @@ -244,8 +244,7 @@ def unidiff(a, ad, b, bd, fn1, fn2, opts (hunkrange, hunklines) coming from _unidiff(). Otherwise, `headers` and `hunks` are empty. -Setting `check_binary` to false will skip the binary check, i.e. when -it has been done in advance. Files are expected to be text in this case. +Set binary=True if either a or b should be taken as a binary file. """ def datetag(date, fn=None): if not opts.git and not opts.nodates: @@ -269,7 +268,7 @@ def unidiff(a, ad, b, bd, fn1, fn2, opts fn1 = util.pconvert(fn1) fn2 = util.pconvert(fn2) -if not opts.text and check_binary and (util.binary(a) or util.binary(b)): +if binary: if a and b and len(a) == len(b) and a == b: return sentinel headerlines = [] diff --git a/mercurial/patch.py b/mercurial/patch.py --- a/mercurial/patch.py +++ b/mercurial/patch.py @@ -2699,12 +2699,9 @@ def trydiff(repo, revs, ctx1, ctx2, modi flag2 = ctx2.flags(f2) # if binary is True, output "summary" or "base85", but not "text diff" if opts.text: -check_binary = True binary = False else: -check_binary = any(f.isbinary() - for f in [fctx1, fctx2] if f is not None) -binary = check_binary +binary = any(f.isbinary() for f in [fctx1, fctx2] if f is not None) if losedatafn and not opts.git: if (binary or @@ -2794,8 +2791,8 @@ def trydiff(repo, revs, ctx1, ctx2, modi uheaders, hunks = mdiff.unidiff(content1, date1, content2, date2, -path1, path2, opts=opts, -check_binary=check_binary) +path1, path2, +binary=binary, opts=opts) header.extend(uheaders) yield fctx1, fctx2, header, hunks ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 2] mdiff: use str.startswith/endswith() instead of slicing
# HG changeset patch # User Yuya Nishihara # Date 1517707994 -32400 # Sun Feb 04 10:33:14 2018 +0900 # Node ID 16b4cc4f0cd6f72f5b7be575a7498ed0017ccea5 # Parent d41b22b06360ceee3a9e6e66df5f57f267318314 mdiff: use str.startswith/endswith() instead of slicing diff --git a/mercurial/mdiff.py b/mercurial/mdiff.py --- a/mercurial/mdiff.py +++ b/mercurial/mdiff.py @@ -274,7 +274,7 @@ def unidiff(a, ad, b, bd, fn1, fn2, bina headerlines = [] hunks = (None, ['Binary file %s has changed\n' % fn1]), elif not a: -without_newline = b[-1:] != '\n' +without_newline = not b.endswith('\n') b = splitnewlines(b) if a is None: l1 = '--- /dev/null%s' % datetag(epoch) @@ -290,7 +290,7 @@ def unidiff(a, ad, b, bd, fn1, fn2, bina hunklines.append(_missing_newline_marker) hunks = (hunkrange, hunklines), elif not b: -without_newline = a[-1:] != '\n' +without_newline = not a.endswith('\n') a = splitnewlines(a) l1 = "--- %s%s%s" % (aprefix, fn1, datetag(ad, fn1)) if b is None: @@ -383,17 +383,17 @@ def _unidiff(t1, t2, opts=defaultopts): # a newline, print only one marker. That's the only case in # which the hunk can end in a shared line without a newline. skip = False -if t1[-1:] != '\n' and astart + alen == len(l1) + 1: +if not t1.endswith('\n') and astart + alen == len(l1) + 1: for i in xrange(len(hunklines) - 1, -1, -1): -if hunklines[i][0:1] in ('-', ' '): -if hunklines[i][0:1] == ' ': +if hunklines[i].startswith(('-', ' ')): +if hunklines[i].startswith(' '): skip = True hunklines[i] += '\n' hunklines.insert(i + 1, _missing_newline_marker) break -if not skip and t2[-1:] != '\n' and bstart + blen == len(l2) + 1: +if not skip and not t2.endswith('\n') and bstart + blen == len(l2) + 1: for i in xrange(len(hunklines) - 1, -1, -1): -if hunklines[i][0:1] == '+': +if hunklines[i].startswith('+'): hunklines[i] += '\n' hunklines.insert(i + 1, _missing_newline_marker) break ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D2022: ui: improve ui.write performance when not coloring on Windows
yuja requested changes to this revision. yuja added inline comments. This revision now requires changes to proceed. INLINE COMMENTS > logcmdutil.py:97 > +oldchunks = chunks > +chunks = patch.difflabel(lambda **kwargs: oldchunks, > opts=diffopts) > +if ui.canbatchlabelwrites(): I slightly prefer passing `chunks` as an argument in place of `oldchunks` trick. But this is really minor nit. If we make all diffui() batchable (e.g. _exportsingle()) as a follow up, difflabel() can just take `chunks` as an argument. > ui.py:881 > > +def writenolabels(self, **opts): > +'''check if write actually uses the label''' Can you remove unused `opts` parameter? I don't think it will be any useful since we want to feed chunks at once where opts may vary. > ui.py:886 > +return True > +return self._colormode is None > + Perhaps "label -> true" would be preferable than double negative "no label -> false". > ui.py:890 > +'''check if write calls with labels are batchable''' > +assert not self.writenolabels() > +# Windows color printing is special, see ``write``. I think this assert is irrelevant since "no label" writes can be batched. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D2022 To: joerg.sonnenberger, #hg-reviewers, yuja Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
mercurial@35903: 22 new changesets
22 new changesets in mercurial: https://www.mercurial-scm.org/repo/hg/rev/87416288be98 changeset: 35882:87416288be98 user:Yuya Nishihara date:Sat Jan 27 14:17:26 2018 +0900 summary: tests: make doctest py3-compatible again https://www.mercurial-scm.org/repo/hg/rev/0d8024be7166 changeset: 35883:0d8024be7166 user:Gregory Szorc date:Thu Feb 01 21:55:06 2018 -0800 summary: internals: document when "hello" and "capabilities" commands were added https://www.mercurial-scm.org/repo/hg/rev/197d10e157ce changeset: 35884:197d10e157ce user:Gregory Szorc date:Fri Feb 02 13:13:46 2018 -0800 summary: httppeer: remove support for connecting to <0.9.1 servers (BC) https://www.mercurial-scm.org/repo/hg/rev/7625b4f7db70 changeset: 35885:7625b4f7db70 user:Yuya Nishihara date:Sun Jan 21 12:26:42 2018 +0900 summary: cmdutil: split functions of log-like commands to new module (API) https://www.mercurial-scm.org/repo/hg/rev/b0014780c7fc changeset: 35886:b0014780c7fc user:Yuya Nishihara date:Sun Jan 21 12:36:43 2018 +0900 summary: logcmdutil: rename classes and functions to conform to our coding style (API) https://www.mercurial-scm.org/repo/hg/rev/572f36e9a780 changeset: 35887:572f36e9a780 user:Yuya Nishihara date:Sun Jan 21 12:48:39 2018 +0900 summary: logcmdutil: drop redundant "log" from function names (API) https://www.mercurial-scm.org/repo/hg/rev/c8e2d6ed1f9e changeset: 35888:c8e2d6ed1f9e user:Yuya Nishihara date:Sun Jan 21 13:03:03 2018 +0900 summary: cmdutil: drop aliases for logcmdutil functions (API) https://www.mercurial-scm.org/repo/hg/rev/e49c39ffeac2 changeset: 35889:e49c39ffeac2 user:Joerg Sonnenberger date:Thu Jan 25 20:00:58 2018 +0100 summary: ui: improve performance for multi-component writes https://www.mercurial-scm.org/repo/hg/rev/44bc37d20271 changeset: 35890:44bc37d20271 user:Matt Harbison date:Fri Feb 02 23:27:30 2018 -0500 summary: context: drop deprecated methods (API) https://www.mercurial-scm.org/repo/hg/rev/75d9dcb64e7d changeset: 35891:75d9dcb64e7d user:Matt Harbison date:Fri Feb 02 23:45:31 2018 -0500 summary: obsolete: drop deprecated methods (API) https://www.mercurial-scm.org/repo/hg/rev/00a56c83ab64 changeset: 35892:00a56c83ab64 user:Matt Harbison date:Fri Feb 02 23:48:25 2018 -0500 summary: revset: drop deprecated evolution predicates https://www.mercurial-scm.org/repo/hg/rev/78f33dedadd0 changeset: 35893:78f33dedadd0 user:Matt Harbison date:Fri Feb 02 23:52:19 2018 -0500 summary: obsutil: drop deprecated methods (API) https://www.mercurial-scm.org/repo/hg/rev/6289482f6ab5 changeset: 35894:6289482f6ab5 user:Matt Harbison date:Fri Feb 02 23:53:57 2018 -0500 summary: templatekw: drop the deprecated '{troubles}' keyword https://www.mercurial-scm.org/repo/hg/rev/265e91da56fd changeset: 35895:265e91da56fd user:Matt Harbison date:Fri Feb 02 23:57:52 2018 -0500 summary: dirstate: drop deprecated methods (API) https://www.mercurial-scm.org/repo/hg/rev/ed3a7300b7b5 changeset: 35896:ed3a7300b7b5 user:Matt Harbison date:Sat Feb 03 00:01:57 2018 -0500 summary: localrepo: drop the deprecated walk() method (API) https://www.mercurial-scm.org/repo/hg/rev/4b1c04082cdc changeset: 35897:4b1c04082cdc user:Yuya Nishihara date:Sat Jan 27 13:09:49 2018 +0900 summary: py3: replace "if ispy3" by encoding.strtolocal() https://www.mercurial-scm.org/repo/hg/rev/a2b3b5c5a25a changeset: 35898:a2b3b5c5a25a user:Yuya Nishihara date:Sat Jan 27 13:11:46 2018 +0900 summary: py3: replace "if ispy3" by pycompat.bytestr() https://www.mercurial-scm.org/repo/hg/rev/d5457d94e1c9 changeset: 35899:d5457d94e1c9 user:Yuya Nishihara date:Sat Jan 27 13:14:06 2018 +0900 summary: py3: replace "if ispy3" by pycompat.sysbytes() or util.forcebytestr() https://www.mercurial-scm.org/repo/hg/rev/72de5c504833 changeset: 35900:72de5c504833 user:Yuya Nishihara date:Sat Jan 27 13:33:31 2018 +0900 summary: py3: factor out helpers to apply string conversion recursively https://www.mercurial-scm.org/repo/hg/rev/f0827211eb1f changeset: 35901:f0827211eb1f user:Yuya Nishihara date:Sat Jan 27 17:12:35 2018 +0900 summary: py3: build repr() of smartset as bytes then convert to str https://www.mercurial-scm.org/repo/hg/rev/2da4144e6716 changeset: 35902:2da4144e6716 user:Yuya Nishihara date:Sat Jan 27 17:13:51 2018 +0900 summary: py3: format revision number as '%d' in debugrevspec https://www.mercurial-scm.org/repo/hg/rev/1a3e6239 changeset: 35903:1a3e6239 bookmark:@ tag: tip user:Yuya Nishihara date:Sat Jan 27 17:31:25 2018