Re: [PATCH] formatter: add context manager interface for convenience
On Wed, Aug 31, 2016 at 12:05:58AM +0900, Yuya Nishihara wrote: > # HG changeset patch > # User Yuya Nishihara > # Date 1472396405 -32400 > # Mon Aug 29 00:00:05 2016 +0900 > # Node ID 41fea255bd1cda8d0f482b2582aec8a29f86c303 > # Parent 90af59b40d8a007fb8811daf0c3e64aca43aa6b0 > formatter: add context manager interface for convenience > > And port "hg files" to test it. > > As you can see, extra indent is necessary to port to this API. I don't think > we should switch every fm.formatter() call to "with" statement. > LGTM ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 6] [convert] Parse perforce changes when needed, not in the constructor
# HG changeset patch
# User David Soria Parra
# Date 1481139471 28800
# Wed Dec 07 11:37:51 2016 -0800
# Node ID 4e7c528ec68f36e26535614e9c8ef067305bdbbe
# Parent e929ae5ce21443ecce43302abb006148c54ed0e3
[convert] Parse perforce changes when needed, not in the constructor
The perforce source is parsing all changes in the constructor and not deferring
it to when needed. This leads to the source parsing changes before the revmap
is available. Let's move the parsing to when we need it the first time.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -79,7 +79,6 @@
if revs and len(revs) > 1:
raise error.Abort(_("p4 source does not support specifying "
"multiple revisions"))
-self._parse(ui, path)
def setrevmap(self, revmap):
self.revmap = revmap
@@ -224,6 +223,8 @@
self.heads = [lastid]
def getheads(self):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return self.heads
def getfile(self, name, rev):
@@ -299,7 +300,11 @@
return self.changeset[rev]
def gettags(self):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return self.tags
def getchangedfiles(self, rev, i):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return sorted([x[0] for x in self.files[rev]])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 6] [convert] Create commits from revmap list if needed
# HG changeset patch
# User David Soria Parra
# Date 1481143876 28800
# Wed Dec 07 12:51:16 2016 -0800
# Node ID 8dfeb4ee8c626d50f0402a9bd1939ee916bf9b42
# Parent e6aa56ec31278dcb4984f65776a3715a92aba180
[convert] Create commits from revmap list if needed
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -314,6 +314,12 @@
return marshal.load(stdout)
def getcommit(self, rev):
+if rev not in self.changeset and rev not in self.revmap:
+raise error.Abort(
+_("cannot find %s in the revmap or parsed changesets") % rev)
+if rev not in self.changeset:
+d = self._fetch_revision(rev)
+return self._construct_commit(d, parents=None)
return self.changeset[rev]
def gettags(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 6] [convert] Allow passing in a revmap
# HG changeset patch
# User David Soria Parra
# Date 1480896014 28800
# Sun Dec 04 16:00:14 2016 -0800
# Node ID e929ae5ce21443ecce43302abb006148c54ed0e3
# Parent eca976dd5da759a1fb1a71c648624a240156adb9
[convert] Allow passing in a revmap
Implement `setrevmap` which can be used to pass a map of existing revisions.
This will allow incremental imports.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -55,6 +55,7 @@
common.checktool('p4', abort=False)
+self.revmap = {}
self.p4changes = {}
self.heads = {}
self.changeset = {}
@@ -80,6 +81,9 @@
"multiple revisions"))
self._parse(ui, path)
+def setrevmap(self, revmap):
+self.revmap = revmap
+
def _parse_view(self, path):
"Read changes affecting the path"
cmd = 'p4 -G changes -s submitted %s' % util.shellquote(path)
@@ -135,6 +139,9 @@
continue
if self.revs and int(change) > int(self.revs[0]):
continue
+if change in self.revmap:
+lastid = change
+continue
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 6] [convert] Create commits from revmap list if needed
# HG changeset patch
# User David Soria Parra
# Date 1481143876 28800
# Wed Dec 07 12:51:16 2016 -0800
# Node ID 8dfeb4ee8c626d50f0402a9bd1939ee916bf9b42
# Parent e6aa56ec31278dcb4984f65776a3715a92aba180
[convert] Create commits from revmap list if needed
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -314,6 +314,12 @@
return marshal.load(stdout)
def getcommit(self, rev):
+if rev not in self.changeset and rev not in self.revmap:
+raise error.Abort(
+_("cannot find %s in the revmap or parsed changesets") % rev)
+if rev not in self.changeset:
+d = self._fetch_revision(rev)
+return self._construct_commit(d, parents=None)
return self.changeset[rev]
def gettags(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 6] [convert] Encapsulate commit data fetching and commit object creation
# HG changeset patch
# User David Soria Parra
# Date 1481140978 28800
# Wed Dec 07 12:02:58 2016 -0800
# Node ID e6aa56ec31278dcb4984f65776a3715a92aba180
# Parent 4e7c528ec68f36e26535614e9c8ef067305bdbbe
[convert] Encapsulate commit data fetching and commit object creation
Split fetching the `describe` form from Perforce and the commit object creation
into two functions. This allows us to reuse the commit construction for
revisions passed from a revmap.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -142,24 +142,17 @@
lastid = change
continue
-cmd = "p4 -G describe -s %s" % change
-stdout = util.popen(cmd, mode='rb')
-d = marshal.load(stdout)
-desc = self.recode(d.get("desc", ""))
-shortdesc = desc.split("\n", 1)[0]
-t = '%s %s' % (d["change"], repr(shortdesc)[1:-1])
-ui.status(util.ellipsis(t, 80) + '\n')
-
if lastid:
parents = [lastid]
else:
parents = []
-date = (int(d["time"]), 0) # timezone not set
-c = common.commit(author=self.recode(d["user"]),
- date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
- parents=parents, desc=desc, branch=None,
- extra={"p4": change, "convert_revision": change})
+d = self._fetch_revision(change)
+c = self._construct_commit(d, parents)
+
+shortdesc = c.desc.splitlines(True)[0].rstrip('\r\n')
+t = '%s %s' % (c.rev, repr(shortdesc)[1:-1])
+ui.status(util.ellipsis(t, 80) + '\n')
files = []
copies = {}
@@ -296,6 +289,30 @@
raise error.Abort(_("convert from p4 does not support --full"))
return self.files[rev], self.copies[rev], set()
+def _construct_commit(self, obj, parents=None):
+"""
+Constructs a common.commit object from an unmarshalled
+`p4 describe` output
+"""
+desc = self.recode(obj.get("desc", ""))
+shortdesc = desc.split("\n", 1)[0]
+
+date = (int(obj["time"]), 0) # timezone not set
+if parents is None:
+parents = []
+
+return common.commit(author=self.recode(obj["user"]),
+date=util.datestr(date, '%Y-%m-%d %H:%M:%S %1%2'),
+parents=parents, desc=desc, branch=None, rev=obj['change'],
+extra={"p4": obj['change'], "convert_revision": obj['change']})
+
+def _fetch_revision(self, rev):
+"""Return an output of `p4 describe` including author, commit date as
+a dictionary."""
+cmd = "p4 -G describe -s %s" % rev
+stdout = util.popen(cmd, mode='rb')
+return marshal.load(stdout)
+
def getcommit(self, rev):
return self.changeset[rev]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 6] [convert] Encapsulate commit data fetching and commit object creation
# HG changeset patch
# User David Soria Parra
# Date 1481140978 28800
# Wed Dec 07 12:02:58 2016 -0800
# Node ID e6aa56ec31278dcb4984f65776a3715a92aba180
# Parent 4e7c528ec68f36e26535614e9c8ef067305bdbbe
[convert] Encapsulate commit data fetching and commit object creation
Split fetching the `describe` form from Perforce and the commit object creation
into two functions. This allows us to reuse the commit construction for
revisions passed from a revmap.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -142,24 +142,17 @@
lastid = change
continue
-cmd = "p4 -G describe -s %s" % change
-stdout = util.popen(cmd, mode='rb')
-d = marshal.load(stdout)
-desc = self.recode(d.get("desc", ""))
-shortdesc = desc.split("\n", 1)[0]
-t = '%s %s' % (d["change"], repr(shortdesc)[1:-1])
-ui.status(util.ellipsis(t, 80) + '\n')
-
if lastid:
parents = [lastid]
else:
parents = []
-date = (int(d["time"]), 0) # timezone not set
-c = common.commit(author=self.recode(d["user"]),
- date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
- parents=parents, desc=desc, branch=None,
- extra={"p4": change, "convert_revision": change})
+d = self._fetch_revision(change)
+c = self._construct_commit(d, parents)
+
+shortdesc = c.desc.splitlines(True)[0].rstrip('\r\n')
+t = '%s %s' % (c.rev, repr(shortdesc)[1:-1])
+ui.status(util.ellipsis(t, 80) + '\n')
files = []
copies = {}
@@ -296,6 +289,30 @@
raise error.Abort(_("convert from p4 does not support --full"))
return self.files[rev], self.copies[rev], set()
+def _construct_commit(self, obj, parents=None):
+"""
+Constructs a common.commit object from an unmarshalled
+`p4 describe` output
+"""
+desc = self.recode(obj.get("desc", ""))
+shortdesc = desc.split("\n", 1)[0]
+
+date = (int(obj["time"]), 0) # timezone not set
+if parents is None:
+parents = []
+
+return common.commit(author=self.recode(obj["user"]),
+date=util.datestr(date, '%Y-%m-%d %H:%M:%S %1%2'),
+parents=parents, desc=desc, branch=None, rev=obj['change'],
+extra={"p4": obj['change'], "convert_revision": obj['change']})
+
+def _fetch_revision(self, rev):
+"""Return an output of `p4 describe` including author, commit date as
+a dictionary."""
+cmd = "p4 -G describe -s %s" % rev
+stdout = util.popen(cmd, mode='rb')
+return marshal.load(stdout)
+
def getcommit(self, rev):
return self.changeset[rev]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 6] [convert] Don't use long list comprehensions
# HG changeset patch
# User David Soria Parra
# Date 1481139493 28800
# Wed Dec 07 11:38:13 2016 -0800
# Node ID eca976dd5da759a1fb1a71c648624a240156adb9
# Parent ade3103f5cf939007ca73af5e9e5853643dc8264
[convert] Don't use long list comprehensions
We are iterating over p4changes. Make the continue condition more clear
and easier to add new conditions in future patches, by removing the list
comprehension and move the condition into the existing for-loop.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -126,14 +126,16 @@
# handle revision limiting
startrev = self.ui.config('convert', 'p4.startrev', default=0)
-self.p4changes = [x for x in self.p4changes
- if ((not startrev or int(x) >= int(startrev)) and
- (not self.revs or int(x) <= int(self.revs[0])))]
# now read the full changelists to get the list of file revisions
ui.status(_('collecting p4 changelists\n'))
lastid = None
for change in self.p4changes:
+if startrev and int(change) < int(startrev):
+continue
+if self.revs and int(change) > int(self.revs[0]):
+continue
+
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
d = marshal.load(stdout)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 6] [convert] Use convert_revision for P4 imports
# HG changeset patch
# User David Soria Parra
# Date 1480895549 28800
# Sun Dec 04 15:52:29 2016 -0800
# Node ID ade3103f5cf939007ca73af5e9e5853643dc8264
# Parent 48bf03a8f718f870256f79cbdf10550ca92c7f25
[convert] Use convert_revision for P4 imports
We are using convert_revisions in other importers. In order to unify this
we are also using convert_revision for Perforce in addition to the original
'p4'.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -151,7 +151,7 @@
c = common.commit(author=self.recode(d["user"]),
date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
parents=parents, desc=desc, branch=None,
- extra={"p4": change})
+ extra={"p4": change, "convert_revision": change})
files = []
copies = {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 6] [convert] Don't use long list comprehensions
# HG changeset patch
# User David Soria Parra
# Date 1481139493 28800
# Wed Dec 07 11:38:13 2016 -0800
# Node ID eca976dd5da759a1fb1a71c648624a240156adb9
# Parent ade3103f5cf939007ca73af5e9e5853643dc8264
[convert] Don't use long list comprehensions
We are iterating over p4changes. Make the continue condition more clear
and easier to add new conditions in future patches, by removing the list
comprehension and move the condition into the existing for-loop.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -126,14 +126,16 @@
# handle revision limiting
startrev = self.ui.config('convert', 'p4.startrev', default=0)
-self.p4changes = [x for x in self.p4changes
- if ((not startrev or int(x) >= int(startrev)) and
- (not self.revs or int(x) <= int(self.revs[0])))]
# now read the full changelists to get the list of file revisions
ui.status(_('collecting p4 changelists\n'))
lastid = None
for change in self.p4changes:
+if startrev and int(change) < int(startrev):
+continue
+if self.revs and int(change) > int(self.revs[0]):
+continue
+
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
d = marshal.load(stdout)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 6 v2] [convert] Parse perforce changes when needed, not in the constructor
# HG changeset patch
# User David Soria Parra
# Date 1481139471 28800
# Wed Dec 07 11:37:51 2016 -0800
# Node ID 4e7c528ec68f36e26535614e9c8ef067305bdbbe
# Parent e929ae5ce21443ecce43302abb006148c54ed0e3
[convert] Parse perforce changes when needed, not in the constructor
The perforce source is parsing all changes in the constructor and not deferring
it to when needed. This leads to the source parsing changes before the revmap
is available. Let's move the parsing to when we need it the first time.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -79,7 +79,6 @@
if revs and len(revs) > 1:
raise error.Abort(_("p4 source does not support specifying "
"multiple revisions"))
-self._parse(ui, path)
def setrevmap(self, revmap):
self.revmap = revmap
@@ -224,6 +223,8 @@
self.heads = [lastid]
def getheads(self):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return self.heads
def getfile(self, name, rev):
@@ -299,7 +300,11 @@
return self.changeset[rev]
def gettags(self):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return self.tags
def getchangedfiles(self, rev, i):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return sorted([x[0] for x in self.files[rev]])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 6 v2] [convert] Create commits from revmap list if needed
# HG changeset patch
# User David Soria Parra
# Date 1481143876 28800
# Wed Dec 07 12:51:16 2016 -0800
# Node ID 8dfeb4ee8c626d50f0402a9bd1939ee916bf9b42
# Parent e6aa56ec31278dcb4984f65776a3715a92aba180
[convert] Create commits from revmap list if needed
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -314,6 +314,12 @@
return marshal.load(stdout)
def getcommit(self, rev):
+if rev not in self.changeset and rev not in self.revmap:
+raise error.Abort(
+_("cannot find %s in the revmap or parsed changesets") % rev)
+if rev not in self.changeset:
+d = self._fetch_revision(rev)
+return self._construct_commit(d, parents=None)
return self.changeset[rev]
def gettags(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 6 v2] [convert] Don't use long list comprehensions
# HG changeset patch
# User David Soria Parra
# Date 1481139493 28800
# Wed Dec 07 11:38:13 2016 -0800
# Node ID eca976dd5da759a1fb1a71c648624a240156adb9
# Parent ade3103f5cf939007ca73af5e9e5853643dc8264
[convert] Don't use long list comprehensions
We are iterating over p4changes. Make the continue condition more clear
and easier to add new conditions in future patches, by removing the list
comprehension and move the condition into the existing for-loop.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -126,14 +126,16 @@
# handle revision limiting
startrev = self.ui.config('convert', 'p4.startrev', default=0)
-self.p4changes = [x for x in self.p4changes
- if ((not startrev or int(x) >= int(startrev)) and
- (not self.revs or int(x) <= int(self.revs[0])))]
# now read the full changelists to get the list of file revisions
ui.status(_('collecting p4 changelists\n'))
lastid = None
for change in self.p4changes:
+if startrev and int(change) < int(startrev):
+continue
+if self.revs and int(change) > int(self.revs[0]):
+continue
+
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
d = marshal.load(stdout)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 6 v2] [convert] Allow passing in a revmap
# HG changeset patch
# User David Soria Parra
# Date 1480896014 28800
# Sun Dec 04 16:00:14 2016 -0800
# Node ID e929ae5ce21443ecce43302abb006148c54ed0e3
# Parent eca976dd5da759a1fb1a71c648624a240156adb9
[convert] Allow passing in a revmap
Implement `setrevmap` which can be used to pass a map of existing revisions.
This will allow incremental imports.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -55,6 +55,7 @@
common.checktool('p4', abort=False)
+self.revmap = {}
self.p4changes = {}
self.heads = {}
self.changeset = {}
@@ -80,6 +81,9 @@
"multiple revisions"))
self._parse(ui, path)
+def setrevmap(self, revmap):
+self.revmap = revmap
+
def _parse_view(self, path):
"Read changes affecting the path"
cmd = 'p4 -G changes -s submitted %s' % util.shellquote(path)
@@ -135,6 +139,9 @@
continue
if self.revs and int(change) > int(self.revs[0]):
continue
+if change in self.revmap:
+lastid = change
+continue
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 6 v2] [convert] Encapsulate commit data fetching and commit object creation
# HG changeset patch
# User David Soria Parra
# Date 1481140978 28800
# Wed Dec 07 12:02:58 2016 -0800
# Node ID e6aa56ec31278dcb4984f65776a3715a92aba180
# Parent 4e7c528ec68f36e26535614e9c8ef067305bdbbe
[convert] Encapsulate commit data fetching and commit object creation
Split fetching the `describe` form from Perforce and the commit object creation
into two functions. This allows us to reuse the commit construction for
revisions passed from a revmap.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -142,24 +142,17 @@
lastid = change
continue
-cmd = "p4 -G describe -s %s" % change
-stdout = util.popen(cmd, mode='rb')
-d = marshal.load(stdout)
-desc = self.recode(d.get("desc", ""))
-shortdesc = desc.split("\n", 1)[0]
-t = '%s %s' % (d["change"], repr(shortdesc)[1:-1])
-ui.status(util.ellipsis(t, 80) + '\n')
-
if lastid:
parents = [lastid]
else:
parents = []
-date = (int(d["time"]), 0) # timezone not set
-c = common.commit(author=self.recode(d["user"]),
- date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
- parents=parents, desc=desc, branch=None,
- extra={"p4": change, "convert_revision": change})
+d = self._fetch_revision(change)
+c = self._construct_commit(d, parents)
+
+shortdesc = c.desc.splitlines(True)[0].rstrip('\r\n')
+t = '%s %s' % (c.rev, repr(shortdesc)[1:-1])
+ui.status(util.ellipsis(t, 80) + '\n')
files = []
copies = {}
@@ -296,6 +289,30 @@
raise error.Abort(_("convert from p4 does not support --full"))
return self.files[rev], self.copies[rev], set()
+def _construct_commit(self, obj, parents=None):
+"""
+Constructs a common.commit object from an unmarshalled
+`p4 describe` output
+"""
+desc = self.recode(obj.get("desc", ""))
+shortdesc = desc.split("\n", 1)[0]
+
+date = (int(obj["time"]), 0) # timezone not set
+if parents is None:
+parents = []
+
+return common.commit(author=self.recode(obj["user"]),
+date=util.datestr(date, '%Y-%m-%d %H:%M:%S %1%2'),
+parents=parents, desc=desc, branch=None, rev=obj['change'],
+extra={"p4": obj['change'], "convert_revision": obj['change']})
+
+def _fetch_revision(self, rev):
+"""Return an output of `p4 describe` including author, commit date as
+a dictionary."""
+cmd = "p4 -G describe -s %s" % rev
+stdout = util.popen(cmd, mode='rb')
+return marshal.load(stdout)
+
def getcommit(self, rev):
return self.changeset[rev]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 6] [convert] Parse perforce changes when needed, not in the constructor
# HG changeset patch
# User David Soria Parra
# Date 1481139471 28800
# Wed Dec 07 11:37:51 2016 -0800
# Node ID 4e7c528ec68f36e26535614e9c8ef067305bdbbe
# Parent e929ae5ce21443ecce43302abb006148c54ed0e3
[convert] Parse perforce changes when needed, not in the constructor
The perforce source is parsing all changes in the constructor and not deferring
it to when needed. This leads to the source parsing changes before the revmap
is available. Let's move the parsing to when we need it the first time.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -79,7 +79,6 @@
if revs and len(revs) > 1:
raise error.Abort(_("p4 source does not support specifying "
"multiple revisions"))
-self._parse(ui, path)
def setrevmap(self, revmap):
self.revmap = revmap
@@ -224,6 +223,8 @@
self.heads = [lastid]
def getheads(self):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return self.heads
def getfile(self, name, rev):
@@ -299,7 +300,11 @@
return self.changeset[rev]
def gettags(self):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return self.tags
def getchangedfiles(self, rev, i):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return sorted([x[0] for x in self.files[rev]])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 6] [convert] Allow passing in a revmap
# HG changeset patch
# User David Soria Parra
# Date 1480896014 28800
# Sun Dec 04 16:00:14 2016 -0800
# Node ID e929ae5ce21443ecce43302abb006148c54ed0e3
# Parent eca976dd5da759a1fb1a71c648624a240156adb9
[convert] Allow passing in a revmap
Implement `setrevmap` which can be used to pass a map of existing revisions.
This will allow incremental imports.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -55,6 +55,7 @@
common.checktool('p4', abort=False)
+self.revmap = {}
self.p4changes = {}
self.heads = {}
self.changeset = {}
@@ -80,6 +81,9 @@
"multiple revisions"))
self._parse(ui, path)
+def setrevmap(self, revmap):
+self.revmap = revmap
+
def _parse_view(self, path):
"Read changes affecting the path"
cmd = 'p4 -G changes -s submitted %s' % util.shellquote(path)
@@ -135,6 +139,9 @@
continue
if self.revs and int(change) > int(self.revs[0]):
continue
+if change in self.revmap:
+lastid = change
+continue
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 6] [convert] Use convert_revision for P4 imports
# HG changeset patch
# User David Soria Parra
# Date 1480895549 28800
# Sun Dec 04 15:52:29 2016 -0800
# Node ID ade3103f5cf939007ca73af5e9e5853643dc8264
# Parent 48bf03a8f718f870256f79cbdf10550ca92c7f25
[convert] Use convert_revision for P4 imports
We are using convert_revisions in other importers. In order to unify this
we are also using convert_revision for Perforce in addition to the original
'p4'.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -151,7 +151,7 @@
c = common.commit(author=self.recode(d["user"]),
date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
parents=parents, desc=desc, branch=None,
- extra={"p4": change})
+ extra={"p4": change, "convert_revision": change})
files = []
copies = {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 6 v2] [convert] Use convert_revision for P4 imports
# HG changeset patch
# User David Soria Parra
# Date 1480895549 28800
# Sun Dec 04 15:52:29 2016 -0800
# Node ID ade3103f5cf939007ca73af5e9e5853643dc8264
# Parent 48bf03a8f718f870256f79cbdf10550ca92c7f25
[convert] Use convert_revision for P4 imports
We are using convert_revisions in other importers. In order to unify this
we are also using convert_revision for Perforce in addition to the original
'p4'.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -151,7 +151,7 @@
c = common.commit(author=self.recode(d["user"]),
date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
parents=parents, desc=desc, branch=None,
- extra={"p4": change})
+ extra={"p4": change, "convert_revision": change})
files = []
copies = {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 6 v3] convert: Create commits from revmap list if needed
# HG changeset patch
# User David Soria Parra
# Date 1481143876 28800
# Wed Dec 07 12:51:16 2016 -0800
# Node ID 109de539306c5bc49d38d6f1c802c4a8d092b485
# Parent be68e4436851b7e20f3b8cb34666418f5840dd66
convert: Create commits from revmap list if needed
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -314,6 +314,12 @@
return marshal.load(stdout)
def getcommit(self, rev):
+if rev not in self.changeset and rev not in self.revmap:
+raise error.Abort(
+_("cannot find %s in the revmap or parsed changesets") % rev)
+if rev not in self.changeset:
+d = self._fetch_revision(rev)
+return self._construct_commit(d, parents=None)
return self.changeset[rev]
def gettags(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 6 v3] convert: Don't use long list comprehensions
# HG changeset patch
# User David Soria Parra
# Date 1481139493 28800
# Wed Dec 07 11:38:13 2016 -0800
# Node ID c0168cd90fd67ccd20271416e0994abe6f95a201
# Parent 1f9a627a126694d67f3664dae387d8b98fa44ddb
convert: Don't use long list comprehensions
We are iterating over p4changes. Make the continue condition more clear
and easier to add new conditions in future patches, by removing the list
comprehension and move the condition into the existing for-loop.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -126,14 +126,16 @@
# handle revision limiting
startrev = self.ui.config('convert', 'p4.startrev', default=0)
-self.p4changes = [x for x in self.p4changes
- if ((not startrev or int(x) >= int(startrev)) and
- (not self.revs or int(x) <= int(self.revs[0])))]
# now read the full changelists to get the list of file revisions
ui.status(_('collecting p4 changelists\n'))
lastid = None
for change in self.p4changes:
+if startrev and int(change) < int(startrev):
+continue
+if self.revs and int(change) > int(self.revs[0]):
+continue
+
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
d = marshal.load(stdout)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 6 v3] convert: Allow passing in a revmap
# HG changeset patch
# User David Soria Parra
# Date 1480896014 28800
# Sun Dec 04 16:00:14 2016 -0800
# Node ID 897be730ab884aa8f7373a34eeb6a288cc1dabc7
# Parent c0168cd90fd67ccd20271416e0994abe6f95a201
convert: Allow passing in a revmap
Implement `setrevmap` which can be used to pass a map of existing revisions.
Ignore those revisions when going through Perforce changes. This will allow
incremental imports.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -55,6 +55,7 @@
common.checktool('p4', abort=False)
+self.revmap = {}
self.p4changes = {}
self.heads = {}
self.changeset = {}
@@ -80,6 +81,9 @@
"multiple revisions"))
self._parse(ui, path)
+def setrevmap(self, revmap):
+self.revmap = revmap
+
def _parse_view(self, path):
"Read changes affecting the path"
cmd = 'p4 -G changes -s submitted %s' % util.shellquote(path)
@@ -135,6 +139,9 @@
continue
if self.revs and int(change) > int(self.revs[0]):
continue
+if change in self.revmap:
+lastid = change
+continue
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 6 v3] convert: Encapsulate commit data fetching and commit object creation
# HG changeset patch
# User David Soria Parra
# Date 1481140978 28800
# Wed Dec 07 12:02:58 2016 -0800
# Node ID be68e4436851b7e20f3b8cb34666418f5840dd66
# Parent e3c34b2fafd7ce1a78d23e35888a33ef10629fbf
convert: Encapsulate commit data fetching and commit object creation
Split fetching the `describe` form from Perforce and the commit object creation
into two functions. This allows us to reuse the commit construction for
revisions passed from a revmap.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -142,24 +142,17 @@
lastid = change
continue
-cmd = "p4 -G describe -s %s" % change
-stdout = util.popen(cmd, mode='rb')
-d = marshal.load(stdout)
-desc = self.recode(d.get("desc", ""))
-shortdesc = desc.split("\n", 1)[0]
-t = '%s %s' % (d["change"], repr(shortdesc)[1:-1])
-ui.status(util.ellipsis(t, 80) + '\n')
-
if lastid:
parents = [lastid]
else:
parents = []
-date = (int(d["time"]), 0) # timezone not set
-c = common.commit(author=self.recode(d["user"]),
- date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
- parents=parents, desc=desc, branch=None,
- extra={"p4": change, "convert_revision": change})
+d = self._fetch_revision(change)
+c = self._construct_commit(d, parents)
+
+shortdesc = c.desc.splitlines(True)[0].rstrip('\r\n')
+t = '%s %s' % (c.rev, repr(shortdesc)[1:-1])
+ui.status(util.ellipsis(t, 80) + '\n')
files = []
copies = {}
@@ -296,6 +289,30 @@
raise error.Abort(_("convert from p4 does not support --full"))
return self.files[rev], self.copies[rev], set()
+def _construct_commit(self, obj, parents=None):
+"""
+Constructs a common.commit object from an unmarshalled
+`p4 describe` output
+"""
+desc = self.recode(obj.get("desc", ""))
+shortdesc = desc.split("\n", 1)[0]
+
+date = (int(obj["time"]), 0) # timezone not set
+if parents is None:
+parents = []
+
+return common.commit(author=self.recode(obj["user"]),
+date=util.datestr(date, '%Y-%m-%d %H:%M:%S %1%2'),
+parents=parents, desc=desc, branch=None, rev=obj['change'],
+extra={"p4": obj['change'], "convert_revision": obj['change']})
+
+def _fetch_revision(self, rev):
+"""Return an output of `p4 describe` including author, commit date as
+a dictionary."""
+cmd = "p4 -G describe -s %s" % rev
+stdout = util.popen(cmd, mode='rb')
+return marshal.load(stdout)
+
def getcommit(self, rev):
return self.changeset[rev]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 6 v3] convert: Use convert_revision for P4 imports
# HG changeset patch
# User David Soria Parra
# Date 1480895549 28800
# Sun Dec 04 15:52:29 2016 -0800
# Node ID 1f9a627a126694d67f3664dae387d8b98fa44ddb
# Parent 48bf03a8f718f870256f79cbdf10550ca92c7f25
convert: Use convert_revision for P4 imports
We are using convert_revisions in other importers. In order to unify this
we are also using convert_revision for Perforce in addition to the original
'p4'.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -151,7 +151,7 @@
c = common.commit(author=self.recode(d["user"]),
date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
parents=parents, desc=desc, branch=None,
- extra={"p4": change})
+ extra={"p4": change, "convert_revision": change})
files = []
copies = {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 6 v3] convert: Parse perforce changes when needed, not in the constructor
# HG changeset patch
# User David Soria Parra
# Date 1481139471 28800
# Wed Dec 07 11:37:51 2016 -0800
# Node ID e3c34b2fafd7ce1a78d23e35888a33ef10629fbf
# Parent 897be730ab884aa8f7373a34eeb6a288cc1dabc7
convert: Parse perforce changes when needed, not in the constructor
The perforce source is parsing all changes in the constructor and not deferring
it to when needed. This leads to the source parsing changes before the revmap
is available. Let's move the parsing to when we need it the first time.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -79,7 +79,6 @@
if revs and len(revs) > 1:
raise error.Abort(_("p4 source does not support specifying "
"multiple revisions"))
-self._parse(ui, path)
def setrevmap(self, revmap):
self.revmap = revmap
@@ -224,6 +223,8 @@
self.heads = [lastid]
def getheads(self):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return self.heads
def getfile(self, name, rev):
@@ -299,7 +300,11 @@
return self.changeset[rev]
def gettags(self):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return self.tags
def getchangedfiles(self, rev, i):
+if len(self.p4changes) == 0:
+self._parse(self.ui, self.path)
return sorted([x[0] for x in self.files[rev]])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 6 v2] [convert] Use convert_revision for P4 imports
On Wed, Dec 07, 2016 at 09:48:28PM +, Kostia Balytskyi wrote: > On 12/7/16 9:30 PM, David Soria Parra wrote: > > # HG changeset patch > > # User David Soria Parra > > # Date 1480895549 28800 > > # Sun Dec 04 15:52:29 2016 -0800 > > # Node ID ade3103f5cf939007ca73af5e9e5853643dc8264 > > # Parent 48bf03a8f718f870256f79cbdf10550ca92c7f25 > > [convert] Use convert_revision for P4 imports > This does not follow our naming scheme "keyword: some short description" woops. resent as v3 ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 6 of 6 v3] convert: Create commits from revmap list if needed
On Wed, Dec 07, 2016 at 10:23:08PM +, Kostia Balytskyi wrote:
> On 12/7/16 9:51 PM, David Soria Parra wrote:
> > # HG changeset patch
> > # User David Soria Parra
> > # Date 1481143876 28800
> > # Wed Dec 07 12:51:16 2016 -0800
> > # Node ID 109de539306c5bc49d38d6f1c802c4a8d092b485
> > # Parent be68e4436851b7e20f3b8cb34666418f5840dd66
> > convert: Create commits from revmap list if needed
> This commit demonstrates the need to add some sort of comment to
> self.revmap = {} in constructor. Otherwise, it's very confusing IMO.
I honestly don't undersetand what is so confusing about it. It's a
revmap and it seems straight forward to me that it's a dictionary
that maps commits, at least thats what all revmaps in convert ever do.
> >
> > diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
> > --- a/hgext/convert/p4.py
> > +++ b/hgext/convert/p4.py
> > @@ -314,6 +314,12 @@
> > return marshal.load(stdout)
> >
> > def getcommit(self, rev):
> > +if rev not in self.changeset and rev not in self.revmap:
> > +raise error.Abort(
> > +_("cannot find %s in the revmap or parsed changesets") %
> > rev)
> > +if rev not in self.changeset:
> > +d = self._fetch_revision(rev)
> > +return self._construct_commit(d, parents=None)
> > return self.changeset[rev]
> >
> > def gettags(self):
> > ___
> > Mercurial-devel mailing list
> > Mercurial-devel@mercurial-scm.org
> > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 8 v4] convert: allow passing in a revmap
# HG changeset patch
# User David Soria Parra
# Date 1481708757 28800
# Wed Dec 14 01:45:57 2016 -0800
# Node ID 4b9d267d2a05970d9320b83f8bf29ec99f8ab40f
# Parent 92b0ad25b4aca99c74d3813ddae163324c1110a7
convert: allow passing in a revmap
Implement `common.setrevmap` which is used to pass in a file with existing
revision mappings. This functionality is used by `convertcmd.convert` if it
exists and allows implementors such as the p4 converter to make use of an
existing mapping.
We are using the revmap to abort scanning and the repository for more
information
if we already have the revision. This means we are allowing incremental imports
in cases where a revmap is provided.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -55,6 +55,7 @@
common.checktool('p4', abort=False)
+self.revmap = {}
self.p4changes = {}
self.heads = []
self.changeset = {}
@@ -77,6 +78,19 @@
"multiple revisions"))
self._parse(ui, path)
+def setrevmap(self, revmap):
+"""Sets the parsed revmap dictionary.
+
+Revmap stores mappings from a source revision to a target revision.
+It is set in convertcmd.convert and provided by the user as a file
+on the commandline.
+
+Revisions in the map are considered beeing present in the
+repository and ignored during _parse(). This allows for incremental
+imports if a revmap is provided.
+"""
+self.revmap = revmap
+
def _parse_view(self, path):
"Read changes affecting the path"
cmd = 'p4 -G changes -s submitted %s' % util.shellquote(path)
@@ -132,6 +146,10 @@
continue
if self.revs and int(change) > int(self.revs[0]):
continue
+if change in self.revmap:
+# Ignore already present revisions, but set the parent pointer.
+lastid = change
+continue
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 7 of 8 v4] convert: encapsulate commit data fetching and commit object creation
# HG changeset patch
# User David Soria Parra
# Date 1481694598 28800
# Tue Dec 13 21:49:58 2016 -0800
# Node ID 26ec2474ab06fdaf1aca034c54309245ded267b0
# Parent c6c26e238617600d0523b44a9043d17f963f9eda
convert: encapsulate commit data fetching and commit object creation
Split fetching the `describe` form from Perforce and the commit object creation
into two functions. This allows us to reuse the commit construction for
revisions passed from a revmap.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -151,24 +151,17 @@
lastid = change
continue
-cmd = "p4 -G describe -s %s" % change
-stdout = util.popen(cmd, mode='rb')
-d = marshal.load(stdout)
-desc = self.recode(d.get("desc", ""))
-shortdesc = desc.split("\n", 1)[0]
-t = '%s %s' % (d["change"], repr(shortdesc)[1:-1])
-ui.status(util.ellipsis(t, 80) + '\n')
-
if lastid:
parents = [lastid]
else:
parents = []
-date = (int(d["time"]), 0) # timezone not set
-c = common.commit(author=self.recode(d["user"]),
- date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
- parents=parents, desc=desc, branch=None,
- extra={"p4": change, "convert_revision": change})
+d = self._fetch_revision(change)
+c = self._construct_commit(d, parents)
+
+shortdesc = c.desc.splitlines(True)[0].rstrip('\r\n')
+t = '%s %s' % (c.rev, repr(shortdesc)[1:-1])
+ui.status(util.ellipsis(t, 80) + '\n')
files = []
copies = {}
@@ -303,6 +296,30 @@
raise error.Abort(_("convert from p4 does not support --full"))
return self.files[rev], self.copies[rev], set()
+def _construct_commit(self, obj, parents=None):
+"""
+Constructs a common.commit object from an unmarshalled
+`p4 describe` output
+"""
+desc = self.recode(obj.get("desc", ""))
+shortdesc = desc.split("\n", 1)[0]
+
+date = (int(obj["time"]), 0) # timezone not set
+if parents is None:
+parents = []
+
+return common.commit(author=self.recode(obj["user"]),
+date=util.datestr(date, '%Y-%m-%d %H:%M:%S %1%2'),
+parents=parents, desc=desc, branch=None, rev=obj['change'],
+extra={"p4": obj['change'], "convert_revision": obj['change']})
+
+def _fetch_revision(self, rev):
+"""Return an output of `p4 describe` including author, commit date as
+a dictionary."""
+cmd = "p4 -G describe -s %s" % rev
+stdout = util.popen(cmd, mode='rb')
+return marshal.load(stdout)
+
def getcommit(self, rev):
return self.changeset[rev]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 8 of 8 v4] convert: return commit objects for revisions in the revmap
# HG changeset patch
# User David Soria Parra
# Date 1481704712 28800
# Wed Dec 14 00:38:32 2016 -0800
# Node ID eb4a6df3de20deb31e681aa7eba408dc352440c3
# Parent 26ec2474ab06fdaf1aca034c54309245ded267b0
convert: return commit objects for revisions in the revmap
Source revision data that exists in the revmap are ignored when pulling
data from Perforce as we consider them already imported. In case where
the `convertcmd.convert` algorithm requests a commit object for such
a revision we are creating it. This is usually the case for parent of
the first imported revision.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -321,6 +321,12 @@
return marshal.load(stdout)
def getcommit(self, rev):
+if rev not in self.changeset and rev not in self.revmap:
+raise error.Abort(
+_("cannot find %s in the revmap or parsed changesets") % rev)
+if rev not in self.changeset:
+d = self._fetch_revision(rev)
+return self._construct_commit(d, parents=None)
return self.changeset[rev]
def gettags(self):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 8 v4] convert: don't use long list comprehensions
# HG changeset patch
# User David Soria Parra
# Date 1481694598 28800
# Tue Dec 13 21:49:58 2016 -0800
# Node ID 506cb0034ebdbe85d2a7174d9214f833b3fed3fb
# Parent 76e77076e203fa0b42ff371c7a326580f27245cb
convert: don't use long list comprehensions
We are iterating over p4changes. Make the continue condition more clear
and easier to add new conditions in future patches, by removing the list
comprehension and move the condition into the existing for-loop.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -126,14 +126,16 @@
# handle revision limiting
startrev = self.ui.config('convert', 'p4.startrev', default=0)
-self.p4changes = [x for x in self.p4changes
- if ((not startrev or int(x) >= int(startrev)) and
- (not self.revs or int(x) <= int(self.revs[0])))]
# now read the full changelists to get the list of file revisions
ui.status(_('collecting p4 changelists\n'))
lastid = None
for change in self.p4changes:
+if startrev and int(change) < int(startrev):
+continue
+if self.revs and int(change) > int(self.revs[0]):
+continue
+
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
d = marshal.load(stdout)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 8 v4] convert: use convert_revision for P4 imports
# HG changeset patch
# User David Soria Parra
# Date 1481694598 28800
# Tue Dec 13 21:49:58 2016 -0800
# Node ID 92b0ad25b4aca99c74d3813ddae163324c1110a7
# Parent 34621390d72ed1e23f30e6c8d0233d46e9d40ef2
convert: use convert_revision for P4 imports
We are using convert_revisions in other importers. In order to unify this
we are also using convert_revision for Perforce in addition to the original
'p4'.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -150,7 +150,7 @@
c = common.commit(author=self.recode(d["user"]),
date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
parents=parents, desc=desc, branch=None,
- extra={"p4": change})
+ extra={"p4": change, "convert_revision": change})
files = []
copies = {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 8 v4] convert: self.heads is a list
# HG changeset patch
# User David Soria Parra
# Date 1481708627 28800
# Wed Dec 14 01:43:47 2016 -0800
# Node ID 09e574c75df2e3547a40f9d9b13464c63235136c
# Parent 506cb0034ebdbe85d2a7174d9214f833b3fed3fb
convert: self.heads is a list
self.heads is used as a list throughout convert and never a dictionary.
Initialize it correctly to a list.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -56,7 +56,7 @@
common.checktool('p4', abort=False)
self.p4changes = {}
-self.heads = {}
+self.heads = []
self.changeset = {}
self.files = {}
self.copies = {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 8 v4] convert: remove unused dictionaries
# HG changeset patch
# User David Soria Parra
# Date 1481708717 28800
# Wed Dec 14 01:45:17 2016 -0800
# Node ID 34621390d72ed1e23f30e6c8d0233d46e9d40ef2
# Parent 09e574c75df2e3547a40f9d9b13464c63235136c
convert: remove unused dictionaries
self.parent, self.lastbranch and self.tags have never been used.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -60,9 +60,6 @@
self.changeset = {}
self.files = {}
self.copies = {}
-self.tags = {}
-self.lastbranch = {}
-self.parent = {}
self.encoding = self.ui.config('convert', 'p4.encoding',
default=convcmd.orig_encoding)
self.depotname = {} # mapping from local name to depot name
@@ -292,7 +289,7 @@
return self.changeset[rev]
def gettags(self):
-return self.tags
+return {}
def getchangedfiles(self, rev, i):
return sorted([x[0] for x in self.files[rev]])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 8 v4] convert: do not provide head revisions if we have no changests to import
# HG changeset patch # User David Soria Parra # Date 1481694598 28800 # Tue Dec 13 21:49:58 2016 -0800 # Node ID c6c26e238617600d0523b44a9043d17f963f9eda # Parent 4b9d267d2a05970d9320b83f8bf29ec99f8ab40f convert: do not provide head revisions if we have no changests to import Don't set a head revision in cases where we have a revmap but no changesets to import, as convertcmd.convert() treats them as heads of to-imported revisions. diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py --- a/hgext/convert/p4.py +++ b/hgext/convert/p4.py @@ -228,7 +228,7 @@ self.copies[change] = copies lastid = change -if lastid: +if lastid and len(self.changeset) > 0: self.heads = [lastid] def getheads(self): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 8 of 8 v4] convert: return commit objects for revisions in the revmap
On Wed, Dec 14, 2016 at 04:53:24PM +, Kostia Balytskyi wrote:
>
>
> On 12/14/16 9:46 AM, David Soria Parra wrote:
> > # HG changeset patch
> > # User David Soria Parra
> > # Date 1481704712 28800
> > # Wed Dec 14 00:38:32 2016 -0800
> > # Node ID eb4a6df3de20deb31e681aa7eba408dc352440c3
> > # Parent 26ec2474ab06fdaf1aca034c54309245ded267b0
> > convert: return commit objects for revisions in the revmap
> >
> > Source revision data that exists in the revmap are ignored when pulling
> > data from Perforce as we consider them already imported. In case where
> > the `convertcmd.convert` algorithm requests a commit object for such
> > a revision we are creating it. This is usually the case for parent of
> > the first imported revision.
> >
> > diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
> > --- a/hgext/convert/p4.py
> > +++ b/hgext/convert/p4.py
> > @@ -321,6 +321,12 @@
> > return marshal.load(stdout)
> >
> > def getcommit(self, rev):
> > +if rev not in self.changeset and rev not in self.revmap:
> > +raise error.Abort(
> > +_("cannot find %s in the revmap or parsed changesets") %
> > rev)
> > +if rev not in self.changeset:
> I think this would be more clear if you used 'if rev in self.revmap'.
> But I think whoever queues it can change it in flight.
that would not be sufficient. We want to fallback to revmap but use
changeset if we have it.
So it would be:
if rev in self.revmap and rev not in self.changeset:
or:
if rev in self.changeset:
return self.changeset[rev]
elif rev in self.revmap:
...
else:
...
I can sent a v5 later.
> > +d = self._fetch_revision(rev)
> > +return self._construct_commit(d, parents=None)
> > return self.changeset[rev]
> >
> > def gettags(self):
> > ___
> > Mercurial-devel mailing list
> > Mercurial-devel@mercurial-scm.org
> > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 8 of 8 v5] convert: return commit objects for revisions in the revmap
# HG changeset patch
# User David Soria Parra
# Date 1481746043 28800
# Wed Dec 14 12:07:23 2016 -0800
# Node ID db65134b6228470e0409136b57f1d4f3bdb44a53
# Parent 26ec2474ab06fdaf1aca034c54309245ded267b0
convert: return commit objects for revisions in the revmap
Source revision data that exists in the revmap are ignored when pulling
data from Perforce as we consider them already imported. In case where
the `convertcmd.convert` algorithm requests a commit object for such
a revision we are creating it. This is usually the case for parent of
the first imported revision.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -321,7 +321,13 @@
return marshal.load(stdout)
def getcommit(self, rev):
-return self.changeset[rev]
+if rev in self.changeset:
+return self.changeset[rev]
+elif rev in self.revmap:
+d = self._fetch_revision(rev)
+return self._construct_commit(d, parents=None)
+raise error.Abort(
+_("cannot find %s in the revmap or parsed changesets") % rev)
def gettags(self):
return {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 8 v5] convert: do not provide head revisions if we have no changests to import
# HG changeset patch # User David Soria Parra # Date 1481694598 28800 # Tue Dec 13 21:49:58 2016 -0800 # Node ID c6c26e238617600d0523b44a9043d17f963f9eda # Parent 4b9d267d2a05970d9320b83f8bf29ec99f8ab40f convert: do not provide head revisions if we have no changests to import Don't set a head revision in cases where we have a revmap but no changesets to import, as convertcmd.convert() treats them as heads of to-imported revisions. diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py --- a/hgext/convert/p4.py +++ b/hgext/convert/p4.py @@ -228,7 +228,7 @@ self.copies[change] = copies lastid = change -if lastid: +if lastid and len(self.changeset) > 0: self.heads = [lastid] def getheads(self): ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 8 v5] convert: use convert_revision for P4 imports
# HG changeset patch
# User David Soria Parra
# Date 1481694598 28800
# Tue Dec 13 21:49:58 2016 -0800
# Node ID 92b0ad25b4aca99c74d3813ddae163324c1110a7
# Parent 34621390d72ed1e23f30e6c8d0233d46e9d40ef2
convert: use convert_revision for P4 imports
We are using convert_revisions in other importers. In order to unify this
we are also using convert_revision for Perforce in addition to the original
'p4'.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -150,7 +150,7 @@
c = common.commit(author=self.recode(d["user"]),
date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
parents=parents, desc=desc, branch=None,
- extra={"p4": change})
+ extra={"p4": change, "convert_revision": change})
files = []
copies = {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 8 v5] convert: allow passing in a revmap
# HG changeset patch
# User David Soria Parra
# Date 1481708757 28800
# Wed Dec 14 01:45:57 2016 -0800
# Node ID 4b9d267d2a05970d9320b83f8bf29ec99f8ab40f
# Parent 92b0ad25b4aca99c74d3813ddae163324c1110a7
convert: allow passing in a revmap
Implement `common.setrevmap` which is used to pass in a file with existing
revision mappings. This functionality is used by `convertcmd.convert` if it
exists and allows implementors such as the p4 converter to make use of an
existing mapping.
We are using the revmap to abort scanning and the repository for more
information
if we already have the revision. This means we are allowing incremental imports
in cases where a revmap is provided.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -55,6 +55,7 @@
common.checktool('p4', abort=False)
+self.revmap = {}
self.p4changes = {}
self.heads = []
self.changeset = {}
@@ -77,6 +78,19 @@
"multiple revisions"))
self._parse(ui, path)
+def setrevmap(self, revmap):
+"""Sets the parsed revmap dictionary.
+
+Revmap stores mappings from a source revision to a target revision.
+It is set in convertcmd.convert and provided by the user as a file
+on the commandline.
+
+Revisions in the map are considered beeing present in the
+repository and ignored during _parse(). This allows for incremental
+imports if a revmap is provided.
+"""
+self.revmap = revmap
+
def _parse_view(self, path):
"Read changes affecting the path"
cmd = 'p4 -G changes -s submitted %s' % util.shellquote(path)
@@ -132,6 +146,10 @@
continue
if self.revs and int(change) > int(self.revs[0]):
continue
+if change in self.revmap:
+# Ignore already present revisions, but set the parent pointer.
+lastid = change
+continue
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 8 v5] convert: don't use long list comprehensions
# HG changeset patch
# User David Soria Parra
# Date 1481694598 28800
# Tue Dec 13 21:49:58 2016 -0800
# Node ID 506cb0034ebdbe85d2a7174d9214f833b3fed3fb
# Parent 76e77076e203fa0b42ff371c7a326580f27245cb
convert: don't use long list comprehensions
We are iterating over p4changes. Make the continue condition more clear
and easier to add new conditions in future patches, by removing the list
comprehension and move the condition into the existing for-loop.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -126,14 +126,16 @@
# handle revision limiting
startrev = self.ui.config('convert', 'p4.startrev', default=0)
-self.p4changes = [x for x in self.p4changes
- if ((not startrev or int(x) >= int(startrev)) and
- (not self.revs or int(x) <= int(self.revs[0])))]
# now read the full changelists to get the list of file revisions
ui.status(_('collecting p4 changelists\n'))
lastid = None
for change in self.p4changes:
+if startrev and int(change) < int(startrev):
+continue
+if self.revs and int(change) > int(self.revs[0]):
+continue
+
cmd = "p4 -G describe -s %s" % change
stdout = util.popen(cmd, mode='rb')
d = marshal.load(stdout)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 8 v5] convert: remove unused dictionaries
# HG changeset patch
# User David Soria Parra
# Date 1481708717 28800
# Wed Dec 14 01:45:17 2016 -0800
# Node ID 34621390d72ed1e23f30e6c8d0233d46e9d40ef2
# Parent 09e574c75df2e3547a40f9d9b13464c63235136c
convert: remove unused dictionaries
self.parent, self.lastbranch and self.tags have never been used.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -60,9 +60,6 @@
self.changeset = {}
self.files = {}
self.copies = {}
-self.tags = {}
-self.lastbranch = {}
-self.parent = {}
self.encoding = self.ui.config('convert', 'p4.encoding',
default=convcmd.orig_encoding)
self.depotname = {} # mapping from local name to depot name
@@ -292,7 +289,7 @@
return self.changeset[rev]
def gettags(self):
-return self.tags
+return {}
def getchangedfiles(self, rev, i):
return sorted([x[0] for x in self.files[rev]])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 8 v5] convert: self.heads is a list
# HG changeset patch
# User David Soria Parra
# Date 1481708627 28800
# Wed Dec 14 01:43:47 2016 -0800
# Node ID 09e574c75df2e3547a40f9d9b13464c63235136c
# Parent 506cb0034ebdbe85d2a7174d9214f833b3fed3fb
convert: self.heads is a list
self.heads is used as a list throughout convert and never a dictionary.
Initialize it correctly to a list.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -56,7 +56,7 @@
common.checktool('p4', abort=False)
self.p4changes = {}
-self.heads = {}
+self.heads = []
self.changeset = {}
self.files = {}
self.copies = {}
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 7 of 8 v5] convert: encapsulate commit data fetching and commit object creation
# HG changeset patch
# User David Soria Parra
# Date 1481694598 28800
# Tue Dec 13 21:49:58 2016 -0800
# Node ID 26ec2474ab06fdaf1aca034c54309245ded267b0
# Parent c6c26e238617600d0523b44a9043d17f963f9eda
convert: encapsulate commit data fetching and commit object creation
Split fetching the `describe` form from Perforce and the commit object creation
into two functions. This allows us to reuse the commit construction for
revisions passed from a revmap.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -151,24 +151,17 @@
lastid = change
continue
-cmd = "p4 -G describe -s %s" % change
-stdout = util.popen(cmd, mode='rb')
-d = marshal.load(stdout)
-desc = self.recode(d.get("desc", ""))
-shortdesc = desc.split("\n", 1)[0]
-t = '%s %s' % (d["change"], repr(shortdesc)[1:-1])
-ui.status(util.ellipsis(t, 80) + '\n')
-
if lastid:
parents = [lastid]
else:
parents = []
-date = (int(d["time"]), 0) # timezone not set
-c = common.commit(author=self.recode(d["user"]),
- date=util.datestr(date, '%Y-%m-%d %H:%M:%S
%1%2'),
- parents=parents, desc=desc, branch=None,
- extra={"p4": change, "convert_revision": change})
+d = self._fetch_revision(change)
+c = self._construct_commit(d, parents)
+
+shortdesc = c.desc.splitlines(True)[0].rstrip('\r\n')
+t = '%s %s' % (c.rev, repr(shortdesc)[1:-1])
+ui.status(util.ellipsis(t, 80) + '\n')
files = []
copies = {}
@@ -303,6 +296,30 @@
raise error.Abort(_("convert from p4 does not support --full"))
return self.files[rev], self.copies[rev], set()
+def _construct_commit(self, obj, parents=None):
+"""
+Constructs a common.commit object from an unmarshalled
+`p4 describe` output
+"""
+desc = self.recode(obj.get("desc", ""))
+shortdesc = desc.split("\n", 1)[0]
+
+date = (int(obj["time"]), 0) # timezone not set
+if parents is None:
+parents = []
+
+return common.commit(author=self.recode(obj["user"]),
+date=util.datestr(date, '%Y-%m-%d %H:%M:%S %1%2'),
+parents=parents, desc=desc, branch=None, rev=obj['change'],
+extra={"p4": obj['change'], "convert_revision": obj['change']})
+
+def _fetch_revision(self, rev):
+"""Return an output of `p4 describe` including author, commit date as
+a dictionary."""
+cmd = "p4 -G describe -s %s" % rev
+stdout = util.popen(cmd, mode='rb')
+return marshal.load(stdout)
+
def getcommit(self, rev):
return self.changeset[rev]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 4] convert: parse perforce data on-demand
# HG changeset patch
# User David Soria Parra
# Date 1482254630 28800
# Tue Dec 20 09:23:50 2016 -0800
# Node ID a047a142f151709f882361916617e762816db619
# Parent 77e1912e2e0817a96633568c400450a9566e7e33
convert: parse perforce data on-demand
We are using read-only attributes that parse the perforce data on
demand. We are reading the data only once whenever an attribute is
requested and use it throughout the import process. This is equivalent
to the previous behavior, but we are avoiding reading from perforce when
we initialize the object, but instead run it during the actual import
process, when the first attribute is requested (usually getheads(), see
`convertcmd.convert`).
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -56,13 +56,8 @@
common.checktool('p4', abort=False)
self.revmap = {}
-self.heads = []
-self.changeset = {}
-self.files = {}
-self.copies = {}
self.encoding = self.ui.config('convert', 'p4.encoding',
default=convcmd.orig_encoding)
-self.depotname = {} # mapping from local name to depot name
self.re_type = re.compile(
"([a-z]+)?(text|binary|symlink|apple|resource|unicode|utf\d+)"
"(\+\w+)?$")
@@ -74,7 +69,6 @@
if revs and len(revs) > 1:
raise error.Abort(_("p4 source does not support specifying "
"multiple revisions"))
-self._parse_once(ui, path)
def setrevmap(self, revmap):
"""Sets the parsed revmap dictionary.
@@ -240,13 +234,29 @@
'depotname': depotname,
}
-def _parse_once(self, ui, path):
-d = self._parse(ui, path)
-self.changeset = d['changeset']
-self.heads = d['heads']
-self.files = d['files']
-self.copies = d['copies']
-self.depotname = d['depotname']
+@util.propertycache
+def _parse_once(self):
+return self._parse(self.ui, self.path)
+
+@util.propertycache
+def copies(self):
+return self._parse_once['copies']
+
+@util.propertycache
+def files(self):
+return self._parse_once['files']
+
+@util.propertycache
+def changeset(self):
+return self._parse_once['changeset']
+
+@util.propertycache
+def heads(self):
+return self._parse_once['heads']
+
+@util.propertycache
+def depotname(self):
+return self._parse_once['depotname']
def getheads(self):
return self.heads
diff --git a/tests/test-convert-p4-filetypes.t
b/tests/test-convert-p4-filetypes.t
--- a/tests/test-convert-p4-filetypes.t
+++ b/tests/test-convert-p4-filetypes.t
@@ -307,11 +307,11 @@
convert
$ hg convert -s p4 $DEPOTPATH dst
initializing destination dst repository
+ scanning source...
reading p4 views
collecting p4 changelists
1 initial
2 keywords
- scanning source...
sorting...
converting...
1 initial
diff --git a/tests/test-convert-p4.t b/tests/test-convert-p4.t
--- a/tests/test-convert-p4.t
+++ b/tests/test-convert-p4.t
@@ -67,12 +67,12 @@
convert
$ hg convert -s p4 $DEPOTPATH dst
initializing destination dst repository
+ scanning source...
reading p4 views
collecting p4 changelists
1 initial
2 change a
3 change b/c
- scanning source...
sorting...
converting...
2 initial
@@ -98,13 +98,10 @@
convert again
$ hg convert -s p4 $DEPOTPATH dst
+ scanning source...
reading p4 views
collecting p4 changelists
- 1 initial
- 2 change a
- 3 change b/c
4 change a b/c
- scanning source...
sorting...
converting...
0 change a b/c
@@ -130,14 +127,10 @@
convert again
$ hg convert -s p4 $DEPOTPATH dst
+ scanning source...
reading p4 views
collecting p4 changelists
- 1 initial
- 2 change a
- 3 change b/c
- 4 change a b/c
5 add d e f
- scanning source...
sorting...
converting...
0 add d e f
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 4] convert: use return value in parse_view() instead of manipulating state
# HG changeset patch
# User David Soria Parra
# Date 1482254630 28800
# Tue Dec 20 09:23:50 2016 -0800
# Node ID 4cb85a7af6deb30abfd4c40ec8e502560150999e
# Parent 392751cbb6c47a43676e324d2f1a4e45f35e624b
convert: use return value in parse_view() instead of manipulating state
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -95,10 +95,12 @@
"Read changes affecting the path"
cmd = 'p4 -G changes -s submitted %s' % util.shellquote(path)
stdout = util.popen(cmd, mode='rb')
+p4changes = {}
for d in loaditer(stdout):
c = d.get("change", None)
if c:
-self.p4changes[c] = True
+p4changes[c] = True
+return p4changes
def _parse(self, ui, path):
"Prepare list of P4 filenames and revisions to import"
@@ -106,7 +108,7 @@
# read client spec or view
if "/" in path:
-self._parse_view(path)
+self.p4changes.update(self._parse_view(path))
if path.startswith("//") and path.endswith("/..."):
views = {path[:-3]:""}
else:
@@ -119,7 +121,7 @@
for client in clientspec:
if client.startswith("View"):
sview, cview = clientspec[client].split()
-self._parse_view(sview)
+self.p4changes.update(self._parse_view(sview))
if sview.endswith("...") and cview.endswith("..."):
sview = sview[:-3]
cview = cview[:-3]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 2 of 4] convert: move localname state to function scope
# HG changeset patch
# User David Soria Parra
# Date 1482254630 28800
# Tue Dec 20 09:23:50 2016 -0800
# Node ID 6f7154489117ac5cc9f0f876e922fef5c44803d8
# Parent 4cb85a7af6deb30abfd4c40ec8e502560150999e
convert: move localname state to function scope
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -64,7 +64,6 @@
self.encoding = self.ui.config('convert', 'p4.encoding',
default=convcmd.orig_encoding)
self.depotname = {} # mapping from local name to depot name
-self.localname = {} # mapping from depot name to local name
self.re_type = re.compile(
"([a-z]+)?(text|binary|symlink|apple|resource|unicode|utf\d+)"
"(\+\w+)?$")
@@ -168,6 +167,7 @@
files = []
copies = {}
copiedfiles = []
+localname = {}
i = 0
while ("depotFile%d" % i) in d and ("rev%d" % i) in d:
oldname = d["depotFile%d" % i]
@@ -181,7 +181,7 @@
self.depotname[filename] = oldname
if (d.get("action%d" % i) == "move/add"):
copiedfiles.append(filename)
-self.localname[oldname] = filename
+localname[oldname] = filename
i += 1
# Collect information about copied files
@@ -208,8 +208,8 @@
j += 1
i += 1
-if copiedoldname and copiedoldname in self.localname:
-copiedfilename = self.localname[copiedoldname]
+if copiedoldname and copiedoldname in localname:
+copiedfilename = localname[copiedoldname]
break
if copiedfilename:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 4] convert: return calculated values from parse() instead of manpulating state
# HG changeset patch
# User David Soria Parra
# Date 1482254630 28800
# Tue Dec 20 09:23:50 2016 -0800
# Node ID 77e1912e2e0817a96633568c400450a9566e7e33
# Parent 6f7154489117ac5cc9f0f876e922fef5c44803d8
convert: return calculated values from parse() instead of manpulating state
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -56,7 +56,6 @@
common.checktool('p4', abort=False)
self.revmap = {}
-self.p4changes = {}
self.heads = []
self.changeset = {}
self.files = {}
@@ -75,7 +74,7 @@
if revs and len(revs) > 1:
raise error.Abort(_("p4 source does not support specifying "
"multiple revisions"))
-self._parse(ui, path)
+self._parse_once(ui, path)
def setrevmap(self, revmap):
"""Sets the parsed revmap dictionary.
@@ -103,11 +102,19 @@
def _parse(self, ui, path):
"Prepare list of P4 filenames and revisions to import"
+p4changes = {}
+changeset = {}
+files_map = {}
+copies_map = {}
+localname = {}
+depotname = {}
+heads = []
+
ui.status(_('reading p4 views\n'))
# read client spec or view
if "/" in path:
-self.p4changes.update(self._parse_view(path))
+p4changes.update(self._parse_view(path))
if path.startswith("//") and path.endswith("/..."):
views = {path[:-3]:""}
else:
@@ -120,7 +127,7 @@
for client in clientspec:
if client.startswith("View"):
sview, cview = clientspec[client].split()
-self.p4changes.update(self._parse_view(sview))
+p4changes.update(self._parse_view(sview))
if sview.endswith("...") and cview.endswith("..."):
sview = sview[:-3]
cview = cview[:-3]
@@ -129,8 +136,8 @@
views[sview] = cview
# list of changes that affect our source files
-self.p4changes = self.p4changes.keys()
-self.p4changes.sort(key=int)
+p4changes = p4changes.keys()
+p4changes.sort(key=int)
# list with depot pathnames, longest first
vieworder = views.keys()
@@ -142,7 +149,7 @@
# now read the full changelists to get the list of file revisions
ui.status(_('collecting p4 changelists\n'))
lastid = None
-for change in self.p4changes:
+for change in p4changes:
if startrev and int(change) < int(startrev):
continue
if self.revs and int(change) > int(self.revs[0]):
@@ -167,7 +174,6 @@
files = []
copies = {}
copiedfiles = []
-localname = {}
i = 0
while ("depotFile%d" % i) in d and ("rev%d" % i) in d:
oldname = d["depotFile%d" % i]
@@ -178,7 +184,7 @@
break
if filename:
files.append((filename, d["rev%d" % i]))
-self.depotname[filename] = oldname
+depotname[filename] = oldname
if (d.get("action%d" % i) == "move/add"):
copiedfiles.append(filename)
localname[oldname] = filename
@@ -186,7 +192,7 @@
# Collect information about copied files
for filename in copiedfiles:
-oldname = self.depotname[filename]
+oldname = depotname[filename]
flcmd = 'p4 -G filelog %s' \
% util.shellquote(oldname)
@@ -218,13 +224,29 @@
ui.warn(_("cannot find source for copied file: %s@%s\n")
% (filename, change))
-self.changeset[change] = c
-self.files[change] = files
-self.copies[change] = copies
+changeset[change] = c
+files_map[change] = files
+copies_map[change] = copies
lastid = change
-if lastid and len(self.changeset) > 0:
-self.heads = [lastid]
+if lastid and len(changeset) > 0:
+heads = [lastid]
+
+return {
+'changeset': changeset,
+'files': files_map,
+'copies': copies_map,
+'heads': heads,
+'depotname': depotname,
+}
+
+def _parse_once(self, ui, path):
+d = self._parse(ui, path)
+self.changeset = d['changeset']
+self.heads = d[
[PATCH RFC] check-commit: add magic string to bypass check-commit
# HG changeset patch # User David Soria Parra # Date 1482293000 28800 # Tue Dec 20 20:03:20 2016 -0800 # Node ID ff1014e78d5a3084eda379ec51b1c0a1fc4f5b2a # Parent 11deeb8980886f1bd45c56e8ac3ab662b4fdb252 check-commit: add magic string to bypass check-commit Allow bypassing check-commit runs by specifying '# no-check-commit' at the beginning of a line. This should be avoided but is useful for upstream imports such as pywatchman which will cause check-code to fail otherwise. diff --git a/contrib/check-commit b/contrib/check-commit --- a/contrib/check-commit +++ b/contrib/check-commit @@ -25,6 +25,7 @@ afterheader = commitheader + r"(?!#)" beforepatch = afterheader + r"(?!\n(?!@@))" +bypass = r"^# no-check-commit" errors = [ (beforepatch + r".*[(]bc[)]", "(BC) needs to be uppercase"), (beforepatch + r".*[(]issue \d\d\d", @@ -59,6 +60,9 @@ exitcode = 0 printed = node is None hits = [] +if re.search(bypass, commit, re.MULTILINE): +return 0 + for exp, msg in errors: for m in re.finditer(exp, commit): end = m.end() ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] tests: exclude bundled pywatchman from check-code test
# HG changeset patch # User David Soria Parra # Date 1482294521 28800 # Tue Dec 20 20:28:41 2016 -0800 # Node ID 26a5b7f91ec50d20782816b83841bd8087478eb0 # Parent b090cdf0e161af0ac6768cb404cc0e51bb0baf00 tests: exclude bundled pywatchman from check-code test pywatchman is imported from upstream and therefore fails to pass linting. We have added 'no-check-code' manually to every file in the past. This is cumbersome and modifies upstream sources. diff --git a/tests/test-check-code.t b/tests/test-check-code.t --- a/tests/test-check-code.t +++ b/tests/test-check-code.t @@ -7,13 +7,8 @@ New errors are not allowed. Warnings are strongly discouraged. (The writing "no-che?k-code" is for not skipping this file when checking.) - $ hg locate -X contrib/python-zstandard | sed 's-\\-/-g' | - > xargs "$check_code" --warnings --per-file=0 || false - Skipping hgext/fsmonitor/pywatchman/__init__.py it has no-che?k-code (glob) - Skipping hgext/fsmonitor/pywatchman/bser.c it has no-che?k-code (glob) - Skipping hgext/fsmonitor/pywatchman/capabilities.py it has no-che?k-code (glob) - Skipping hgext/fsmonitor/pywatchman/msc_stdint.h it has no-che?k-code (glob) - Skipping hgext/fsmonitor/pywatchman/pybser.py it has no-che?k-code (glob) + $ hg locate -X contrib/python-zstandard -X hgext/fsmonitor/pywatchman | + > sed 's-\\-/-g' | xargs "$check_code" --warnings --per-file=0 || false Skipping i18n/polib.py it has no-che?k-code (glob) Skipping mercurial/httpclient/__init__.py it has no-che?k-code (glob) Skipping mercurial/httpclient/_readers.py it has no-che?k-code (glob) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] tests: add magic string to bypass check-commit
# HG changeset patch
# User David Soria Parra
# Date 1482356655 28800
# Wed Dec 21 13:44:15 2016 -0800
# Node ID dba02153520bf599a723a6641a8040c7966d2226
# Parent 392751cbb6c47a43676e324d2f1a4e45f35e624b
tests: add magic string to bypass check-commit
Allow bypassing test-check-commit runs by specifying '# no-check-commit'
in the description. This should be avoided but is useful for upstream
imports such as pywatchman which will cause check-code to fail otherwise.
diff --git a/tests/test-check-commit.t b/tests/test-check-commit.t
--- a/tests/test-check-commit.t
+++ b/tests/test-check-commit.t
@@ -8,7 +8,7 @@
$ cd $TESTDIR/..
- $ for node in `hg log --rev 'not public() and ::.' --template
'{node|short}\n'`; do
+ $ for node in `hg log --rev 'not public() and ::. and not desc("#
no-check-commit")' --template '{node|short}\n'`; do
>hg export $node | contrib/check-commit > ${TESTTMP}/check-commit.out
>if [ $? -ne 0 ]; then
>echo "Revision $node does not comply with rules"
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH v2] tests: add magic string to bypass check-commit
# HG changeset patch
# User David Soria Parra
# Date 1482356655 28800
# Wed Dec 21 13:44:15 2016 -0800
# Node ID dba02153520bf599a723a6641a8040c7966d2226
# Parent 392751cbb6c47a43676e324d2f1a4e45f35e624b
tests: add magic string to bypass check-commit
Allow bypassing test-check-commit runs by specifying '# no-check-commit'
in the description. This should be avoided but is useful for upstream
imports such as pywatchman which will cause check-code to fail otherwise.
diff --git a/tests/test-check-commit.t b/tests/test-check-commit.t
--- a/tests/test-check-commit.t
+++ b/tests/test-check-commit.t
@@ -8,7 +8,7 @@
$ cd $TESTDIR/..
- $ for node in `hg log --rev 'not public() and ::.' --template
'{node|short}\n'`; do
+ $ for node in `hg log --rev 'not public() and ::. and not desc("#
no-check-commit")' --template '{node|short}\n'`; do
>hg export $node | contrib/check-commit > ${TESTTMP}/check-commit.out
>if [ $? -ne 0 ]; then
>echo "Revision $node does not comply with rules"
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 3 website] Makefile: update makefile to not mention blatter
On Fri, Dec 23, 2016 at 11:08:11AM -0500, Augie Fackler wrote: > @@ -1,7 +1,4 @@ > -all:: build > - > -build: > - python ../blatter/blatter/__init__.py blat > +all:: serve > > deploy-ssh: > ssh selenic 'cd /home/hg/www && hg pull -u' we proabbly want to get rid of that too ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 3 of 3 website] donate: new page to route donations to conservancy
On Fri, Dec 23, 2016 at 11:08:12AM -0500, Augie Fackler wrote: > # HG changeset patch > # User Augie Fackler > # Date 1482509162 18000 > # Fri Dec 23 11:06:02 2016 -0500 > # Node ID f48d84ce78aed4a3d063b08da0f7178028192501 > # Parent 4d7de32b44efc1a385466e9bbafe4ac5a2177c34 > donate: new page to route donations to conservancy if it matters, series LGTM. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] pager: exit cleanly on SIGPIPE (BC)
On Wed, Feb 08, 2017 at 07:47:39AM -0800, Simon Farnsworth wrote: > # HG changeset patch > # User Simon Farnsworth > # Date 1486568650 28800 > # Wed Feb 08 07:44:10 2017 -0800 > # Node ID d50cda2a403786836d1f0d5c99401599dc4f43ec > # Parent 1f51b4658f21bbb797e922d155c1046eddccf91d > pager: exit cleanly on SIGPIPE (BC) > LGTM ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 04 of 10 ipv6 V2] tinyproxy: use IPv6 if HGIPV6 is set to 1
On Fri, Feb 17, 2017 at 10:51:18AM -0800, Jun Wu wrote: > # HG changeset patch > # User Jun Wu > # Date 1487228025 28800 > # Wed Feb 15 22:53:45 2017 -0800 > # Node ID c34ad0f0fc375de9e2d96d17ddcb644d9054d154 > # Parent 5e76ef1698fde7a47fa78003beac71324793d051 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > c34ad0f0fc37 > tinyproxy: use IPv6 if HGIPV6 is set to 1 > > This patch makes tinyproxy.py work in IPv6 mode if HGIPV6 is set to 1. > > This will make test-http-proxy.t pass on IPv6 machines. > I might be missing something here as I am a bit rusty on Mercurial internals, I feel we are not doing the right thing here. I think the series is not the correct way to approach the issue. When IPv6 and IPv4 is available we should bind to both sockets and have the client doing proper destination address selection according to rfc6724. According to the rfc preferring IPv6 over IPv4 should be the default and I feel introducing a specific switch does not test the correct behavior. In order to obtain the available localhost addresses we should use socket.getaddrinfo() to obtain a list of available v4/v6 addresses and bind on both. This should be doable without using HGIPV6 in tinyproxy and dumbhttp. We can use this list to glob() 127.0.0.1 correctly, even in the case of FreeBSD jails and don't rely on hardcoded ::1/127.0.0.1 addresses. This should allow us to use IPv6 on a test host correctly when avaialble and fallback to IPv4 graciously. I see why we want to introduce switches in tests for v4/v6 only, but I feel that the current implementation does not correctly test default addr selection (which might be part of causing the flakeyness). /D ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 04 of 10 ipv6 V2] tinyproxy: use IPv6 if HGIPV6 is set to 1
On Mon, Feb 20, 2017 at 07:03:36PM -0800, Jun Wu wrote: > > Thanks for pointing out the RFC. > > The V1 series was ipv6 by default. But buildbot was not happy with all > changes. In order to make all changesets green, I have to temporarily > disable V6 handling, therefore V2 and IPv4 by default. > > Since the changes were committed, we could change it to V6 by default now. > Maybe add a flag like --netaddr-family=auto|ipv6|ipv4 That would be an okay idea > > In order to obtain the available localhost addresses we should use > > socket.getaddrinfo() to obtain a list of available v4/v6 addresses and > > bind on both. This should be doable without using HGIPV6 in tinyproxy > As long as "hg serve" sticks to one address family, I don't think it's > useful to make tinyproxy bind on both. > "HGIPV6" is a way to provide hgrc's "web.ipv6" for non-hgrc-aware programs. > So if you'd like to remove "HGIPV6", maybe try remove "web.ipv6" first. > > > and dumbhttp. We can use this list to glob() 127.0.0.1 correctly, even in > > the > > case of FreeBSD jails and don't rely on hardcoded ::1/127.0.0.1 addresses. > > This > > This looks like a separate issue. Note that "getaddrinfo()" just reads > "/etc/hosts", instead of the real interfaces. The list returned by > "getaddrinfo()" cannot be trusted, if we want real interface info. You are right. In this case it should be determining LOCALIP: 1. try to use getaddrinfo() of localhost 2. if unavailable, use ::1/127.0.0.1 loopback 3. otherwise abort > > should allow us to use IPv6 on a test host correctly when avaialble and > > fallback > > to IPv4 graciously. I see why we want to introduce switches in tests for > > v4/v6 > > only, but I feel that the current implementation does not correctly test > > default > > addr selection (which might be part of causing the flakeyness). > > Previously, there is no IPv6 support at all. The system has to have a > working IPv4 configuration to run tests. And the port check logic only > checked "localhost", which is simply wrong since "hg serve" binds on all > interfaces. > > So in weird situations, the tests will just fail, like: > > - All network interfaces (incl. "lo") do not have IPv4 addresses. > - The "localhost" hostname does not resolve to an IPv4 address. > - The port being used by "hg serve" is free on "lo", but not all > interfaces. > > My patches should have made tests just work instead of failing, in the above > situations. I think my point is that the correct thing to do, is to bind on v6/v4 if available at all times (that includes hg serve, i look into this) instead of make it a switch, as that would defeat the address resolution rfc. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 1 of 6 RFC] ui: refactoring handling of trusted, user and overlay
# HG changeset patch
# User David Soria Parra
# Date 1489349204 25200
# Sun Mar 12 13:06:44 2017 -0700
# Node ID 99514a82d5b23c75bd6da38e522acfd14a618c14
# Parent 1c3352d7eaf24533ad52d4b8a024211e9189fb0b
ui: refactoring handling of trusted, user and overlay
We are using obscure names such as _ocfg for overlay-config in the UI. This is
sometimes confusing and not very flexible. We are moving this into a dictionary
now that has a specific ordering in which we would apply multiple layers of
configuration. At the moment this is not needed as we always pick either
user-config or trusted-config and overlay it, but it gets us a good machinery to
add a defaults layer for ui.compat.
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -146,9 +146,7 @@
self._bufferapplylabels = None
self.quiet = self.verbose = self.debugflag = self.tracebackflag = False
self._reportuntrusted = True
-self._ocfg = config.config() # overlay
-self._tcfg = config.config() # trusted
-self._ucfg = config.config() # untrusted
+self._cfg = self.cfg()
self._trustusers = set()
self._trustgroups = set()
self.callhooks = True
@@ -167,10 +165,6 @@
self.fin = src.fin
self.pageractive = src.pageractive
self._disablepager = src._disablepager
-
-self._tcfg = src._tcfg.copy()
-self._ucfg = src._ucfg.copy()
-self._ocfg = src._ocfg.copy()
self._trustusers = src._trustusers.copy()
self._trustgroups = src._trustgroups.copy()
self.environ = src.environ
@@ -179,6 +173,8 @@
self._colormode = src._colormode
self._terminfoparams = src._terminfoparams.copy()
self._styles = src._styles.copy()
+for k in self._cfg.keys():
+self._cfg[k] = src._cfg[k].copy()
self.fixconfig()
@@ -296,21 +292,28 @@
del cfg['templatealias'][k]
if trusted:
-self._tcfg.update(cfg)
-self._tcfg.update(self._ocfg)
-self._ucfg.update(cfg)
-self._ucfg.update(self._ocfg)
+self._cfg['trusted'].update(cfg)
+self._cfg['trusted'].update(self._cfg['overlay'])
+self._cfg['user'].update(cfg)
+self._cfg['user'].update(self._cfg['overlay'])
if root is None:
root = os.path.expanduser('~')
self.fixconfig(root=root)
+def cfg(self):
+# Ordered in ascneding order of preference.
+return util.sortdict(
+[('user', config.config()),
+('trusted', config.config()),
+('overlay', config.config())])
+
def fixconfig(self, root=None, section=None):
if section in (None, 'paths'):
# expand vars and ~
# translate paths relative to root (or home) into absolute paths
root = root or pycompat.getcwd()
-for c in self._tcfg, self._ucfg, self._ocfg:
+for c in self._cfg.values():
for n, p in c.items('paths'):
# Ignore sub-options.
if ':' in n:
@@ -345,21 +348,22 @@
self._trustgroups.update(self.configlist('trusted', 'groups'))
def backupconfig(self, section, item):
-return (self._ocfg.backup(section, item),
-self._tcfg.backup(section, item),
-self._ucfg.backup(section, item),)
+return {k: cfg.backup(section, item) for k, cfg in self._cfg.items()}
+
def restoreconfig(self, data):
-self._ocfg.restore(data[0])
-self._tcfg.restore(data[1])
-self._ucfg.restore(data[2])
+for k, d in data.items():
+self._cfg[k].restore(d)
def setconfig(self, section, name, value, source=''):
-for cfg in (self._ocfg, self._tcfg, self._ucfg):
+for cfg in self._cfg.values():
cfg.set(section, name, value, source)
self.fixconfig(section=section)
def _data(self, untrusted):
-return untrusted and self._ucfg or self._tcfg
+if untrusted:
+return self._cfg['user']
+else:
+return self._cfg['trusted']
def configsource(self, section, name, untrusted=False):
return self._data(untrusted).source(section, name)
@@ -380,7 +384,7 @@
if self.debugflag and not untrusted and self._reportuntrusted:
for n in alternates:
-uvalue = self._ucfg.get(section, n)
+uvalue = self._cfg['user'].get(section, n)
if uvalue is not None and uvalue != value:
self.debug("ignoring untrusted configuration option &
[PATCH 2 of 6 RFC] ui: add a defaults layer to the config
# HG changeset patch
# User David Soria Parra
# Date 1489349206 25200
# Sun Mar 12 13:06:46 2017 -0700
# Node ID 084a30b54810d05c8b44d22fcab2ef6f783a9f7c
# Parent 99514a82d5b23c75bd6da38e522acfd14a618c14
ui: add a defaults layer to the config
Add a defaults layer to the config that has the least precedence. This will
allow us to load defaults based on compatibility settings without interferring
with trusted or user.
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -304,7 +304,8 @@
def cfg(self):
# Ordered in ascneding order of preference.
return util.sortdict(
-[('user', config.config()),
+[('defaults', config.config()),
+('user', config.config()),
('trusted', config.config()),
('overlay', config.config())])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 5 of 6 RFC] help: document compat mode
# HG changeset patch # User David Soria Parra # Date 1489349206 25200 # Sun Mar 12 13:06:46 2017 -0700 # Node ID 2dfa752cf0751010867944aa707bd7a722d3ea87 # Parent 8efd04667ba8d2939a937f549e803bc3b56f70a7 help: document compat mode diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt --- a/mercurial/help/config.txt +++ b/mercurial/help/config.txt @@ -1837,6 +1837,13 @@ changes, abort the commit. (default: False) +``compat`` +String: Compatibility mode for the ui. Possible values are compat +or latest (default: compat). ``compat`` provides backwards compatible +ui behavior. ``latest`` enables additional config settings aiming to +improve user experience. (see ``HGPLAIN`` for compatibility modes in +non-interactive environments) + ``debug`` Print debugging information. (default: False) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 3 of 6 RFC] ui: don't return certain layers when they are supposed to be displayed
# HG changeset patch
# User David Soria Parra
# Date 1489349206 25200
# Sun Mar 12 13:06:46 2017 -0700
# Node ID ef5ce5325596fe5fef014a320abae0f0d5980f3d
# Parent 084a30b54810d05c8b44d22fcab2ef6f783a9f7c
ui: don't return certain layers when they are supposed to be displayed
This is a hack to distinguish visibility for a certain configuration. As we have
layers in place, we can give them a visibility, allowing us to have internal
settings switched on without them showing up in `hg showconfig`. This is useful
for extension to set a list of default flags to trigger behavior.
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -360,11 +360,15 @@
cfg.set(section, name, value, source)
self.fixconfig(section=section)
-def _data(self, untrusted):
+def _data(self, untrusted, includeinternal=True):
+res = {}
+if includeinternal:
+res = self._cfg['defaults']
if untrusted:
-return self._cfg['user']
+res.update(self._cfg['user'])
else:
-return self._cfg['trusted']
+res.update(self._cfg['trusted'])
+return res
def configsource(self, section, name, untrusted=False):
return self._data(untrusted).source(section, name)
@@ -670,7 +674,7 @@
return items
def walkconfig(self, untrusted=False):
-cfg = self._data(untrusted)
+cfg = self._data(untrusted, includeinternal=False)
for section in cfg.sections():
for name, value in self.configitems(section, untrusted):
yield section, name, value
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 4 of 6 RFC] compat: module to handle different ui.compat settings
# HG changeset patch
# User David Soria Parra
# Date 1489349206 25200
# Sun Mar 12 13:06:46 2017 -0700
# Node ID 8efd04667ba8d2939a937f549e803bc3b56f70a7
# Parent ef5ce5325596fe5fef014a320abae0f0d5980f3d
compat: module to handle different ui.compat settings
We are introducing ui.compat. It defaults to 'compat' which means Mercurial is
supposed to behave backwards compatible. At the moment it supports another mode
called 'latest' which can enable bc-breaking configurations to change the
default behavior of commands. The layer provides an ordered list of
compatibility levels and returns a combined list of configurations up to a given
compatibility level. For example, given settings 'compat', 'hg4.2',
'hg4.3', 'latest', a request for 'hg4.3' will return the combined settings for
'compat', 'hg4.2' and 'hg4.3' with later levels overwrriten existing
configurations.
diff --git a/mercurial/compat.py b/mercurial/compat.py
new file mode 100644
--- /dev/null
+++ b/mercurial/compat.py
@@ -0,0 +1,48 @@
+# compat.py - handlign compatibility settings
+#
+# Copyright 2005-2017 Mercurial Steering Committee
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+from __future__ import absolute_import
+
+import collections
+from . import (
+util,
+)
+
+# The initialization msut be done with a list and not a dict, as a list
+# is sorted while a dictionary is not.
+COMPAT = util.sortdict([
+('latest', {
+'diff': {
+'git': 'True',
+'showfunc': 'True',
+},
+'ui': {
+'color': 'auto',
+'interface': 'curses',
+},
+})],
+)
+
+def modernize(ui):
+compats = compatlevel(ui)
+for section, d in compats.items():
+for key, value in d.items():
+ui._cfg['defaults'].set(section, key, value)
+
+def compatlevel(ui):
+if ui.plain('compat'):
+requested = 'compat'
+else:
+requested = ui.config('ui', 'compat', 'compat')
+
+result = {}
+for level, configs in COMPAT.items():
+result.update(configs)
+if level == requested:
+# defaults is sorted. We can abort once we reached
+# the requested level.
+break
+return result
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 6 of 6 RFC] dispatch: modernize the ui after finalizing config loading
# HG changeset patch
# User David Soria Parra
# Date 1489349206 25200
# Sun Mar 12 13:06:46 2017 -0700
# Node ID 9eca95dec659b3ff37608fd257d888520599124d
# Parent 2dfa752cf0751010867944aa707bd7a722d3ea87
dispatch: modernize the ui after finalizing config loading
We have to carefuly load the defaults layer in the ui object after we have read
configurations and set --config, as we are modifying configurations based on a
config settings. In our case we have to ensure that 'ui.compat=' can be set from
--config or any hgrc, but when we detect it have to add our setting as early as
possible. Therefore we choose dispatch to add it, right after we finalize
--config parsing.
diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -26,6 +26,7 @@
cmdutil,
color,
commands,
+compat,
debugcommands,
demandimport,
encoding,
@@ -178,6 +179,7 @@
for sec, name, val in cfgs:
req.repo.ui.setconfig(sec, name, val, source='--config')
+compat.modernize(ui)
# developer config: ui.debugger
debugger = ui.config("ui", "debugger")
debugmod = pdb
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 09 of 10 RFC v2] compat: add ui.color=auto to latest
# HG changeset patch
# User David Soria Parra
# Date 1489358241 25200
# Sun Mar 12 15:37:21 2017 -0700
# Node ID b114907de6ce0f3d07c7be37e6570cc0a9456df6
# Parent 8d4b79f7c6012aa1595fa3fbe184bc6849e3d81c
compat: add ui.color=auto to latest
diff --git a/mercurial/compat.py b/mercurial/compat.py
--- a/mercurial/compat.py
+++ b/mercurial/compat.py
@@ -20,6 +20,9 @@
'git': 'True',
'showfunc': 'True',
},
+'ui': {
+'color': 'auto',
+},
})],
)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 04 of 10 RFC v2] compat: module to handle different ui.compat settings
# HG changeset patch
# User David Soria Parra
# Date 1489349206 25200
# Sun Mar 12 13:06:46 2017 -0700
# Node ID 0986fb7be0f584f5169f93377f9d013f87381ea7
# Parent ef5ce5325596fe5fef014a320abae0f0d5980f3d
compat: module to handle different ui.compat settings
We are introducing ui.compat. It defaults to 'compat' which means Mercurial is
supposed to behave backwards compatible. At the moment it supports another mode
called 'latest' which can enable bc-breaking configurations to change the
default behavior of commands. The layer provides an ordered list of
compatibility levels and returns a combined list of configurations up to a given
compatibility level. For example, given settings 'compat', 'hg4.2',
'hg4.3', 'latest', a request for 'hg4.3' will return the combined settings for
'compat', 'hg4.2' and 'hg4.3' with later levels overwrriten existing
configurations.
diff --git a/mercurial/compat.py b/mercurial/compat.py
new file mode 100644
--- /dev/null
+++ b/mercurial/compat.py
@@ -0,0 +1,40 @@
+# compat.py - handlign compatibility settings
+#
+# Copyright 2005-2017 Mercurial Steering Committee
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+from __future__ import absolute_import
+
+import collections
+from . import (
+util,
+)
+
+# The initialization msut be done with a list and not a dict, as a list
+# is sorted while a dictionary is not.
+COMPAT = util.sortdict([
+('compat', {}),
+('latest', {})],
+)
+
+def modernize(ui):
+compats = compatlevel(ui)
+for section, d in compats.items():
+for key, value in d.items():
+ui._cfg['defaults'].set(section, key, value)
+
+def compatlevel(ui):
+if ui.plain('compat'):
+requested = 'compat'
+else:
+requested = ui.config('ui', 'compat', 'compat')
+
+result = {}
+for level, configs in COMPAT.items():
+result.update(configs)
+if level == requested:
+# defaults is sorted. We can abort once we reached
+# the requested level.
+break
+return result
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 07 of 10 RFC v2] compat: add diff.git=True to latest
# HG changeset patch
# User David Soria Parra
# Date 1489358214 25200
# Sun Mar 12 15:36:54 2017 -0700
# Node ID 7256c598a7f493bcc999259a4cbd88d86077746d
# Parent 63065d4cb13dc3acb9177473896fc3c9c9c7993b
compat: add diff.git=True to latest
diff --git a/mercurial/compat.py b/mercurial/compat.py
--- a/mercurial/compat.py
+++ b/mercurial/compat.py
@@ -15,7 +15,11 @@
# is sorted while a dictionary is not.
COMPAT = util.sortdict([
('compat', {}),
-('latest', {})],
+('latest', {
+'diff': {
+'git': 'True',
+},
+})],
)
def modernize(ui):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 01 of 10 RFC v2] ui: refactoring handling of trusted, user and overlay
# HG changeset patch
# User David Soria Parra
# Date 1489349204 25200
# Sun Mar 12 13:06:44 2017 -0700
# Node ID 99514a82d5b23c75bd6da38e522acfd14a618c14
# Parent 1c3352d7eaf24533ad52d4b8a024211e9189fb0b
ui: refactoring handling of trusted, user and overlay
We are using obscure names such as _ocfg for overlay-config in the UI. This is
sometimes confusing and not very flexible. We are moving this into a dictionary
now that has a specific ordering in which we would apply multiple layers of
configuration. At the moment this is not needed as we always pick either
user-config or trusted-config and overlay it, but it gets us a good machinery to
add a defaults layer for ui.compat.
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -146,9 +146,7 @@
self._bufferapplylabels = None
self.quiet = self.verbose = self.debugflag = self.tracebackflag = False
self._reportuntrusted = True
-self._ocfg = config.config() # overlay
-self._tcfg = config.config() # trusted
-self._ucfg = config.config() # untrusted
+self._cfg = self.cfg()
self._trustusers = set()
self._trustgroups = set()
self.callhooks = True
@@ -167,10 +165,6 @@
self.fin = src.fin
self.pageractive = src.pageractive
self._disablepager = src._disablepager
-
-self._tcfg = src._tcfg.copy()
-self._ucfg = src._ucfg.copy()
-self._ocfg = src._ocfg.copy()
self._trustusers = src._trustusers.copy()
self._trustgroups = src._trustgroups.copy()
self.environ = src.environ
@@ -179,6 +173,8 @@
self._colormode = src._colormode
self._terminfoparams = src._terminfoparams.copy()
self._styles = src._styles.copy()
+for k in self._cfg.keys():
+self._cfg[k] = src._cfg[k].copy()
self.fixconfig()
@@ -296,21 +292,28 @@
del cfg['templatealias'][k]
if trusted:
-self._tcfg.update(cfg)
-self._tcfg.update(self._ocfg)
-self._ucfg.update(cfg)
-self._ucfg.update(self._ocfg)
+self._cfg['trusted'].update(cfg)
+self._cfg['trusted'].update(self._cfg['overlay'])
+self._cfg['user'].update(cfg)
+self._cfg['user'].update(self._cfg['overlay'])
if root is None:
root = os.path.expanduser('~')
self.fixconfig(root=root)
+def cfg(self):
+# Ordered in ascneding order of preference.
+return util.sortdict(
+[('user', config.config()),
+('trusted', config.config()),
+('overlay', config.config())])
+
def fixconfig(self, root=None, section=None):
if section in (None, 'paths'):
# expand vars and ~
# translate paths relative to root (or home) into absolute paths
root = root or pycompat.getcwd()
-for c in self._tcfg, self._ucfg, self._ocfg:
+for c in self._cfg.values():
for n, p in c.items('paths'):
# Ignore sub-options.
if ':' in n:
@@ -345,21 +348,22 @@
self._trustgroups.update(self.configlist('trusted', 'groups'))
def backupconfig(self, section, item):
-return (self._ocfg.backup(section, item),
-self._tcfg.backup(section, item),
-self._ucfg.backup(section, item),)
+return {k: cfg.backup(section, item) for k, cfg in self._cfg.items()}
+
def restoreconfig(self, data):
-self._ocfg.restore(data[0])
-self._tcfg.restore(data[1])
-self._ucfg.restore(data[2])
+for k, d in data.items():
+self._cfg[k].restore(d)
def setconfig(self, section, name, value, source=''):
-for cfg in (self._ocfg, self._tcfg, self._ucfg):
+for cfg in self._cfg.values():
cfg.set(section, name, value, source)
self.fixconfig(section=section)
def _data(self, untrusted):
-return untrusted and self._ucfg or self._tcfg
+if untrusted:
+return self._cfg['user']
+else:
+return self._cfg['trusted']
def configsource(self, section, name, untrusted=False):
return self._data(untrusted).source(section, name)
@@ -380,7 +384,7 @@
if self.debugflag and not untrusted and self._reportuntrusted:
for n in alternates:
-uvalue = self._ucfg.get(section, n)
+uvalue = self._cfg['user'].get(section, n)
if uvalue is not None and uvalue != value:
self.debug("ignoring untrusted configuration option &
[PATCH 06 of 10 RFC v2] dispatch: modernize the ui after finalizing config loading
# HG changeset patch
# User David Soria Parra
# Date 1489349206 25200
# Sun Mar 12 13:06:46 2017 -0700
# Node ID 63065d4cb13dc3acb9177473896fc3c9c9c7993b
# Parent d402acdf7c439dc369b6ab3a6888078a2895978a
dispatch: modernize the ui after finalizing config loading
We have to carefuly load the defaults layer in the ui object after we have read
configurations and set --config, as we are modifying configurations based on a
config settings. In our case we have to ensure that 'ui.compat=' can be set from
--config or any hgrc, but when we detect it have to add our setting as early as
possible. Therefore we choose dispatch to add it, right after we finalize
--config parsing.
diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -26,6 +26,7 @@
cmdutil,
color,
commands,
+compat,
debugcommands,
demandimport,
encoding,
@@ -178,6 +179,7 @@
for sec, name, val in cfgs:
req.repo.ui.setconfig(sec, name, val, source='--config')
+compat.modernize(ui)
# developer config: ui.debugger
debugger = ui.config("ui", "debugger")
debugmod = pdb
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 10 of 10 RFC v2] compat: add ui.interface=curses to latest
# HG changeset patch
# User David Soria Parra
# Date 1489358250 25200
# Sun Mar 12 15:37:30 2017 -0700
# Node ID 4ae47aa5224bbca0f9bc35b2b0aef0ab01059ccc
# Parent b114907de6ce0f3d07c7be37e6570cc0a9456df6
compat: add ui.interface=curses to latest
diff --git a/mercurial/compat.py b/mercurial/compat.py
--- a/mercurial/compat.py
+++ b/mercurial/compat.py
@@ -22,6 +22,7 @@
},
'ui': {
'color': 'auto',
+'interface': 'curses',
},
})],
)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 03 of 10 RFC v2] ui: don't return certain layers when they are supposed to be displayed
# HG changeset patch
# User David Soria Parra
# Date 1489349206 25200
# Sun Mar 12 13:06:46 2017 -0700
# Node ID ef5ce5325596fe5fef014a320abae0f0d5980f3d
# Parent 084a30b54810d05c8b44d22fcab2ef6f783a9f7c
ui: don't return certain layers when they are supposed to be displayed
This is a hack to distinguish visibility for a certain configuration. As we have
layers in place, we can give them a visibility, allowing us to have internal
settings switched on without them showing up in `hg showconfig`. This is useful
for extension to set a list of default flags to trigger behavior.
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -360,11 +360,15 @@
cfg.set(section, name, value, source)
self.fixconfig(section=section)
-def _data(self, untrusted):
+def _data(self, untrusted, includeinternal=True):
+res = {}
+if includeinternal:
+res = self._cfg['defaults']
if untrusted:
-return self._cfg['user']
+res.update(self._cfg['user'])
else:
-return self._cfg['trusted']
+res.update(self._cfg['trusted'])
+return res
def configsource(self, section, name, untrusted=False):
return self._data(untrusted).source(section, name)
@@ -670,7 +674,7 @@
return items
def walkconfig(self, untrusted=False):
-cfg = self._data(untrusted)
+cfg = self._data(untrusted, includeinternal=False)
for section in cfg.sections():
for name, value in self.configitems(section, untrusted):
yield section, name, value
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 02 of 10 RFC v2] ui: add a defaults layer to the config
# HG changeset patch
# User David Soria Parra
# Date 1489349206 25200
# Sun Mar 12 13:06:46 2017 -0700
# Node ID 084a30b54810d05c8b44d22fcab2ef6f783a9f7c
# Parent 99514a82d5b23c75bd6da38e522acfd14a618c14
ui: add a defaults layer to the config
Add a defaults layer to the config that has the least precedence. This will
allow us to load defaults based on compatibility settings without interferring
with trusted or user.
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -304,7 +304,8 @@
def cfg(self):
# Ordered in ascneding order of preference.
return util.sortdict(
-[('user', config.config()),
+[('defaults', config.config()),
+('user', config.config()),
('trusted', config.config()),
('overlay', config.config())])
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 05 of 10 RFC v2] help: document compat mode
# HG changeset patch # User David Soria Parra # Date 1489349206 25200 # Sun Mar 12 13:06:46 2017 -0700 # Node ID d402acdf7c439dc369b6ab3a6888078a2895978a # Parent 0986fb7be0f584f5169f93377f9d013f87381ea7 help: document compat mode diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt --- a/mercurial/help/config.txt +++ b/mercurial/help/config.txt @@ -1837,6 +1837,13 @@ changes, abort the commit. (default: False) +``compat`` +String: Compatibility mode for the ui. Possible values are compat +or latest (default: compat). ``compat`` provides backwards compatible +ui behavior. ``latest`` enables additional config settings aiming to +improve user experience. (see ``HGPLAIN`` for compatibility modes in +non-interactive environments) + ``debug`` Print debugging information. (default: False) ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH 08 of 10 RFC v2] compat: add diff.showfunc=True to latest
# HG changeset patch
# User David Soria Parra
# Date 1489358226 25200
# Sun Mar 12 15:37:06 2017 -0700
# Node ID 8d4b79f7c6012aa1595fa3fbe184bc6849e3d81c
# Parent 7256c598a7f493bcc999259a4cbd88d86077746d
compat: add diff.showfunc=True to latest
diff --git a/mercurial/compat.py b/mercurial/compat.py
--- a/mercurial/compat.py
+++ b/mercurial/compat.py
@@ -18,6 +18,7 @@
('latest', {
'diff': {
'git': 'True',
+'showfunc': 'True',
},
})],
)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH v2] exewrapper: prefer HackableMercurial python if availbale
On Mon, Mar 13, 2017 at 01:06:41PM -0700, Kostia Balytskyi wrote: > # HG changeset patch > # User Kostia Balytskyi > # Date 1489434253 25200 > # Mon Mar 13 12:44:13 2017 -0700 > # Node ID 83925d3a3306e7023b4b43757812285b40fcd90b > # Parent 7548522742b5f4f9f5c0881ae4a2783ecda2f969 > exewrapper: prefer HackableMercurial python if availbale > > Currently hg.exe will only try to load python27.dll from hg-python > subdir if PYTHONHOME environment variable is not set. I think that > it is better to check whether 'hg-python' subdir exists and load > python from it in that case, regardless of environment. This allows > for reliable approach of distributing Mercurial with its own Python. FWIW this looks good to me ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] ui: don't read the same config file twice
# HG changeset patch
# User David Soria Parra
# Date 1489459557 25200
# Mon Mar 13 19:45:57 2017 -0700
# Node ID 7e599b4f534e27a0462bb1263208cc1c8d71
# Parent 3d3109339b57341b333c1112beb41dd281fa944a
ui: don't read the same config file twice
During dispatch and localrepo setup we might read the same file twice if
the repository we are loading is the currnet working directory/.hg which is the
default case. We now keep a set of filenames we read around and abort
if we read the config already. It would be nice to do this in a decorator,
but I have not found a reliable, simple way to hash arguments.
diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -160,6 +160,7 @@
self._colormode = None
self._terminfoparams = {}
self._styles = {}
+self._readfilenames = set()
if src:
self.fout = src.fout
@@ -258,6 +259,9 @@
def readconfig(self, filename, root=None, trust=False,
sections=None, remap=None):
+if filename in self._readfilenames:
+return
+
try:
fp = open(filename, u'rb')
except IOError:
@@ -304,6 +308,7 @@
if root is None:
root = os.path.expanduser('~')
self.fixconfig(root=root)
+self._readfilenames.add(filename)
def fixconfig(self, root=None, section=None):
if section in (None, 'paths'):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] ui: don't read the same config file twice
On Mon, Mar 13, 2017 at 07:56:09PM -0700, Jun Wu wrote: > This is a BC because loading order matters: good catch. let's drop it until i find a better solution. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 04 of 10] scmutil: add a rcpath-like method to return multiple config sources
On Mon, Mar 13, 2017 at 08:44:49PM -0700, Jun Wu wrote:
> scmutil: add a rcpath-like method to return multiple config sources
nice series :)
> +def rccomponents():
> +'''return an ordered [(type, obj)] about where to load configs.
> +
> +respect $HGRCPATH. if $HGRCPATH is empty, only .hg/hgrc of current repo
> is
> +used. if $HGRCPATH is not set, the platform default will be used.
> +
> +if a directory is provided, *.rc files under it will be used.
> +
> +type could be either 'path' or 'items', if type is 'path', obj is a
> string,
> +and is the config file path. if type is 'items', obj is a list of
> (section,
> +name, value, source) that should fill the config directly.
> +'''
> +global _rccomponents
> +if _rccomponents is None:
> +if 'HGRCPATH' in encoding.environ:
> +# assume HGRCPATH is all about user configs so environments, so
> +# environments can be overrided.
I think this should be 'environments can be 'overridden'. You duplicated 'so
environments'.
> +_rccomponents = [('items', envconfig(_sysenvlist))]
> +for p in encoding.environ['HGRCPATH'].split(pycompat.ospathsep):
> +if not p:
> +continue
> +p = util.expandpath(p)
Just a nit, maybe move this into a helper function?
def readpath(p):
p = util.expandpath(p)
if os.path.isdir(p):
entries = [(f,k) for (f,k) in osutil.listdir(p) if f.endswith('.rc')]
for f, kind in entries:
yield ('path', os.path.join(p,f))
else
return ('path', p)
_rccomponents.extend(readpath(p))
> +if os.path.isdir(p):
> +for f, kind in osutil.listdir(p):
> +if f.endswith('.rc'):
> +_rccomponents.append(('path', os.path.join(p,
> f)))
> +else:
> +_rccomponents.append(('path', p))
> +else:
> +syspath, userpath = osrcpath()
> +
> +def pathize(path):
> +return ('path', path)
> +
> +_rccomponents = map(pathize, syspath)
> +_rccomponents.append(('items', envconfig(_sysenvlist)))
> +_rccomponents.extend(map(pathize, userpath))
I see we barely use map and prefer list-comprehensions. I don't mind either, i
am just not sure if we have a coding style around it.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2] rebase: add flag to require destination
On Mon, Mar 13, 2017 at 08:15:58PM -0700, Augie Fackler wrote: > (+martinvonz,marmoute for my probably-bad idea) > > > On Mar 13, 2017, at 20:14, Ryan McElroy wrote: > > On 3/12/17 5:48 PM, Gregory Szorc wrote: > > > >> On Sun, Mar 12, 2017 at 12:06 PM, Augie Fackler wrote: > >> On Sat, Mar 11, 2017 at 06:03:13PM -0800, Ryan McElroy wrote: > >> > # HG changeset patch > >> > # User Ryan McElroy > >> > # Date 1489283611 28800 > >> > # Sat Mar 11 17:53:31 2017 -0800 > >> > # Node ID 7c7f442027b6a0cd51b1f06b01913f53f4f9e9cd > >> > # Parent a788a4660443dfc33c5c1c58eec78e20150404d9 > >> > rebase: add flag to require destination > >> > >> These both look mechanically fine to me, but I'm a little skeptical > >> about the configuration sections in play (update and rebase > >> respectively), so I'll leave them for other parties to examine and > >> bikeshed. > >> > >> The ship has likely already sailed, but IMO it would be nice if there a > >> unified [command] (or similar) section that controls behavior of commands, > >> specifically with regards to command arguments. But, a section per command > >> sharing the name of the command is acceptable. I just don't think options > >> for command argument behavior appearing in random sections is very user > >> friendly. > > > > A possibility is to group them all under the same section, but lets please > > have it *not* be ui, which is already super overloaded. > > > > I think pyd suggested a "[behavior]" section for this kind of thing. I'd be > > happy to send a v2 with this format: > > > > """ > > [behavior] > > rebase.requiredest = True > > update.requiredest = True > > """ > > > > (Both would still default to false of course) > > > > What do you think of this proposal? > > Seems good. Feels like a good place for per-command pager enable/disable as > well? And maybe also Martin's new merge config knob... > > What do people think about that as a place to collect those types of things? I think this is pretty good and probably also where we want to put the ui.compat knob (as in behavior.compat). I think a unified section is good as we will likely add more of these knobs for the compat behavior in cases where defaults.* or other options don't work (e.g. due to behavior depending on what other arguments are passed, etc). ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 7 of 7 V3] util: enable hardlink for copyfile
On Sun, Mar 12, 2017 at 01:23:56AM -0800, Jun Wu wrote: > # HG changeset patch > # User Jun Wu > # Date 1489309403 28800 > # Sun Mar 12 01:03:23 2017 -0800 > # Node ID 9da40a9e54c419490a2ff23b9dda7acc109f81cd > # Parent de28b62236a7d47a896bc4aba2bd95dcd8defc87 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > 9da40a9e54c4 > util: enable hardlink for copyfile I've looked through this in the last hours and I think the patch series is fine. It turns out it is much more conservative then gnulib on which df relies, which just read /proc/self/mountinfo and /proc/self/mounts and can potentially be fooled. Note that we probably want to per-process cache the output of _getfstypetable(). ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] patch: diff --stat can parse lines starting with -- or ++ (issue5479)
On Sun, Mar 12, 2017 at 01:32:05PM +0900, Matthieu Laneuville wrote:
> # HG changeset patch
> # User Matthieu Laneuville
> # Date 1489292061 -32400
> # Sun Mar 12 13:14:21 2017 +0900
> # Node ID a79e4ec615dd14a65d1730e2795be41ffededc9e
> # Parent 62939e0148f170b67ca8c7374f36c413b67fd387
> patch: diff --stat can parse lines starting with -- or ++ (issue5479)
>
> Lines starting with '--' or '++' would look like headers when removed or
> added,
> and therefore ignored by diffstatdata. This patch allow diffstatdata to make
> the
> difference between header lines (i.e. first '---' and '+++' lines after a line
> starting with 'diff') and usual lines starting with '--' or '++'.
>
> diff -r 62939e0148f1 -r a79e4ec615dd mercurial/patch.py
> --- a/mercurial/patch.py Wed Mar 08 18:11:41 2017 -0500
> +++ b/mercurial/patch.py Sun Mar 12 13:14:21 2017 +0900
> @@ -2597,6 +2597,8 @@
>
> for line in lines:
> if line.startswith('diff'):
> +seen_header_add = False
> +seen_header_rem = False
> addresult()
> # set numbers to 0 anyway when starting new file
> adds, removes, isbinary = 0, 0, False
> @@ -2605,9 +2607,13 @@
> elif line.startswith('diff -r'):
> # format: "diff -r ... -r ... filename"
> filename = diffre.search(line).group(1)
> -elif line.startswith('+') and not line.startswith('+++ '):
> +elif line.startswith('+++ ') and not seen_header_add:
> +seen_header_add = True
> +elif line.startswith('--- ') and not seen_header_rem:
> +seen_header_rem = True
> +elif line.startswith('+') and seen_header_add:
I can't think of anything but a misformatted patch, but I think we should check
for both seen_header_add and seen_header_rem to determine we have seen the
header:
seen_header = seen_header or (seen_header_rem and seen_header_add)
elif line.startswith('+') and seen_header:
...
elif line.startswith('-') and seen_header:
...
that makes it clear that we start counting after we have seen the full header
--- and +++ lines.
As an alternative we might be able to take the first occurence of @@ ... @@ to
denote the end of the header as it's part of unified diffs.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] fancyopts: making config defaults actually override defaults
On Sat, Mar 11, 2017 at 06:03:30PM -0800, Rodrigo Damazio Bovendorp via
Mercurial-devel wrote:
> # HG changeset patch
> # User Rodrigo Damazio
> # Date 1489274373 28800
> # Sat Mar 11 15:19:33 2017 -0800
> # Node ID 8c833b81a994e2d3304c3b06793f536355528aab
> # Parent 62939e0148f170b67ca8c7374f36c413b67fd387
> fancyopts: making config defaults actually override defaults
>
Overall this LGTM, and clearly makes defaults much better :). My concern
is that we are encouraging the use of defaults again, while they are
deprecated. Defaults have inherent problems that they overwrite arguments
which might be mutable exclusive with others (e.g. --graph in incoming
and outgoing), or lead to undesired behavior if it's set by an admin. an exmaple
is if you would specifiy defaults.update.check=True, the user will not find an
--no-check option in the help message or anywhere else. This is not a problem if
we assume defaults are alway set by the user and he knows about them.
> diff -r 62939e0148f1 -r 8c833b81a994 mercurial/fancyopts.py
> --- a/mercurial/fancyopts.py Wed Mar 08 18:11:41 2017 -0500
> +++ b/mercurial/fancyopts.py Sat Mar 11 15:19:33 2017 -0800
> @@ -142,18 +142,27 @@
> t = type(obj)
> if callable(obj):
> state[name] = defmap[name](val)
> -elif t is type(1):
> -try:
> -state[name] = int(val)
> -except ValueError:
> -raise error.Abort(_('invalid value %r for option %s, '
> - 'expected int') % (val, opt))
> -elif t is type(''):
> -state[name] = val
> elif t is type([]):
> state[name].append(val)
> -elif t is type(None) or t is type(False):
> -state[name] = boolval
> +else:
> +# non-callable single value.
> +state[name] = parsevalue(name, val, t, boolval)
>
> # return unparsed args
> return args
> +
> +def parsevalue(name, val, typ, boolval=True):
we have parsebool in util, maybe it's a good idea to put a more generic version
there?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 2 of 2 v2] rebase: add flag to require destination
On Tue, Mar 14, 2017 at 05:56:16PM -0700, Ryan McElroy wrote: > # HG changeset patch > # User Ryan McElroy > # Date 1489538624 25200 > # Tue Mar 14 17:43:44 2017 -0700 > # Node ID 8d8b783803f43d5e2d86916c39e9554139752fe6 > # Parent 2dc26c57e60e7e7bf46a276e8a498a9746bd9271 > rebase: add flag to require destination > This looks good to me. I was wondering if we want to provide separate knobs for these commands which might lead to config overhead or provide more comprehensive "ui" improvement knobs such as "commands.requiredest" to move people to a better model in logical steps. e.g. I am a user who likes a slightly enhanced user experience. ui.compat= is a bit too much for me, but update destinations is a good idea. Do i have to find all places where we use destinations to update or do I want to select a logical step? I personally think while fine granualar steps are nice, I'd probably lean towards logical steps as it provides a more consistent behavior for users (e.g. assume an extension Y that we don't know of can opt into using "commands.requiredest", which at the moment it cannot unless it depends on "commands.update.requiredest" which is missleading. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] convert: Fix the handling of empty changlist descriptions in P4
# HG changeset patch
# User David Soria Parra
# Date 1490209978 18000
# Wed Mar 22 14:12:58 2017 -0500
# Node ID ff24a4168c2f05fd367b0b126e52559ef89af77f
# Parent 102f291807c92864a2231e5e925d6cd64783bb59
convert: Fix the handling of empty changlist descriptions in P4
Empty changelist descriptions are valid in Perforce. If we encounter one of
them we are currently running into an IndexError. In case of empty commit
messages set the commit message to **empty changelist description**, which
follows Perforce terminology.
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -161,7 +161,12 @@
d = self._fetch_revision(change)
c = self._construct_commit(d, parents)
-shortdesc = c.desc.splitlines(True)[0].rstrip('\r\n')
+descarr = c.desc.splitlines(True)
+if len(descarr) > 0:
+shortdesc = descarr[0].rstrip('\r\n')
+else:
+shortdesc = '**empty changelist description**'
+
t = '%s %s' % (c.rev, repr(shortdesc)[1:-1])
ui.status(util.ellipsis(t, 80) + '\n')
diff --git a/tests/test-convert-p4.t b/tests/test-convert-p4.t
--- a/tests/test-convert-p4.t
+++ b/tests/test-convert-p4.t
@@ -141,5 +141,23 @@
rev=1 desc="change a" tags="" files="a"
rev=0 desc="initial" tags="" files="a b/c"
+empty commit message
+ $ p4 edit a
+ //depot/test-mercurial-import/a#3 - opened for edit
+ $ echo a >> a
+ $ p4 submit -d ""
+ Submitting change 6.
+ Locking 1 files ...
+ edit //depot/test-mercurial-import/a#4
+ Change 6 submitted.
+ $ hg convert -s p4 $DEPOTPATH dst
+ scanning source...
+ reading p4 views
+ collecting p4 changelists
+ 6 **empty changelist description**
+ sorting...
+ converting...
+ 0
+
exit trap:
stopping the p4 server
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] help: correct title undescore length
# HG changeset patch # User David Soria Parra # Date 1490290521 -3600 # Thu Mar 23 18:35:21 2017 +0100 # Node ID a58314d8ab7f4cd910c238441779731ad16372d6 # Parent 55c6788c54e2faf80ec14f2b0844bfe429012bc3 help: correct title undescore length The underscores for a title must have the same lenght as the title, otherwise gendoc will throw an error. This leads to `make install` fail. diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt --- a/mercurial/help/config.txt +++ b/mercurial/help/config.txt @@ -415,7 +415,7 @@ extension). ``commands`` --- + ``update.requiredest`` Require that the user pass a destination when running ``hg update``. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 1 of 3] setup: use a more strict way to test BSD or OSX's statfs
On Thu, Mar 23, 2017 at 10:32:52PM -0700, Jun Wu wrote: > # HG changeset patch > # User Jun Wu > # Date 1490332536 25200 > # Thu Mar 23 22:15:36 2017 -0700 > # Node ID 597a29c947fe2b9f9ac0a6a03cf710ab9f69757c > # Parent 2c02bb7fd7fc1212029dc903527e35a9efb7dbe1 > # Available At https://bitbucket.org/quark-zju/hg-draft > # hg pull https://bitbucket.org/quark-zju/hg-draft -r > 597a29c947fe > setup: use a more strict way to test BSD or OSX's statfs > > We want to use the `f_fstypename` field to get the filesystem type. Test it > directly. The new macro HAVE_BSD_STATFS implys the old HAVE_SYS_MOUNT_H and > HAVE_SYS_PARAM_H. So the latter ones are removed. > I think the series is good. I wonder if it makes sense to have a more generic hasmember() function in setup.py. I have a patch for that as I was working on ZFS and UFS detection myself. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH] worker: flush ui buffers before running the worker
# HG changeset patch
# User David Soria Parra
# Date 1490721698 25200
# Tue Mar 28 10:21:38 2017 -0700
# Node ID b89199982355728a3741edf5518c72ccc52ee33c
# Parent 331cc4433efe0d897bb16ad4ff08a3fbe850869b
worker: flush ui buffers before running the worker
a91c6275 introduces flushing ui buffers after a worker finished. If the ui was
not flushed before the worker was started, fork will copy the existing buffers
to the worker. This causes messages issued before the worker started to be
written to the terminal for each worker.
We are now flushing the ui before we start a worker and add an appropriate test
which will fail before this patch.
diff --git a/mercurial/worker.py b/mercurial/worker.py
--- a/mercurial/worker.py
+++ b/mercurial/worker.py
@@ -133,6 +133,7 @@
if problem[0]:
killworkers()
oldchldhandler = signal.signal(signal.SIGCHLD, sigchldhandler)
+ui.flush()
for pargs in partition(args, workers):
pid = os.fork()
if pid == 0:
diff --git a/tests/test-worker.t b/tests/test-worker.t
new file mode 100644
--- /dev/null
+++ b/tests/test-worker.t
@@ -0,0 +1,52 @@
+Test UI worker interaction
+
+ $ cat > t.py < from __future__ import absolute_import, print_function
+ > from mercurial import (
+ > worker, ui as uimod, cmdutil
+ > )
+ > def runme(ui, args):
+ > for arg in args:
+ > ui.status('run\n')
+ > yield 1, arg
+ > cmdtable = {}
+ > command = cmdutil.command(cmdtable)
+ > @command('test', [], 'hg test [COST]')
+ > def t(ui, repo, cost=1.0):
+ > cost = float(cost)
+ > ui.status('start\n')
+ > runs = worker.worker(ui, cost, runme, (ui,), range(8))
+ > for n, i in runs:
+ > pass
+ > ui.status('done\n')
+ > EOF
+ $ abspath=`pwd`/t.py
+ $ hg init
+
+Run tests with worker enable by forcing a heigh cost
+
+ $ hg --config "extensions.t=$abspath" test 10.0
+ start
+ run
+ run
+ run
+ run
+ run
+ run
+ run
+ run
+ done
+
+Run tests without worker by forcing a low cost
+
+ $ hg --config "extensions.t=$abspath" test 0.001
+ start
+ run
+ run
+ run
+ run
+ run
+ run
+ run
+ run
+ done
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [1,of,2,RFC] RFC: implement immutable config objects
On Mon, Mar 27, 2017 at 11:38:06AM -0700, Jun Wu wrote:
> # HG changeset patch
> # User Jun Wu
> # Date 1490635856 25200
> # Mon Mar 27 10:30:56 2017 -0700
> # Node ID 4eb7c76340791f379a34f9df4ec42e0c8b9b2a2f
> # Parent 4a8d065bbad80d3b3401010375bc80165404aa87
> RFC: implement immutable config objects
I like the overall approach and that it's parallel to config.config() and
ui.config. We might in later patches combine config.config into this.
> +class abstractimmutableconfig(object):
> +"""minimal interface defined for a read-only config accessor
> +
> +The immutable config object should get its state set and frozen during
> +__init__ and should not have any setconfig-like method.
> +
> +The immutable config layer knows about sections, and should probably uses
> +ordered dict for each section. To simplify things, this layer does not
> care
> +about "source", "unset" or any filesystem IO. They're up to the upper
> layer
> +to deal with. For example, the upper layer could store "None" as
> "unset"s,
> +and store (value, source) as a tuple together.
> +"""
> +
> +def __init__(self, title):
> +"""title: useful to identify the config"""
> +self.title = title
> +
> +def subconfigs(self, section=None):
> +"""return a list-ish of child config objects filtered by section"""
> +raise NotImplementedError
> +
> +def get(self, section, item):
> +"""return value or None"""
> +return self.getsection(section).get(item)
> +
> +def getsection(self, section):
> +"""return a dict-ish"""
> +raise NotImplementedError
> +
> +def sections(self):
> +"""return an iter-able"""
> +raise NotImplementedError
> +
> +class atomicconfig(abstractimmutableconfig):
> +"""immutable config that converted from a list-ish"""
> +
> +def __init__(self, title, entries=None):
> +"""
> +title: a title used to identify the atomicconfig
> +entries: a list-ish of (section, item, value)
> +"""
> +super(atomicconfig, self).__init__(title)
> +self._sections = util.sortdict()
Can we use collections.defaultdict(util.sortdict) here?
> +for entry in entries:
> +section, item, value = entry
> +if section not in self._sections:
> +self._sections[section] = util.sortdict()
> +if item is not None:
> +self._sections[section][item] = value
> +
> +def subconfigs(self, section=None):
> +return ()
> +
> +def getsection(self, section):
> +return self._sections.get(section, {})
> +
> +def sections(self):
> +return self._sections.keys()
> +
> +class mergedconfig(abstractimmutableconfig):
> +"""immutable config that is a merge of a list of immutable configs"""
> +
> +def __init__(self, title, subconfigs):
> +super(mergedconfig, self).__init__(title)
> +self._subconfigs = tuple(subconfigs) # make it immutable
> +self._cachedsections = {}
> +
> +def subconfigs(self, section=None):
I am not sure about this API. We already have getsection() from
abstractimmutableconfig. I think this method does two things depending
on the arguments.
> +if section is None:
> +return self._subconfigs
> +else:
> +return self._sectionconfigs.get(section, ())
> +
> +def sections(self):
> +return self._sectionconfigs.keys()
> +
> +@util.propertycache
> +def _sectionconfigs(self):
> +"""{section: [subconfig]}"""
> +sectionconfigs = {}
> +for subconfig in self._subconfigs:
> +for section in subconfig.sections():
> +sectionconfigs.setdefault(section, []).append(subconfig)
> +return sectionconfigs
> +
> +def getsection(self, section):
> +items = self._cachedsections.get(section, None)
> +if items is None:
> +subconfigs = self._sectionconfigs.get(section, [])
> +if len(subconfigs) == 1:
> +# no need to merge configs
> +items = subconfigs[0].getsection(section)
> +else:
> +# merge configs
> +items = util.sortdict()
> +for subconfig in subconfigs:
> +subconfigitems = subconfig.getsection(section).items()
Could we do an `items.update(subconfig.getsection(section))`?
> +for item, value in subconfigitems:
> +items[item] = value
> +self._cachedsections[section] = items
> +return items
> +
> +def append(self, subconfig):
> +"""return a new mergedconfig with the new subconfig appended"""
> +return mergedconfig(self.title, list(self._subconfigs) + [subconfig])
> +
> +def prepend(self, subconfig):
> +"""return a new mergedconfig with the new subconfig prepended"""
> +return mergedconfig(self.
Re: [2,of,2,RFC] RFC: switch to immutable configs
On Mon, Mar 27, 2017 at 11:38:07AM -0700, Jun Wu wrote:
> # HG changeset patch
> # User Jun Wu
> # Date 1490639836 25200
> # Mon Mar 27 11:37:16 2017 -0700
> # Node ID 13bee3e959f04f970f2fc0a01120f0b30d725b84
> # Parent 4eb7c76340791f379a34f9df4ec42e0c8b9b2a2f
> RFC: switch to immutable configs
I personally like the overall direction
> +def _filterconfig(subconfig):
> +'''remove configs according to HGPLAIN and HGPLAINEXCEPT
> +
> +subconfig is an immutable config object. Returns an immutable config
> object
> +with related fields filtered.
> +'''
> +filters = {}
> +if _isplain():
> +def filterui(items):
> +result = util.sortdict(items)
> +for k in ('debug', 'fallbackencoding', 'quiet', 'slash',
> + 'logtemplate', 'statuscopies', 'style',
> + 'traceback', 'verbose'):
> +if k in result:
> +del result[k]
> +return result
> +
> +filters['ui'] = filterui
> +filters['defaults'] = {}
> +if _isplain('alias'):
> +filters['alias'] = {}
> +if _isplain('revsetalias'):
> +filters['revsetalias'] = {}
> +if _isplain('templatealias'):
> +filters['templatealias'] = {}
> +if _isplain('commands'):
> +filters['commands'] = {}
> +
> +if filters:
> +return config.filteredconfig('filter', subconfig, filters)
> +else:
> +return subconfig
> +
> +def _getconfig(configroot, section, name, default=None, index=0):
> +'''get value (index=0) or source (index=-1) from an immutable config'''
> +value = configroot.getsection(section).get(name, (None,))[index]
> +if value is None:
> +value = default
> +return value
> +
> +def _buildconfigroot(cfg, ocfg, gcfg):
> +return config.mergedconfig('root', [cfg, ocfg, gcfg])
> +
> +def dependson(*fields):
> +'''cache result which gets invalidates if any field changes'''
> +
> +def decorator(oldfunc):
> +cached = [[None], None] # cache key, result
> +def getcachekey(self):
> +return [getattr(self, f, None) for f in fields]
> +
> +def newfunc(self):
> +newkey = getcachekey(self)
> +oldkey = cached[0]
> +if oldkey == newkey:
> +return cached[1]
> +result = oldfunc(self)
> +cached[:] = [newkey, result]
> +return result
> +newfunc.__name__ = oldfunc.__name__
> +newfunc.__doc__ = oldfunc.__doc__
> +return newfunc
> +return decorator
> +
> class ui(object):
> def __init__(self, src=None):
> @@ -146,16 +257,7 @@ class ui(object):
> # This exists to prevent an extra list lookup.
> self._bufferapplylabels = None
> -self.quiet = self.verbose = self.debugflag = self.tracebackflag =
> False
> -self._reportuntrusted = True
> -self._ocfg = config.config() # overlay
> -self._tcfg = config.config() # trusted
> -self._ucfg = config.config() # untrusted
> -self._trustusers = set()
> -self._trustgroups = set()
> self.callhooks = True
> # Insecure server connections requested.
> self.insecureconnections = False
> -# Blocked time
> -self.logblockedtimes = False
> # color mode: see mercurial/color.py for possible value
> self._colormode = None
> @@ -170,9 +272,9 @@ class ui(object):
> self._disablepager = src._disablepager
>
> -self._tcfg = src._tcfg.copy()
> -self._ucfg = src._ucfg.copy()
> -self._ocfg = src._ocfg.copy()
> -self._trustusers = src._trustusers.copy()
> -self._trustgroups = src._trustgroups.copy()
> +# immutable configs can be reused without copying
> +self._ocfgs = src._ocfgs
> +self._tcfgs = src._tcfgs
> +self._ucfgs = src._ucfgs
> +
> self.environ = src.environ
> self.callhooks = src.callhooks
> @@ -182,6 +284,4 @@ class ui(object):
> self._styles = src._styles.copy()
>
> -self.fixconfig()
> -
> self.httppasswordmgrdb = src.httppasswordmgrdb
> self._blockedtimes = src._blockedtimes
> @@ -199,4 +299,9 @@ class ui(object):
> self._blockedtimes = collections.defaultdict(int)
>
> +# immutable configs
> +self._ocfgs = config.mergedconfig('setconfig', []) # overlay
> +self._tcfgs = config.mergedconfig('loaded', []) # trusted
i would probably actually call them 'trusted' instead of 'loaded'
> +self._ucfgs = config.mergedconfig('loaded', []) #
> trusted+untrusted
with merge configs in, we would split up trusted+untrusted into trusted and
untrusted?
> +def setconfig(self, section, name, value, source='', priority=None):
I think we can do better by using the title, e.g. 'untrusted', 's
Re: [PATCH 4 of 5] util: add helper to convert between LF and native EOL
On Wed, Mar 29, 2017 at 11:46:40PM +0900, Yuya Nishihara wrote: > # HG changeset patch > # User Yuya Nishihara > # Date 1490791215 -32400 > # Wed Mar 29 21:40:15 2017 +0900 > # Node ID c43fe15a4a4bec542b7411087a1ad0233cb5614f > # Parent 24dce83e6706dfaad602e9ff0440ac40f6a4cc37 > util: add helper to convert between LF and native EOL > Theese patches look good to me, and I would mark them pre-reviewed if I could. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 5 of 5] ui: use bytes IO and convert EOL manually in ui.editor()
On Wed, Mar 29, 2017 at 11:46:41PM +0900, Yuya Nishihara wrote: > # HG changeset patch > # User Yuya Nishihara > # Date 1490791418 -32400 > # Wed Mar 29 21:43:38 2017 +0900 > # Node ID 81aa3738f703e4354b0f853e87548a139b129bdd > # Parent c43fe15a4a4bec542b7411087a1ad0233cb5614f > ui: use bytes IO and convert EOL manually in ui.editor() > I am not sure if we should convert line endings. In particular crecord is calling ui.edit() with a patch. A patch that would have CRLF line endings would be displayed wrong and if someone wants to add a lineending or remove it in the process, we would wrongly convert it to LF on Windows machines. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 5 of 5] ui: use bytes IO and convert EOL manually in ui.editor()
On Sat, Apr 01, 2017 at 07:00:07PM +0900, Yuya Nishihara wrote: > On Fri, 31 Mar 2017 09:52:37 -0700, David Soria Parra wrote: > > On Wed, Mar 29, 2017 at 11:46:41PM +0900, Yuya Nishihara wrote: > > > # HG changeset patch > > > # User Yuya Nishihara > > > # Date 1490791418 -32400 > > > # Wed Mar 29 21:43:38 2017 +0900 > > > # Node ID 81aa3738f703e4354b0f853e87548a139b129bdd > > > # Parent c43fe15a4a4bec542b7411087a1ad0233cb5614f > > > ui: use bytes IO and convert EOL manually in ui.editor() > > > > > I am not sure if we should convert line endings. In particular crecord is > > calling ui.edit() with a patch. A patch that would have CRLF line endings > > would > > be displayed wrong and if someone wants to add a lineending or remove it in > > the > > process, we would wrongly convert it to LF on Windows machines. > > We need EOL conversion on Windows because of notepad.exe. For crecord, we > might want to add ui.edit(text=False) option. That could work. I feel all user input either in the form of crecords patch or comittemplates should be returned to the user as is. Internal strings, such as the default comittemplate or similar things should be converted to the platform specific. An ui.edit(..., raw=True) switch could work ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH 4 of 4] formatter: use templatefilters.json()
On Sun, Apr 02, 2017 at 12:28:24PM +0900, Yuya Nishihara wrote: > # HG changeset patch > # User Yuya Nishihara > # Date 1491102137 -32400 > # Sun Apr 02 12:02:17 2017 +0900 > # Node ID 6deb06584ae481c751b86782e2c8567e4996d04e > # Parent eba702fdc933ed84d5f53225bfe87ebc40c7831e > formatter: use templatefilters.json() This series LGTM for what it's worth. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH] sortdict: fix .pop() to return a value
On Sun, Apr 09, 2017 at 10:10:01PM +0900, Yuya Nishihara wrote: > # HG changeset patch > # User Yuya Nishihara > # Date 1491706629 -32400 > # Sun Apr 09 11:57:09 2017 +0900 > # Node ID 48a7a1f77a9489e3c4b5f862243782ceae80eaf9 > # Parent 9259cf823690e4fcd34a4d2ecd57ced2060d2b3d > sortdict: fix .pop() to return a value > > My future patch will need it. Generally this looks fine to me. > diff --git a/mercurial/util.py b/mercurial/util.py > --- a/mercurial/util.py > +++ b/mercurial/util.py > @@ -555,11 +555,11 @@ class sortdict(dict): > dict.__delitem__(self, key) > self._list.remove(key) > def pop(self, key, *args, **kwargs): > -dict.pop(self, key, *args, **kwargs) > try: > self._list.remove(key) > except ValueError: > pass > +return dict.pop(self, key, *args, **kwargs) Note that this breaks the insert/delete asymmetry. We usually insert into the _list first and then add it to the dictionary and the current version of pop deletes from the dictionary first and then from the list ensuring an add to list add to dict del from dict del from list order. Your patch breaks this. I wonder if can lead to unwanted behavior. If we want to be on the save side we should probably do res = dict.pop(...) try: self._list.remove() ... return res ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: https://www.mercurial-scm.org/downloads doesn't have download links?
On Sun, May 14, 2017 at 11:34:30AM -0700, Siddharth Agarwal wrote: > A coworker alerted me to this: https://www.mercurial-scm.org/downloads > doesn't appear to have any download links. For Windows there should be a > link to a TortoiseHg build or the MSI. https://www.mercurial-scm.org/repo/hg-website/rev/c65b81353b8f seems to be the cause. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
[PATCH RFC] revset: lookup descendents for negative arguments to ancestor operator
# HG changeset patch
# User David Soria Parra
# Date 1495905909 25200
# Sat May 27 10:25:09 2017 -0700
# Node ID 47c710c797624b6d5c84c493a873ea8beb563a74
# Parent b647b923486f38d83b92089eafa9faafaa79785d
revset: lookup descendents for negative arguments to ancestor operator
Negative offsets to the `~` operator now search for descendents. The search is
aborted when a node has more than one child as we do not have a definition for
'nth child'. Optionally we can introduce such a notion and take the nth child
ordered by rev number.
The current revset language does provides a short operator for ancestor lookup
but not for descendents. This gives user a simple revset to move to the previous
changeset, e.g. `hg up '.~1'` but not to the 'next' changeset. With this change
userse can now use `.~-1` as a shortcut to move to the next changeset.
This fits better into allowing users to specify revisions via revsets and
avoiding the need for special `hg next` and `hg prev` operations.
The alternative to negative offsets is adding a new operator. We do not have
many operators in ascii left that do not require bash escaping (',', '_', and
'/' come to mind). If we decide that we should add a more convenient short
operator such as ('/', e.g. './1') we can later add it and allow ascendents
lookup via negative numbers.
diff --git a/mercurial/help/revisions.txt b/mercurial/help/revisions.txt
--- a/mercurial/help/revisions.txt
+++ b/mercurial/help/revisions.txt
@@ -97,6 +97,7 @@
``x~n``
The nth first ancestor of x; ``x~0`` is x; ``x~3`` is ``x^^^``.
+ For n < 0, the nth unambiguous descendent of x.
``x ## y``
Concatenate strings and identifiers into one string.
diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -378,12 +378,33 @@
# Like ``ancestors(set)`` but follows only the first parents.
return _ancestors(repo, subset, x, followfirst=True)
+def _childrenspec(repo, subset, x, n, order):
+"""Changesets that are the Nth child of a changeset
+in set.
+"""
+cs = set()
+for r in getset(repo, fullreposet(repo), x):
+for i in range(n):
+c = repo[r].children()
+if len(c) == 0:
+break
+if len(c) > 1:
+raise error.RepoLookupError(
+_("revision in set has more than one child"))
+r = c[0]
+else:
+cs.add(r)
+return subset & cs
+
def ancestorspec(repo, subset, x, n, order):
"""``set~n``
Changesets that are the Nth ancestor (first parents only) of a changeset
in set.
"""
n = getinteger(n, _("~ expects a number"))
+if n < 0:
+# children lookup
+return _childrenspec(repo, subset, x, -n, order)
ps = set()
cl = repo.changelog
for r in getset(repo, fullreposet(repo), x):
diff --git a/tests/test-revset.t b/tests/test-revset.t
--- a/tests/test-revset.t
+++ b/tests/test-revset.t
@@ -2881,6 +2881,14 @@
$ log 'merge()^^^'
1
+ $ log '(merge() | 0)~-1'
+ 7
+ 1
+ $ log 'merge()~-1'
+ 7
+ $ log 'tip~-1'
+ $ log '(tip | merge())~-1'
+ 7
$ log 'merge()~0'
6
$ log 'merge()~1'
@@ -2901,6 +2909,10 @@
hg: parse error: ^ expects a number 0, 1, or 2
[255]
+ $ log 'branchpoint()~-1'
+ abort: revset has more than one child!
+ [255]
+
Bogus function gets suggestions
$ log 'add()'
hg: parse error: unknown identifier: add
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH RFC] revset: lookup descendents for negative arguments to ancestor operator
On Wed, May 31, 2017 at 10:50:15PM +0900, Yuya Nishihara wrote: > On Mon, 29 May 2017 23:28:20 +0900, Yuya Nishihara wrote: > > On Mon, 29 May 2017 14:45:39 +0200, Denis Laxalde wrote: > > > > I agree it's a bit puzzling, but I also think it's an acceptable extension > > to > > the `~` operator as long as descendants history is linear. However, from > > implementation POV, it would be unnecessarily expensive to make sure there's > > no more than one child. > > So I'm against the current proposal. FWIW, if we can define the first > descendant as the nearest child in revision number space, the computation > will be cheaper. I considered this and decided to not introduce this notion of nearest child. I feel that the notion of revnumber as an ordering is unintuitive for users of a decentralized version control system as it adds an concept of ordering on the existing dag, and rev numbers are rarely used by users for ordering (hence people use :: far more than : in revsets). I feel while the computation is faster it might often lead to unexpected results. The user must know the revision numbers of children in order to know what 'hg update .~-2' would do. That's why I felt restricting it the way i did in the patch was more sensibile for userbility. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH RFC] revset: lookup descendents for negative arguments to ancestor operator
On Wed, May 31, 2017 at 10:34:55AM -0700, Sean Farley wrote: > David Soria Parra writes: > > > On Wed, May 31, 2017 at 10:50:15PM +0900, Yuya Nishihara wrote: > >> On Mon, 29 May 2017 23:28:20 +0900, Yuya Nishihara wrote: > >> > On Mon, 29 May 2017 14:45:39 +0200, Denis Laxalde wrote: > >> > > >> > I agree it's a bit puzzling, but I also think it's an acceptable > >> > extension to > >> > the `~` operator as long as descendants history is linear. However, from > >> > implementation POV, it would be unnecessarily expensive to make sure > >> > there's > >> > no more than one child. > >> > >> So I'm against the current proposal. FWIW, if we can define the first > >> descendant as the nearest child in revision number space, the computation > >> will be cheaper. > > > > I considered this and decided to not introduce this notion of nearest > > child. I feel that the notion of revnumber as an ordering is unintuitive for > > users of a decentralized version control system as it adds an concept of > > ordering on the existing dag, > > But this is what Mercurial has always done. I think we have always used it as an optional method rather than a way to disambiguate. > > and rev numbers are rarely used by users for > > ordering (hence people use :: far more than : in revsets). > > I don't think that's sound reasoning. As a counter argument, Bitbucket > manually sorts each git repo so that it has a faux rev number. We then > use that number for children / descendant calculations. > > > I feel while the > > computation is faster it might often lead to unexpected results. The user > > must > > know the revision numbers of children in order to know what 'hg update .~-2' > > would do. That's why I felt restricting it the way i did in the patch was > > more > > sensibile for userbility. > > I'm unsure. On one hand, I'd like to get something through. But on the > other, we have years of precedence of using rev numbers. I don't mind changing the proposal if we agree. I prefer having something in and don't feel too strongly about my current approach, although i think it's much more user friendly than relying on revnumbers. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH RFC] revset: lookup descendents for negative arguments to ancestor operator
On Wed, May 31, 2017 at 02:17:58PM -0400, Augie Fackler wrote: > > It's easy to allow child traversal only if it's unambigious, and come > back to it later and be more permissive later. It'll be hard to go the > other way though. > > (I'm lightly in favor of this series, +0-ish, but I need to re-read > mpm's operator plan and see how they overlap, it's been too long.) > What's the conclusion on the RFC patch? Is that something we want with the current restrictions? Of other people like greg, sid, etc need to weight in? -David ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: [PATCH RFC] ui: add support for a tweakdefaults knob
On Sat, Jun 17, 2017 at 12:14:22AM +0900, Yuya Nishihara wrote: > On Wed, 14 Jun 2017 21:37:21 -0400, Augie Fackler wrote: > > # HG changeset patch > > # User Augie Fackler > > # Date 1497488194 14400 > > # Wed Jun 14 20:56:34 2017 -0400 > > # Node ID 0e5ea7a86a8021d02218c35b07366ac6081ab3fb > > # Parent 3abba5bc34546951b11b1bd3f5e5c77b90d950d1 > > ui: add support for a tweakdefaults knob > > Queued this, thanks. > I briefly glanced over this. I like the direciton, but prefer our initial proposal of using ui.compat= with multiple compat knobs. I can revive my patch for this, but I was initially holding back to wait for Jun's immutable config work. ui.compat would allow us to allow to add multiple configuration points, e.g. ui.compat=hg4.0, ui.compat=latest. I feel that the name `tweakdefaults` is not perfectly suitable if this becomes a one-stop-shop for changing behavior rather than just setting defaults. Besides these minors things, I am hugely in favor of this. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Re: Upstreaming Facebook extensions
On Tue, Nov 14, 2017 at 10:36:36AM -0800, Jun Wu wrote: > Excerpts from Matt Harbison's message of 2017-11-13 22:49:56 -0500: > > OK, good to know. > > > > So is upstreaming lfs in a single patch and marking it experimental a > > reasonable next step, or does this need to incubate in hg-experimental a > > bit more? I didn't get it working outside of the tests (it said something > > about not having a common changegroup version), but it looks like there's > > some low hanging fruit like registering the config options, and tracking > > down some warning in url.py about an "unquoted realm" or similar. > > There are some ongoing tweaks about transferring files in threads on > Windows. But I think it's fine to also have an upstream version. The format > is considered stable. I think we should upstream this. Besides the tweaks for transferring files there is still work that needs to be done on UI, resumable downloads and additional consistency checking. One of the major things to work on if we upstream is how it interacts with largefiles. ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
