[PATCH evolve-ext] evolve: fix the way evolve checks whether dirstate.write has arguments

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1474560467 25200
#  Thu Sep 22 09:07:47 2016 -0700
# Branch stable
# Node ID ac125f907dfe8f85d0c2771b3f71eaec788d9ea0
# Parent  8f902ec9ed9a296d92c2c2df1536af8c44b5321c
evolve: fix the way evolve checks whether dirstate.write has arguments


Earlier dirstate.write had the following signature: def write(self, tr=_token)
Since it had a default argument, func_defaults would list it.
In 52ff07e1de9150a8cec8e6cbe02125fb67978b8d the signature was changed to be
def write(self, tr), thus making the func_defaults check incorrect. This breaks
some evolve tests.

diff --git a/hgext/evolve.py b/hgext/evolve.py
--- a/hgext/evolve.py
+++ b/hgext/evolve.py
@@ -1024,7 +1024,7 @@ def bmactive(repo):
 ### dirstate compatibility layer < hg 3.6
 
 def writedirstate(dirstate, tr):
-if dirstate.write.func_defaults is not None: # mercurial 3.6 and above
+if dirstate.write.func_code.co_argcount: # mercurial 3.6 and above
 return dirstate.write(tr)
 return dirstate.write()
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH evolve-ext] evolve: fix test breaks related to double->single quote changes

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1474560930 25200
#  Thu Sep 22 09:15:30 2016 -0700
# Branch stable
# Node ID bc4c9029017b8dbc35d4bb47b9be23f2c85ca121
# Parent  ac125f907dfe8f85d0c2771b3f71eaec788d9ea0
evolve: fix test breaks related to double->single quote changes

diff --git a/tests/test-evolve.t b/tests/test-evolve.t
--- a/tests/test-evolve.t
+++ b/tests/test-evolve.t
@@ -1126,7 +1126,7 @@ Enabling commands selectively, no comman
summary   summarize working directory state
updateupdate working directory (or switch revisions)
   
-  (use "hg help" for the full list of commands or "hg -v" for details)
+  (use 'hg help' for the full list of commands or 'hg -v' for details)
   [255]
   $ hg fold
   hg: unknown command 'fold'
@@ -1152,7 +1152,7 @@ Enabling commands selectively, no comman
summary   summarize working directory state
updateupdate working directory (or switch revisions)
   
-  (use "hg help" for the full list of commands or "hg -v" for details)
+  (use 'hg help' for the full list of commands or 'hg -v' for details)
   [255]
 Enabling commands selectively, only fold enabled, next is still unknown
   $ cat >> $HGRCPATH <https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH remotenames-ext] tests: fix failures related to double->single quote changes

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1474562531 25200
#  Thu Sep 22 09:42:11 2016 -0700
# Node ID 3aa5c4dbf5086615dbbe42b0d2d9fd5ca0488cf1
# Parent  7a6c5ff76f225c8ebe9baef9d5ef753da915aa8b
tests: fix failures related to double->single quote changes

diff --git a/tests/test-pull-rebase.t b/tests/test-pull-rebase.t
--- a/tests/test-pull-rebase.t
+++ b/tests/test-pull-rebase.t
@@ -119,7 +119,7 @@ Tests the behavior of a pull followed by
   searching for changes
   no changes found
   abort: can't rebase public changeset 4557926d2166
-  (see "hg help phases" for details)
+  (see 'hg help phases' for details)
   [255]
 
 Tests that there are no race condition between pulling changesets and remote 
bookmarks
diff --git a/tests/test-tracking.t b/tests/test-tracking.t
--- a/tests/test-tracking.t
+++ b/tests/test-tracking.t
@@ -14,7 +14,7 @@ Make sure we don't fail when rebase does
   
   rebasecommand to move sets of revisions to a different ancestor
   
-  (use "hg help extensions" for information on enabling extensions)
+  (use 'hg help extensions' for information on enabling extensions)
   [255]
   $ echo "rebase=" >> $HGRCPATH
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH remotenames-ext] bookmarks: adopt the formatter isplain method

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1475508913 25200
#  Mon Oct 03 08:35:13 2016 -0700
# Node ID bc204ce8544ee98aed0119cbf5eec6a3618bf0db
# Parent  e4c0713ea86204b900a2e1cce238f61e4bef2062
bookmarks: adopt the formatter isplain method

Main hg repo changed the way formatter is supposed to be checked for
plainness: previously people would run `if not fm`, thus relying on
`__nonzero__` method, but now there is `isplain` method of formatter.
Remotenames needs to adopt it.

diff --git a/remotenames.py b/remotenames.py
--- a/remotenames.py
+++ b/remotenames.py
@@ -1047,7 +1047,7 @@ def displaylocalbookmarks(ui, repo, opts
 fm = ui.formatter('bookmarks', opts)
 hexfn = fm.hexfunc
 marks = repo._bookmarks
-if len(marks) == 0 and not fm:
+if len(marks) == 0 and (not fm or  fm.isplain()):
 ui.status(_("no bookmarks set\n"))
 
 tracking = _readtracking(repo)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH remotenames-ext] tests: adjust tests to respect negation in cmd flags

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1475505045 25200
#  Mon Oct 03 07:30:45 2016 -0700
# Node ID e4c0713ea86204b900a2e1cce238f61e4bef2062
# Parent  3aa5c4dbf5086615dbbe42b0d2d9fd5ca0488cf1
tests: adjust tests to respect negation in cmd flags

diff --git a/tests/test-remotenames.t b/tests/test-remotenames.t
--- a/tests/test-remotenames.t
+++ b/tests/test-remotenames.t
@@ -289,9 +289,9 @@ Test loading with hggit
   $ echo "hggit=" >> $HGRCPATH
   $ hg help bookmarks  | grep -A 3 -- '--track'
-t --track BOOKMARK track this bookmark or remote name
-   -u --untrackremove tracking for this bookmark
-   -a --allshow both remote and local bookmarks
-  --remote show only remote bookmarks
+   -u --[no-]untrack   remove tracking for this bookmark
+   -a --[no-]all   show both remote and local bookmarks
+  --[no-]remoteshow only remote bookmarks
 
 Test branches marked as closed are not loaded
   $ cd ../alpha
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH remotenames-ext] bookmarks: adopt the formatter isplain method

[Kostia Balytskyi]

On 10/3/16, 5:29 PM, "Mercurial-devel on behalf of Kevin Bullock" 
 wrote:

> On Oct 3, 2016, at 10:48, Kostia Balytskyi  wrote:
> 
> # HG changeset patch
    > # User Kostia Balytskyi 
> # Date 1475508913 25200
> #  Mon Oct 03 08:35:13 2016 -0700
> # Node ID bc204ce8544ee98aed0119cbf5eec6a3618bf0db
> # Parent  e4c0713ea86204b900a2e1cce238f61e4bef2062
> bookmarks: adopt the formatter isplain method
> 
> Main hg repo changed the way formatter is supposed to be checked for
> plainness: previously people would run `if not fm`, thus relying on
> `__nonzero__` method, but now there is `isplain` method of formatter.
> Remotenames needs to adopt it.
> 
> diff --git a/remotenames.py b/remotenames.py
> --- a/remotenames.py
> +++ b/remotenames.py
> @@ -1047,7 +1047,7 @@ def displaylocalbookmarks(ui, repo, opts
> fm = ui.formatter('bookmarks', opts)
> hexfn = fm.hexfunc
> marks = repo._bookmarks
> -if len(marks) == 0 and not fm:
> +if len(marks) == 0 and (not fm or  fm.isplain()):
Nit: extra space here --^
Yeah, sorry for that. I expect that this can be fixed in-flight, but can resend.

pacem in terris / мир / शान्ति / ‎‫سَلاَم‬ / 平和
Kevin R. Bullock

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel



___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH evolve-ext] tests: adjust tests to respect cmd flag negation from core

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1475575213 25200
#  Tue Oct 04 03:00:13 2016 -0700
# Branch stable
# Node ID 322d537de6654e211c2f10e5c55d974fdc0c90b2
# Parent  8f902ec9ed9a296d92c2c2df1536af8c44b5321c
tests: adjust tests to respect cmd flag negation from core

diff --git a/tests/test-amend.t b/tests/test-amend.t
--- a/tests/test-amend.t
+++ b/tests/test-amend.t
@@ -142,20 +142,20 @@ Check the help
   
   options ([+] can be repeated):
   
-   -A --addremove   mark new/missing files as added/removed before
+   -A --[no-]addremove  mark new/missing files as added/removed before
 committing
-   -e --editinvoke editor on commit messages
-  --close-branchmark a branch as closed, hiding it from the branch
+   -e --[no-]edit   invoke editor on commit messages
+  --[no-]close-branch   mark a branch as closed, hiding it from the branch
 list
-   -s --secret  use the secret phase for committing
+   -s --[no-]secret use the secret phase for committing
-I --include PATTERN [+] include names matching the given patterns
-X --exclude PATTERN [+] exclude names matching the given patterns
-m --message TEXTuse text as commit message
-l --logfile FILEread commit message from file
-d --date DATE   record the specified date as commit date
-u --user USER   record the specified user as committer
-   -D --current-daterecord the current date as commit date
-   -U --current-userrecord the current user as committer
-   -i --interactive use interactive mode
+   -D --[no-]current-date   record the current date as commit date
+   -U --[no-]current-user   record the current user as committer
+   -i --[no-]interactiveuse interactive mode
   
   (some details hidden, use --verbose to show complete help)
diff --git a/tests/test-tutorial.t b/tests/test-tutorial.t
--- a/tests/test-tutorial.t
+++ b/tests/test-tutorial.t
@@ -457,7 +457,7 @@ of the `uncommit` command to splitting a
   
   options ([+] can be repeated):
   
-   -a --all uncommit all changes when no arguments given
+   -a --[no-]alluncommit all changes when no arguments given
-r --rev VALUE   revert commit content to REV instead
-I --include PATTERN [+] include names matching the given patterns
-X --exclude PATTERN [+] exclude names matching the given patterns
@@ -490,7 +490,7 @@ The tutorial part is not written yet but
   options ([+] can be repeated):
   
-r --rev VALUE [+] revision to fold
-  --exact only fold specified revisions
+  --[no-]exactonly fold specified revisions
-m --message TEXT  use text as commit message
-l --logfile FILE  read commit message from file
-d --date DATE record the specified date as commit date
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 8 shelve-ext] shelve: move mutableancestors to not be a closure

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478527237 28800
#  Mon Nov 07 06:00:37 2016 -0800
# Node ID 391086e9759e7787fbf8eb2b78202bfc2267245c
# Parent  b75505c45921802d9480f1629fca4fdd7f9394d3
shelve: move mutableancestors to not be a closure

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -273,24 +273,24 @@ def getshelvename(repo, parent, opts):
 raise error.Abort(_("shelved change names may not start with '.'"))
 return name
 
-def _docreatecmd(ui, repo, pats, opts):
-def mutableancestors(ctx):
-"""return all mutable ancestors for ctx (included)
+def mutableancestors(ctx):
+"""return all mutable ancestors for ctx (included)
 
-Much faster than the revset ancestors(ctx) & draft()"""
-seen = set([nodemod.nullrev])
-visit = collections.deque()
-visit.append(ctx)
-while visit:
-ctx = visit.popleft()
-yield ctx.node()
-for parent in ctx.parents():
-rev = parent.rev()
-if rev not in seen:
-seen.add(rev)
-if parent.mutable():
-visit.append(parent)
+Much faster than the revset ancestors(ctx) & draft()"""
+seen = set([nodemod.nullrev])
+visit = collections.deque()
+visit.append(ctx)
+while visit:
+ctx = visit.popleft()
+yield ctx.node()
+for parent in ctx.parents():
+rev = parent.rev()
+if rev not in seen:
+seen.add(rev)
+if parent.mutable():
+visit.append(parent)
 
+def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
 if len(parents) > 1:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 8 shelve-ext] shelve: move actual created commit shelving to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478543425 28800
#  Mon Nov 07 10:30:25 2016 -0800
# Node ID 914e385828e4597d1e1ae0a97d8cd24a32d8e029
# Parent  31ef2cba32928a93982ea6adea2d189030ce18bf
shelve: move actual created commit shelving to a separate function


Currently, this code does not have any branching, it just bundles
a commit and saves a patch file. Later, obsolescense-based shelve
will be added, so this code will also create some obsmarkers and
will be one of the few places where obsshelve will be different
from traditional shelve.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -330,6 +330,12 @@ def _nothingtoshelvemessaging(ui, repo, 
 else:
 ui.status(_("nothing changed\n"))
 
+def _shelvecreatedcommit(repo, node, name):
+bases = list(mutableancestors(repo[node]))
+shelvedfile(repo, name, 'hg').writebundle(bases, node)
+cmdutil.export(repo, [node],
+   fp=shelvedfile(repo, name, 'patch').opener('wb'),
+   opts=mdiff.diffopts(git=True))
 
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
@@ -382,12 +388,7 @@ def _docreatecmd(ui, repo, pats, opts):
 _nothingtoshelvemessaging(ui, repo, pats, opts)
 return 1
 
-bases = list(mutableancestors(repo[node]))
-shelvedfile(repo, name, 'hg').writebundle(bases, node)
-cmdutil.export(repo, [node],
-   fp=shelvedfile(repo, name, 'patch').opener('wb'),
-   opts=mdiff.diffopts(git=True))
-
+_shelvecreatedcommit(repo, node, name)
 
 if ui.formatted():
 desc = util.ellipsis(desc, ui.termwidth())
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 8 shelve-ext] shelve: move shelve name generation to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478527237 28800
#  Mon Nov 07 06:00:37 2016 -0800
# Node ID 334f462a1c345e5fa453ea641281ef9713789718
# Parent  b2d851fac63c8e12605948c7d182f86974b5096c
shelve: move shelve name generation to a separate function

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -243,6 +243,36 @@ def createcmd(ui, repo, pats, opts):
 cmdutil.checkunfinished(repo)
 return _docreatecmd(ui, repo, pats, opts)
 
+def getshelvename(repo, parent, opts):
+"""Decide on the name this shelve is going to have"""
+def gennames():
+yield label
+for i in xrange(1, 100):
+yield '%s-%02d' % (label, i)
+name = opts.get('name')
+label = repo._activebookmark or parent.branch() or 'default'
+# slashes aren't allowed in filenames, therefore we rename it
+label = label.replace('/', '_')
+
+if name:
+if shelvedfile(repo, name, 'hg').exists():
+e = _("a shelved change named '%s' already exists") % name
+raise error.Abort(e)
+else:
+for n in gennames():
+if not shelvedfile(repo, n, 'hg').exists():
+name = n
+break
+else:
+raise error.Abort(_("too many shelved changes named '%s'") % label)
+
+# ensure we are not creating a subdirectory or a hidden file
+if '/' in name or '\\' in name:
+raise error.Abort(_('shelved change names may not contain slashes'))
+if name.startswith('.'):
+raise error.Abort(_("shelved change names may not start with '.'"))
+return name
+
 def _docreatecmd(ui, repo, pats, opts):
 def mutableancestors(ctx):
 """return all mutable ancestors for ctx (included)
@@ -270,15 +300,6 @@ def _docreatecmd(ui, repo, pats, opts):
 
 # we never need the user, so we use a generic user for all shelve 
operations
 user = 'shelve@localhost'
-label = repo._activebookmark or parent.branch() or 'default'
-
-# slashes aren't allowed in filenames, therefore we rename it
-label = label.replace('/', '_')
-
-def gennames():
-yield label
-for i in xrange(1, 100):
-yield '%s-%02d' % (label, i)
 
 if parent.node() != nodemod.nullid:
 desc = "changes to: %s" % parent.description().split('\n', 1)[0]
@@ -287,8 +308,7 @@ def _docreatecmd(ui, repo, pats, opts):
 
 if not opts.get('message'):
 opts['message'] = desc
-
-name = opts.get('name')
+name = getshelvename(repo, parent, opts)
 
 lock = tr = None
 try:
@@ -298,24 +318,6 @@ def _docreatecmd(ui, repo, pats, opts):
 # pull races. ensure we don't print the abort message to stderr.
 tr = repo.transaction('commit', report=lambda x: None)
 
-if name:
-if shelvedfile(repo, name, 'hg').exists():
-raise error.Abort(_("a shelved change named '%s' already 
exists"
-   ) % name)
-else:
-for n in gennames():
-if not shelvedfile(repo, n, 'hg').exists():
-name = n
-break
-else:
-raise error.Abort(_("too many shelved changes named '%s'") %
- label)
-
-# ensure we are not creating a subdirectory or a hidden file
-if '/' in name or '\\' in name:
-raise error.Abort(_('shelved change names may not contain 
slashes'))
-if name.startswith('.'):
-raise error.Abort(_("shelved change names may not start with '.'"))
 interactive = opts.get('interactive', False)
 includeunknown = (opts.get('unknown', False) and
   not opts.get('addremove', False))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 8 shelve-ext] shelve: move argument processing out of the locked code

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478527237 28800
#  Mon Nov 07 06:00:37 2016 -0800
# Node ID b75505c45921802d9480f1629fca4fdd7f9394d3
# Parent  334f462a1c345e5fa453ea641281ef9713789718
shelve: move argument processing out of the locked code

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -298,7 +298,8 @@ def _docreatecmd(ui, repo, pats, opts):
 parent = parents[0]
 origbranch = wctx.branch()
 
-# we never need the user, so we use a generic user for all shelve 
operations
+# we never need the user, so we use a
+# generic user for all shelve operations
 user = 'shelve@localhost'
 
 if parent.node() != nodemod.nullid:
@@ -309,6 +310,9 @@ def _docreatecmd(ui, repo, pats, opts):
 if not opts.get('message'):
 opts['message'] = desc
 name = getshelvename(repo, parent, opts)
+interactive = opts.get('interactive', False)
+includeunknown = (opts.get('unknown', False) and
+  not opts.get('addremove', False))
 
 lock = tr = None
 try:
@@ -318,10 +322,6 @@ def _docreatecmd(ui, repo, pats, opts):
 # pull races. ensure we don't print the abort message to stderr.
 tr = repo.transaction('commit', report=lambda x: None)
 
-interactive = opts.get('interactive', False)
-includeunknown = (opts.get('unknown', False) and
-  not opts.get('addremove', False))
-
 extra={}
 if includeunknown:
 s = repo.status(match=scmutil.match(repo[None], pats, opts),
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 8 shelve-ext] shelve: move 'nothing changed' messaging to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478543425 28800
#  Mon Nov 07 10:30:25 2016 -0800
# Node ID 31ef2cba32928a93982ea6adea2d189030ce18bf
# Parent  30a215efa288dea9294407037ed962d9ec466b23
shelve: move 'nothing changed' messaging to a separate function

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -322,6 +322,15 @@ def getcommitfunc(extra, interactive, ed
 
 return interactivecommitfunc if interactive else commitfunc
 
+def _nothingtoshelvemessaging(ui, repo, pats, opts):
+stat = repo.status(match=scmutil.match(repo[None], pats, opts))
+if stat.deleted:
+ui.status(_("nothing changed (%d missing files, see "
+"'hg status')\n") % len(stat.deleted))
+else:
+ui.status(_("nothing changed\n"))
+
+
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
@@ -370,12 +379,7 @@ def _docreatecmd(ui, repo, pats, opts):
 node = cmdutil.dorecord(ui, repo, commitfunc, None,
 False, cmdutil.recordfilter, *pats, **opts)
 if not node:
-stat = repo.status(match=scmutil.match(repo[None], pats, opts))
-if stat.deleted:
-ui.status(_("nothing changed (%d missing files, see "
-"'hg status')\n") % len(stat.deleted))
-else:
-ui.status(_("nothing changed\n"))
+_nothingtoshelvemessaging(ui, repo, pats, opts)
 return 1
 
 bases = list(mutableancestors(repo[node]))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 8 of 8 shelve-ext] shelve: move unknown files handling to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478543425 28800
#  Mon Nov 07 10:30:25 2016 -0800
# Node ID 3a015218113bab2d079327c89609796392d37652
# Parent  914e385828e4597d1e1ae0a97d8cd24a32d8e029
shelve: move unknown files handling to a separate function

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -337,6 +337,13 @@ def _shelvecreatedcommit(repo, node, nam
fp=shelvedfile(repo, name, 'patch').opener('wb'),
opts=mdiff.diffopts(git=True))
 
+def _includeunknownfiles(repo, pats, opts, extra):
+s = repo.status(match=scmutil.match(repo[None], pats, opts),
+unknown=True)
+if s.unknown:
+extra['shelve_unknown'] = '\0'.join(s.unknown)
+repo[None].add(s.unknown)
+
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
@@ -365,13 +372,9 @@ def _docreatecmd(ui, repo, pats, opts):
 # pull races. ensure we don't print the abort message to stderr.
 tr = repo.transaction('commit', report=lambda x: None)
 
-extra={}
+extra = {}
 if includeunknown:
-s = repo.status(match=scmutil.match(repo[None], pats, opts),
-unknown=True)
-if s.unknown:
-extra['shelve_unknown'] = '\0'.join(s.unknown)
-repo[None].add(s.unknown)
+_includeunknownfiles(repo, pats, opts, extra)
 
 if _iswctxonnewbranch(repo) and not _isbareshelve(pats, opts):
 # In non-bare shelve we don't store newly created branch
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 8 shelve-ext] shelve: move possible shelve file extensions to a single place

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478529210 28800
#  Mon Nov 07 06:33:30 2016 -0800
# Node ID b2d851fac63c8e12605948c7d182f86974b5096c
# Parent  d500ddae7494772e5eb867fccc6876f5f0c21dac
shelve: move possible shelve file extensions to a single place

This and a couple of following patches are a preparation to
implementing obsolescense-enabled shelve which was discussed
on a Sprint. If this refactoring is not done, shelve is going
to look even more hackish than now.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -62,6 +62,7 @@ testedwith = 'ships-with-hg-core'
 
 backupdir = 'shelve-backup'
 shelvedir = 'shelved'
+shelvefileextensions = ['hg', 'patch']
 
 class shelvedfile(object):
 """Helper for the file storing a single shelve
@@ -221,7 +222,7 @@ def cleanupoldbackups(repo):
 # keep it, because timestamp can't decide exact order of backups
 continue
 base = f[:-3]
-for ext in 'hg patch'.split():
+for ext in shelvefileextensions:
 try:
 vfs.unlink(base + '.' + ext)
 except OSError as err:
@@ -399,7 +400,7 @@ def cleanupcmd(ui, repo):
 with repo.wlock():
 for (name, _type) in repo.vfs.readdir(shelvedir):
 suffix = name.rsplit('.', 1)[-1]
-if suffix in ('hg', 'patch'):
+if suffix in shelvefileextensions:
 shelvedfile(repo, name).movetobackup()
 cleanupoldbackups(repo)
 
@@ -410,8 +411,15 @@ def deletecmd(ui, repo, pats):
 with repo.wlock():
 try:
 for name in pats:
-for suffix in 'hg patch'.split():
-shelvedfile(repo, name, suffix).movetobackup()
+for suffix in shelvefileextensions:
+shfile = shelvedfile(repo, name, suffix)
+# patch file is necessary, as it should
+# be present for any kind of shelve,
+# but the .hg file is optional as in future we
+# will add obsolete shelve with does not create a
+# bundle
+if shfile.exists() or suffix == 'patch':
+shfile.movetobackup()
 cleanupoldbackups(repo)
 except OSError as err:
 if err.errno != errno.ENOENT:
@@ -557,7 +565,7 @@ def restorebranch(ui, repo, branchtorest
 def unshelvecleanup(ui, repo, name, opts):
 """remove related files after an unshelve"""
 if not opts.get('keep'):
-for filetype in 'hg patch'.split():
+for filetype in shelvefileextensions:
 shelvedfile(repo, name, filetype).movetobackup()
 cleanupoldbackups(repo)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 8 shelve-ext] shelve: move commitfunc creation to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478605764 28800
#  Tue Nov 08 03:49:24 2016 -0800
# Node ID 30a215efa288dea9294407037ed962d9ec466b23
# Parent  391086e9759e7787fbf8eb2b78202bfc2267245c
shelve: move commitfunc creation to a separate function

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -64,6 +64,11 @@ backupdir = 'shelve-backup'
 shelvedir = 'shelved'
 shelvefileextensions = ['hg', 'patch']
 
+# we never need the user, so we use a
+# generic user for all shelve operations
+shelveuser = 'shelve@localhost'
+
+
 class shelvedfile(object):
 """Helper for the file storing a single shelve
 
@@ -290,6 +295,33 @@ def mutableancestors(ctx):
 if parent.mutable():
 visit.append(parent)
 
+
+def getcommitfunc(extra, interactive, editor=False):
+def commitfunc(ui, repo, message, match, opts):
+hasmq = util.safehasattr(repo, 'mq')
+if hasmq:
+saved, repo.mq.checkapplied = repo.mq.checkapplied, False
+backup = repo.ui.backupconfig('phases', 'new-commit')
+try:
+repo.ui.setconfig('phases', 'new-commit', phases.secret)
+editor_ = False
+if editor:
+editor_ = cmdutil.getcommiteditor(editform='shelve.shelve',
+  **opts)
+return repo.commit(message, shelveuser, opts.get('date'), match,
+   editor=editor_, extra=extra)
+finally:
+repo.ui.restoreconfig(backup)
+if hasmq:
+repo.mq.checkapplied = saved
+
+def interactivecommitfunc(ui, repo, *pats, **opts):
+match = scmutil.match(repo['.'], pats, {})
+message = opts['message']
+return commitfunc(ui, repo, message, match, opts)
+
+return interactivecommitfunc if interactive else commitfunc
+
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
@@ -298,10 +330,6 @@ def _docreatecmd(ui, repo, pats, opts):
 parent = parents[0]
 origbranch = wctx.branch()
 
-# we never need the user, so we use a
-# generic user for all shelve operations
-user = 'shelve@localhost'
-
 if parent.node() != nodemod.nullid:
 desc = "changes to: %s" % parent.description().split('\n', 1)[0]
 else:
@@ -335,30 +363,11 @@ def _docreatecmd(ui, repo, pats, opts):
 # at bundled commit
 repo.dirstate.setbranch(repo['.'].branch())
 
-def commitfunc(ui, repo, message, match, opts):
-hasmq = util.safehasattr(repo, 'mq')
-if hasmq:
-saved, repo.mq.checkapplied = repo.mq.checkapplied, False
-backup = repo.ui.backupconfig('phases', 'new-commit')
-try:
-repo.ui. setconfig('phases', 'new-commit', phases.secret)
-editor = cmdutil.getcommiteditor(editform='shelve.shelve',
- **opts)
-return repo.commit(message, user, opts.get('date'), match,
-   editor=editor, extra=extra)
-finally:
-repo.ui.restoreconfig(backup)
-if hasmq:
-repo.mq.checkapplied = saved
-
-def interactivecommitfunc(ui, repo, *pats, **opts):
-match = scmutil.match(repo['.'], pats, {})
-message = opts['message']
-return commitfunc(ui, repo, message, match, opts)
+commitfunc = getcommitfunc(extra, interactive, editor=True)
 if not interactive:
 node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
 else:
-node = cmdutil.dorecord(ui, repo, interactivecommitfunc, None,
+node = cmdutil.dorecord(ui, repo, commitfunc, None,
 False, cmdutil.recordfilter, *pats, **opts)
 if not node:
 stat = repo.status(match=scmutil.match(repo[None], pats, opts))
@@ -739,21 +748,8 @@ def _dounshelve(ui, repo, *shelved, **op
 if s.modified or s.added or s.removed or s.deleted:
 ui.status(_("temporarily committing pending changes "
 "(restore with 'hg unshelve --abort')\n"))
-def commitfunc(ui, repo, message, match, opts):
-hasmq = util.safehasattr(repo, 'mq')
-if hasmq:
-saved, repo.mq.checkapplied = repo.mq.checkapplied, False
-
-backup = repo.ui.backupconfig('phases', 'new-commit')
-try:
-repo.ui

Re: [PATCH 1 of 8 shelve-ext] shelve: move possible shelve file extensions to a single place

[Kostia Balytskyi]

I have already found some bugs in these refactorings, which surface themselves 
when I am trying to add obsolescence-based shelve. I will fix those and send a 
v2. In a meantime, any comments are very welcome.

On 11/8/16, 1:51 PM, "Mercurial-devel on behalf of Kostia Balytskyi" 
 wrote:

# HG changeset patch
    # User Kostia Balytskyi 
# Date 1478529210 28800
#  Mon Nov 07 06:33:30 2016 -0800
# Node ID b2d851fac63c8e12605948c7d182f86974b5096c
# Parent  d500ddae7494772e5eb867fccc6876f5f0c21dac
shelve: move possible shelve file extensions to a single place

This and a couple of following patches are a preparation to
implementing obsolescense-enabled shelve which was discussed
on a Sprint. If this refactoring is not done, shelve is going
to look even more hackish than now.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -62,6 +62,7 @@ testedwith = 'ships-with-hg-core'
 
 backupdir = 'shelve-backup'
 shelvedir = 'shelved'
+shelvefileextensions = ['hg', 'patch']
 
 class shelvedfile(object):
 """Helper for the file storing a single shelve
@@ -221,7 +222,7 @@ def cleanupoldbackups(repo):
 # keep it, because timestamp can't decide exact order of 
backups
 continue
 base = f[:-3]
-for ext in 'hg patch'.split():
+for ext in shelvefileextensions:
 try:
 vfs.unlink(base + '.' + ext)
 except OSError as err:
@@ -399,7 +400,7 @@ def cleanupcmd(ui, repo):
 with repo.wlock():
 for (name, _type) in repo.vfs.readdir(shelvedir):
 suffix = name.rsplit('.', 1)[-1]
-if suffix in ('hg', 'patch'):
+if suffix in shelvefileextensions:
 shelvedfile(repo, name).movetobackup()
 cleanupoldbackups(repo)
 
@@ -410,8 +411,15 @@ def deletecmd(ui, repo, pats):
 with repo.wlock():
 try:
 for name in pats:
-for suffix in 'hg patch'.split():
-shelvedfile(repo, name, suffix).movetobackup()
+for suffix in shelvefileextensions:
+shfile = shelvedfile(repo, name, suffix)
+# patch file is necessary, as it should
+# be present for any kind of shelve,
+# but the .hg file is optional as in future we
+# will add obsolete shelve with does not create a
+# bundle
+if shfile.exists() or suffix == 'patch':
+shfile.movetobackup()
 cleanupoldbackups(repo)
 except OSError as err:
 if err.errno != errno.ENOENT:
@@ -557,7 +565,7 @@ def restorebranch(ui, repo, branchtorest
 def unshelvecleanup(ui, repo, name, opts):
 """remove related files after an unshelve"""
 if not opts.get('keep'):
-for filetype in 'hg patch'.split():
+for filetype in shelvefileextensions:
 shelvedfile(repo, name, filetype).movetobackup()
 cleanupoldbackups(repo)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel&d=DQIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=j4MF6YwF5GBNZqgBlHpqrc4maAhwOiM-cjjiuF_MQoE&s=6ECsPeS8UkS-4d7x6CRYFYeXVrlio1N5b8eBTUHIu3s&e=
 


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 8 shelve-ext] shelve: move shelve name generation to a separate function

[Kostia Balytskyi]

On 11/9/16, 10:01 AM, "Durham Goode"  wrote:
On 11/8/16 1:51 PM, Kostia Balytskyi wrote:
> # HG changeset patch
> # User Kostia Balytskyi 
> # Date 1478527237 28800
> #  Mon Nov 07 06:00:37 2016 -0800
> # Node ID 334f462a1c345e5fa453ea641281ef9713789718
> # Parent  b2d851fac63c8e12605948c7d182f86974b5096c
> shelve: move shelve name generation to a separate function
>
> diff --git a/hgext/shelve.py b/hgext/shelve.py
> --- a/hgext/shelve.py
> +++ b/hgext/shelve.py
> @@ -243,6 +243,36 @@ def createcmd(ui, repo, pats, opts):
>   cmdutil.checkunfinished(repo)
>   return _docreatecmd(ui, repo, pats, opts)
>   
> +def getshelvename(repo, parent, opts):
> +"""Decide on the name this shelve is going to have"""
> +def gennames():
> +yield label
> +for i in xrange(1, 100):
> +yield '%s-%02d' % (label, i)
> +name = opts.get('name')
> +label = repo._activebookmark or parent.branch() or 'default'
> +# slashes aren't allowed in filenames, therefore we rename it
> +label = label.replace('/', '_')
> +
> +if name:
> +if shelvedfile(repo, name, 'hg').exists():
> +e = _("a shelved change named '%s' already exists") % name
> +raise error.Abort(e)
> +else:
> +for n in gennames():
> +if not shelvedfile(repo, n, 'hg').exists():
> +name = n
> +break
> +else:
> +raise error.Abort(_("too many shelved changes named '%s'") % 
label)
> +
> +# ensure we are not creating a subdirectory or a hidden file
> +if '/' in name or '\\' in name:
> +raise error.Abort(_('shelved change names may not contain 
slashes'))
> +if name.startswith('.'):
> +raise error.Abort(_("shelved change names may not start with 
'.'"))
> +return name
> +
>   def _docreatecmd(ui, repo, pats, opts):
>   def mutableancestors(ctx):
>   """return all mutable ancestors for ctx (included)
> @@ -270,15 +300,6 @@ def _docreatecmd(ui, repo, pats, opts):
>   
>   # we never need the user, so we use a generic user for all shelve 
operations
>   user = 'shelve@localhost'
> -label = repo._activebookmark or parent.branch() or 'default'
> -
> -# slashes aren't allowed in filenames, therefore we rename it
> -label = label.replace('/', '_')
> -
> -def gennames():
> -yield label
> -for i in xrange(1, 100):
> -yield '%s-%02d' % (label, i)
>   
>   if parent.node() != nodemod.nullid:
>   desc = "changes to: %s" % parent.description().split('\n', 1)[0]
> @@ -287,8 +308,7 @@ def _docreatecmd(ui, repo, pats, opts):
>   
>   if not opts.get('message'):
>   opts['message'] = desc
> -
> -name = opts.get('name')
> +name = getshelvename(repo, parent, opts)
This moves the name generation from inside the lock to outside the 
lock.  In theory, this could mean two hg processes pick the same name 
and overwrite each other.  I'd just move this line to be inside the lock.
Good catch, I will move it back inside the lock.
>   
>   lock = tr = None
>   try:
> @@ -298,24 +318,6 @@ def _docreatecmd(ui, repo, pats, opts):
>   # pull races. ensure we don't print the abort message to stderr.
>   tr = repo.transaction('commit', report=lambda x: None)
>   
> -if name:
> -if shelvedfile(repo, name, 'hg').exists():
> -raise error.Abort(_("a shelved change named '%s' already 
exists"
> -   ) % name)
> -else:
> -for n in gennames():
> -if not shelvedfile(repo, n, 'hg').exists():
> -name = n
> -break
> -else:
> -raise error.Abort(_("too many shelved changes named 
'%s'") %
> - label)
> -
> -# ensure we are not creating a

Re: [PATCH 3 of 8 shelve-ext] shelve: move argument processing out of the locked code

[Kostia Balytskyi]

On 11/9/16, 10:04 AM, "Durham Goode"  wrote:

On 11/8/16 1:51 PM, Kostia Balytskyi wrote:

> # HG changeset patch
> # User Kostia Balytskyi 
> # Date 1478527237 28800
> #  Mon Nov 07 06:00:37 2016 -0800
> # Node ID b75505c45921802d9480f1629fca4fdd7f9394d3
> # Parent  334f462a1c345e5fa453ea641281ef9713789718
> shelve: move argument processing out of the locked code
I'd always mention why we're making this change, in addition to what the 
change is.  Otherwise this patch just looks like unnecessary code churn 
(though I'm sure there's a reason).
Well, there’s not a serious reason. I just thought that since I am doing 
large-ish shelve refactoring, I will move as many things out of lock as 
possible and cmd-option assigning to variables seemed like a good candidate. I 
will omit this commit in the v2.

>
> diff --git a/hgext/shelve.py b/hgext/shelve.py
> --- a/hgext/shelve.py
> +++ b/hgext/shelve.py
> @@ -298,7 +298,8 @@ def _docreatecmd(ui, repo, pats, opts):
>   parent = parents[0]
>   origbranch = wctx.branch()
>   
> -# we never need the user, so we use a generic user for all shelve 
operations
> +# we never need the user, so we use a
> +# generic user for all shelve operations
>   user = 'shelve@localhost'
>   
>   if parent.node() != nodemod.nullid:
> @@ -309,6 +310,9 @@ def _docreatecmd(ui, repo, pats, opts):
>   if not opts.get('message'):
>   opts['message'] = desc
>   name = getshelvename(repo, parent, opts)
> +interactive = opts.get('interactive', False)
> +includeunknown = (opts.get('unknown', False) and
> +  not opts.get('addremove', False))
>   
>   lock = tr = None
>   try:
> @@ -318,10 +322,6 @@ def _docreatecmd(ui, repo, pats, opts):
>   # pull races. ensure we don't print the abort message to stderr.
>   tr = repo.transaction('commit', report=lambda x: None)
>   
> -interactive = opts.get('interactive', False)
> -includeunknown = (opts.get('unknown', False) and
> -  not opts.get('addremove', False))
> -
>   extra={}
>   if includeunknown:
>   s = repo.status(match=scmutil.match(repo[None], pats, opts),
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> 
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel&d=DQIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=nuarHzhP1wi1T9iURRCj1A&m=32Le2MPDkI3ZXTxso0AkblXXgdoLzKA2dTGqmvQlpYY&s=RaUpD8EqP_sb6Dkg_j2H7-QE3P5a81HLM1xmz70IPl4&e=



___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 8 shelve-ext v2] shelve: move 'nothing changed' messaging to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478777581 28800
#  Thu Nov 10 03:33:01 2016 -0800
# Node ID 455f00c4d332ae42498a4089aa4a54f135c802ed
# Parent  5e827a6196fe903c16123068ce8b312e5840ae57
shelve: move 'nothing changed' messaging to a separate function

This has nothing to do with the future obsshelve implementation, I just
thought that moving this messaging to a separate function will improve
shelve code readability.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -320,6 +320,14 @@ def getcommitfunc(extra, interactive, ed
 
 return interactivecommitfunc if interactive else commitfunc
 
+def _nothingtoshelvemessaging(ui, repo, pats, opts):
+stat = repo.status(match=scmutil.match(repo[None], pats, opts))
+if stat.deleted:
+ui.status(_("nothing changed (%d missing files, see "
+"'hg status')\n") % len(stat.deleted))
+else:
+ui.status(_("nothing changed\n"))
+
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
@@ -369,12 +377,7 @@ def _docreatecmd(ui, repo, pats, opts):
 node = cmdutil.dorecord(ui, repo, commitfunc, None,
 False, cmdutil.recordfilter, *pats, **opts)
 if not node:
-stat = repo.status(match=scmutil.match(repo[None], pats, opts))
-if stat.deleted:
-ui.status(_("nothing changed (%d missing files, see "
-"'hg status')\n") % len(stat.deleted))
-else:
-ui.status(_("nothing changed\n"))
+_nothingtoshelvemessaging(ui, repo, pats, opts)
 return 1
 
 bases = list(mutableancestors(repo[node]))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 8 shelve-ext v2] shelve: move mutableancestors to not be a closure

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478777047 28800
#  Thu Nov 10 03:24:07 2016 -0800
# Node ID 98fdb6cda2b2cb38435560134c52a4ebb8859b45
# Parent  dc50b5aa86a26417f3f6dee97ea762da3d891f1d
shelve: move mutableancestors to not be a closure

There's no value in it being a closure and everyone who tries to read
the outer function code will be distracted by it. IMO moving it out
significantly improves readability, especially given how clear it is
what mutableancestors function does from its name.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -273,24 +273,24 @@ def getshelvename(repo, parent, opts):
 raise error.Abort(_("shelved change names may not start with '.'"))
 return name
 
-def _docreatecmd(ui, repo, pats, opts):
-def mutableancestors(ctx):
-"""return all mutable ancestors for ctx (included)
+def mutableancestors(ctx):
+"""return all mutable ancestors for ctx (included)
 
-Much faster than the revset ancestors(ctx) & draft()"""
-seen = set([nodemod.nullrev])
-visit = collections.deque()
-visit.append(ctx)
-while visit:
-ctx = visit.popleft()
-yield ctx.node()
-for parent in ctx.parents():
-rev = parent.rev()
-if rev not in seen:
-seen.add(rev)
-if parent.mutable():
-visit.append(parent)
+Much faster than the revset ancestors(ctx) & draft()"""
+seen = set([nodemod.nullrev])
+visit = collections.deque()
+visit.append(ctx)
+while visit:
+ctx = visit.popleft()
+yield ctx.node()
+for parent in ctx.parents():
+rev = parent.rev()
+if rev not in seen:
+seen.add(rev)
+if parent.mutable():
+visit.append(parent)
 
+def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
 if len(parents) > 1:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 7 of 8 shelve-ext v2] shelve: move unknown files handling to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478776828 28800
#  Thu Nov 10 03:20:28 2016 -0800
# Node ID d77543b3c0f6cd8dfc00be6060ef93743d98eeeb
# Parent  f2a82865a03b0f36478fae50f60b00aa64d806c2
shelve: move unknown files handling to a separate function

This change has nothing to do with future obsshelve introduction,
it is done just for readability purposes.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -335,6 +335,13 @@ def _shelvecreatedcommit(repo, node, nam
fp=shelvedfile(repo, name, 'patch').opener('wb'),
opts=mdiff.diffopts(git=True))
 
+def _includeunknownfiles(repo, pats, opts, extra):
+s = repo.status(match=scmutil.match(repo[None], pats, opts),
+unknown=True)
+if s.unknown:
+extra['shelve_unknown'] = '\0'.join(s.unknown)
+repo[None].add(s.unknown)
+
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
@@ -364,13 +371,9 @@ def _docreatecmd(ui, repo, pats, opts):
   not opts.get('addremove', False))
 
 name = getshelvename(repo, parent, opts)
-extra={}
+extra = {}
 if includeunknown:
-s = repo.status(match=scmutil.match(repo[None], pats, opts),
-unknown=True)
-if s.unknown:
-extra['shelve_unknown'] = '\0'.join(s.unknown)
-repo[None].add(s.unknown)
+_includeunknownfiles(repo, pats, opts, extra)
 
 if _iswctxonnewbranch(repo) and not _isbareshelve(pats, opts):
 # In non-bare shelve we don't store newly created branch
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 8 shelve-ext v2] shelve: move commitfunc creation to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478777191 28800
#  Thu Nov 10 03:26:31 2016 -0800
# Node ID 5e827a6196fe903c16123068ce8b312e5840ae57
# Parent  98fdb6cda2b2cb38435560134c52a4ebb8859b45
shelve: move commitfunc creation to a separate function

Special commitfuncs are created as closures at least twice in shelve's
code and one time special commitfunc is used within another closure.
They all serve very specific purposes like temporarily tweak some
configuration or enable editor, etc. This is not immediately important
to someone reading shelve code, so I think moving this logic to a separate
function is a good idea.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -64,6 +64,10 @@ backupdir = 'shelve-backup'
 shelvedir = 'shelved'
 shelvefileextensions = ['hg', 'patch']
 
+# we never need the user, so we use a
+# generic user for all shelve operations
+shelveuser = 'shelve@localhost'
+
 class shelvedfile(object):
 """Helper for the file storing a single shelve
 
@@ -290,6 +294,32 @@ def mutableancestors(ctx):
 if parent.mutable():
 visit.append(parent)
 
+def getcommitfunc(extra, interactive, editor=False):
+def commitfunc(ui, repo, message, match, opts):
+hasmq = util.safehasattr(repo, 'mq')
+if hasmq:
+saved, repo.mq.checkapplied = repo.mq.checkapplied, False
+backup = repo.ui.backupconfig('phases', 'new-commit')
+try:
+repo.ui.setconfig('phases', 'new-commit', phases.secret)
+editor_ = False
+if editor:
+editor_ = cmdutil.getcommiteditor(editform='shelve.shelve',
+  **opts)
+return repo.commit(message, shelveuser, opts.get('date'), match,
+   editor=editor_, extra=extra)
+finally:
+repo.ui.restoreconfig(backup)
+if hasmq:
+repo.mq.checkapplied = saved
+
+def interactivecommitfunc(ui, repo, *pats, **opts):
+match = scmutil.match(repo['.'], pats, {})
+message = opts['message']
+return commitfunc(ui, repo, message, match, opts)
+
+return interactivecommitfunc if interactive else commitfunc
+
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
@@ -298,9 +328,6 @@ def _docreatecmd(ui, repo, pats, opts):
 parent = parents[0]
 origbranch = wctx.branch()
 
-# we never need the user, so we use a generic user for all shelve 
operations
-user = 'shelve@localhost'
-
 if parent.node() != nodemod.nullid:
 desc = "changes to: %s" % parent.description().split('\n', 1)[0]
 else:
@@ -335,30 +362,11 @@ def _docreatecmd(ui, repo, pats, opts):
 # at bundled commit
 repo.dirstate.setbranch(repo['.'].branch())
 
-def commitfunc(ui, repo, message, match, opts):
-hasmq = util.safehasattr(repo, 'mq')
-if hasmq:
-saved, repo.mq.checkapplied = repo.mq.checkapplied, False
-backup = repo.ui.backupconfig('phases', 'new-commit')
-try:
-repo.ui. setconfig('phases', 'new-commit', phases.secret)
-editor = cmdutil.getcommiteditor(editform='shelve.shelve',
- **opts)
-return repo.commit(message, user, opts.get('date'), match,
-   editor=editor, extra=extra)
-finally:
-repo.ui.restoreconfig(backup)
-if hasmq:
-repo.mq.checkapplied = saved
-
-def interactivecommitfunc(ui, repo, *pats, **opts):
-match = scmutil.match(repo['.'], pats, {})
-message = opts['message']
-return commitfunc(ui, repo, message, match, opts)
+commitfunc = getcommitfunc(extra, interactive, editor=True)
 if not interactive:
 node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
 else:
-node = cmdutil.dorecord(ui, repo, interactivecommitfunc, None,
+node = cmdutil.dorecord(ui, repo, commitfunc, None,
 False, cmdutil.recordfilter, *pats, **opts)
 if not node:
 stat = repo.status(match=scmutil.match(repo[None], pats, opts))
@@ -741,21 +749,8 @@ def _dounshelve(ui, repo, *shelved, **op
 if s.modified or s.added or s.removed or s.deleted:
 ui.status(_("temporarily committing pending changes "
 "(restore with 'hg unshelve --abort')\n&qu

[PATCH 1 of 8 shelve-ext v2] shelve: move possible shelve file extensions to a single place

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478776040 28800
#  Thu Nov 10 03:07:20 2016 -0800
# Node ID ee1dc341d44f9408bdf960eb5a97abf1a1c69930
# Parent  494d5cec0b07653a6b5dec768dea92ffa987
shelve: move possible shelve file extensions to a single place

This and a couple of following patches are a preparation to
implementing obsolescense-enabled shelve which was discussed
on a Sprint. If this refactoring is not done, shelve is going
to look even more hackish than now.

This particular commit introduces a slight behavior change. Previously,
if only .hg/shelve/name.patch file exists, but .hg/name.hg does not,
'hg shelve -d name' would fail saying "shelve not found". Now deletion
will only fail if .patch file does not exist (since .patch is used
as an indicator of an existing shelve). Other shelve files being absent
are skipped silently to accommodate for future introduction of obs-based
shelve, which will mean that for some shelves .hg and .patch files exist,
while for others .hg and .oshelve.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -62,6 +62,7 @@ testedwith = 'ships-with-hg-core'
 
 backupdir = 'shelve-backup'
 shelvedir = 'shelved'
+shelvefileextensions = ['hg', 'patch']
 
 class shelvedfile(object):
 """Helper for the file storing a single shelve
@@ -221,7 +222,7 @@ def cleanupoldbackups(repo):
 # keep it, because timestamp can't decide exact order of backups
 continue
 base = f[:-3]
-for ext in 'hg patch'.split():
+for ext in shelvefileextensions:
 try:
 vfs.unlink(base + '.' + ext)
 except OSError as err:
@@ -399,7 +400,7 @@ def cleanupcmd(ui, repo):
 with repo.wlock():
 for (name, _type) in repo.vfs.readdir(shelvedir):
 suffix = name.rsplit('.', 1)[-1]
-if suffix in ('hg', 'patch'):
+if suffix in shelvefileextensions:
 shelvedfile(repo, name).movetobackup()
 cleanupoldbackups(repo)
 
@@ -410,8 +411,15 @@ def deletecmd(ui, repo, pats):
 with repo.wlock():
 try:
 for name in pats:
-for suffix in 'hg patch'.split():
-shelvedfile(repo, name, suffix).movetobackup()
+for suffix in shelvefileextensions:
+shfile = shelvedfile(repo, name, suffix)
+# patch file is necessary, as it should
+# be present for any kind of shelve,
+# but the .hg file is optional as in future we
+# will add obsolete shelve with does not create a
+# bundle
+if shfile.exists() or suffix == 'patch':
+shfile.movetobackup()
 cleanupoldbackups(repo)
 except OSError as err:
 if err.errno != errno.ENOENT:
@@ -557,8 +565,10 @@ def restorebranch(ui, repo, branchtorest
 def unshelvecleanup(ui, repo, name, opts):
 """remove related files after an unshelve"""
 if not opts.get('keep'):
-for filetype in 'hg patch'.split():
-shelvedfile(repo, name, filetype).movetobackup()
+for filetype in shelvefileextensions:
+shfile = shelvedfile(repo, name, filetype)
+if shfile.exists():
+shfile.movetobackup()
 cleanupoldbackups(repo)
 
 def unshelvecontinue(ui, repo, state, opts):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 8 shelve-ext v2] shelve: move shelve name generation to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478776975 28800
#  Thu Nov 10 03:22:55 2016 -0800
# Node ID dc50b5aa86a26417f3f6dee97ea762da3d891f1d
# Parent  ee1dc341d44f9408bdf960eb5a97abf1a1c69930
shelve: move shelve name generation to a separate function


This has nothing to do with future obsshelve introduction, done just
for readability pursposes.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -243,6 +243,36 @@ def createcmd(ui, repo, pats, opts):
 cmdutil.checkunfinished(repo)
 return _docreatecmd(ui, repo, pats, opts)
 
+def getshelvename(repo, parent, opts):
+"""Decide on the name this shelve is going to have"""
+def gennames():
+yield label
+for i in xrange(1, 100):
+yield '%s-%02d' % (label, i)
+name = opts.get('name')
+label = repo._activebookmark or parent.branch() or 'default'
+# slashes aren't allowed in filenames, therefore we rename it
+label = label.replace('/', '_')
+
+if name:
+if shelvedfile(repo, name, 'hg').exists():
+e = _("a shelved change named '%s' already exists") % name
+raise error.Abort(e)
+else:
+for n in gennames():
+if not shelvedfile(repo, n, 'hg').exists():
+name = n
+break
+else:
+raise error.Abort(_("too many shelved changes named '%s'") % label)
+
+# ensure we are not creating a subdirectory or a hidden file
+if '/' in name or '\\' in name:
+raise error.Abort(_('shelved change names may not contain slashes'))
+if name.startswith('.'):
+raise error.Abort(_("shelved change names may not start with '.'"))
+return name
+
 def _docreatecmd(ui, repo, pats, opts):
 def mutableancestors(ctx):
 """return all mutable ancestors for ctx (included)
@@ -270,15 +300,6 @@ def _docreatecmd(ui, repo, pats, opts):
 
 # we never need the user, so we use a generic user for all shelve 
operations
 user = 'shelve@localhost'
-label = repo._activebookmark or parent.branch() or 'default'
-
-# slashes aren't allowed in filenames, therefore we rename it
-label = label.replace('/', '_')
-
-def gennames():
-yield label
-for i in xrange(1, 100):
-yield '%s-%02d' % (label, i)
 
 if parent.node() != nodemod.nullid:
 desc = "changes to: %s" % parent.description().split('\n', 1)[0]
@@ -288,8 +309,6 @@ def _docreatecmd(ui, repo, pats, opts):
 if not opts.get('message'):
 opts['message'] = desc
 
-name = opts.get('name')
-
 lock = tr = None
 try:
 lock = repo.lock()
@@ -298,28 +317,11 @@ def _docreatecmd(ui, repo, pats, opts):
 # pull races. ensure we don't print the abort message to stderr.
 tr = repo.transaction('commit', report=lambda x: None)
 
-if name:
-if shelvedfile(repo, name, 'hg').exists():
-raise error.Abort(_("a shelved change named '%s' already 
exists"
-   ) % name)
-else:
-for n in gennames():
-if not shelvedfile(repo, n, 'hg').exists():
-name = n
-break
-else:
-raise error.Abort(_("too many shelved changes named '%s'") %
- label)
-
-# ensure we are not creating a subdirectory or a hidden file
-if '/' in name or '\\' in name:
-raise error.Abort(_('shelved change names may not contain 
slashes'))
-if name.startswith('.'):
-raise error.Abort(_("shelved change names may not start with '.'"))
 interactive = opts.get('interactive', False)
 includeunknown = (opts.get('unknown', False) and
   not opts.get('addremove', False))
 
+name = getshelvename(repo, parent, opts)
 extra={}
 if includeunknown:
 s = repo.status(match=scmutil.match(repo[None], pats, opts),
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 6 of 8 shelve-ext v2] shelve: move actual created commit shelving to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478776040 28800
#  Thu Nov 10 03:07:20 2016 -0800
# Node ID f2a82865a03b0f36478fae50f60b00aa64d806c2
# Parent  455f00c4d332ae42498a4089aa4a54f135c802ed
shelve: move actual created commit shelving to a separate function


Currently, this code does not have any branching, it just bundles
a commit and saves a patch file. Later, obsolescense-based shelve
will be added, so this code will also create some obsmarkers and
will be one of the few places where obsshelve will be different
from traditional shelve.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -328,6 +328,13 @@ def _nothingtoshelvemessaging(ui, repo, 
 else:
 ui.status(_("nothing changed\n"))
 
+def _shelvecreatedcommit(repo, node, name):
+bases = list(mutableancestors(repo[node]))
+shelvedfile(repo, name, 'hg').writebundle(bases, node)
+cmdutil.export(repo, [node],
+   fp=shelvedfile(repo, name, 'patch').opener('wb'),
+   opts=mdiff.diffopts(git=True))
+
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
@@ -380,12 +387,7 @@ def _docreatecmd(ui, repo, pats, opts):
 _nothingtoshelvemessaging(ui, repo, pats, opts)
 return 1
 
-bases = list(mutableancestors(repo[node]))
-shelvedfile(repo, name, 'hg').writebundle(bases, node)
-cmdutil.export(repo, [node],
-   fp=shelvedfile(repo, name, 'patch').opener('wb'),
-   opts=mdiff.diffopts(git=True))
-
+_shelvecreatedcommit(repo, node, name)
 
 if ui.formatted():
 desc = util.ellipsis(desc, ui.termwidth())
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 8 of 8 shelve-ext v2] shelve: move shelve-finishing logic to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478776541 28800
#  Thu Nov 10 03:15:41 2016 -0800
# Node ID 2188194ca1ee86953855e0d2fb9396ec18636ed9
# Parent  d77543b3c0f6cd8dfc00be6060ef93743d98eeeb
shelve: move shelve-finishing logic to a separate function

With future obs-based shelve, finishing shelve will be different
from just aborting a transaction and I would like to keep both
variants of this functionality in a separate function.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -342,6 +342,9 @@ def _includeunknownfiles(repo, pats, opt
 extra['shelve_unknown'] = '\0'.join(s.unknown)
 repo[None].add(s.unknown)
 
+def _finishshelve(repo):
+_aborttransaction(repo)
+
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
 parents = wctx.parents()
@@ -399,7 +402,7 @@ def _docreatecmd(ui, repo, pats, opts):
 if origbranch != repo['.'].branch() and not _isbareshelve(pats, opts):
 repo.dirstate.setbranch(origbranch)
 
-_aborttransaction(repo)
+_finishshelve(repo)
 finally:
 lockmod.release(tr, lock)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] rebase: update _rebaseset inside transaction

[Kostia Balytskyi]

This looks good to me.


On 11/10/16, 11:56 AM, "Mercurial-devel on behalf of Durham Goode" 
 wrote:

# HG changeset patch
# User Durham Goode 
# Date 1478778918 28800
#  Thu Nov 10 03:55:18 2016 -0800
# Node ID f3bfa374cea66cd5bf17645ccb6eef60c4d8a5aa
# Parent  3fd53cc1aad882ac9191d735acdbbc2d7103
rebase: update _rebaseset inside transaction

Rebase has the concept of the _rebaseset, which is used to prevent the 
commits
being rebased from being hidden until after the rebase is complete. 
Previously,
the _rebaseset was being cleared at the very end of rebase, after the last
transaction had already closed. This meant that the repo filteredrevs cache 
was
not updated (since no invalidation had been triggered, since we're outside 
of a
transaction), which meant future changelog reads saw commits as visible that
should've been hidden.

This patch moves the _rebaseset clearing to be inside the last rebase
transaction, so that after the transaction the filteredrevs is appropriately
invalidated and will be up-to-date on the next read.

This showed up in an extension that combines rebase, obsolete, and inhibit, 
so
I'm not sure how to create a test case in hg core to repro it. But we have a
test case in the extension repo.

diff --git a/hgext/rebase.py b/hgext/rebase.py
--- a/hgext/rebase.py
+++ b/hgext/rebase.py
@@ -495,6 +495,12 @@ class rebaseruntime(object):
 if self.activebookmark not in repo._bookmarks:
 # active bookmark was divergent one and has been 
deleted
 self.activebookmark = None
+# Clear the _rebaseset as part of the transaction, so
+# post-transaction hooks can see the final repo visibility 
state.
+# (We could've moved clearstatus() into the transaction to do 
this,
+# but it contains non-transactional changes, like unlinking 
files).
+_clearrebasesetvisibiliy(repo)
+
 clearstatus(repo)
 clearcollapsemsg(repo)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel&d=DQIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=xBhMhQEBjcvk2_j-7abA2vhcPvMIrWwmaYKktXxNFrE&s=EeMzphKg1_owVFcLHqcurMoyC7PVMQvZB97lXb7W9f8&e=
 


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 5 shelve-ext] shelve: move rebasing logic to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478804230 28800
#  Thu Nov 10 10:57:10 2016 -0800
# Node ID 36d052f32bf4efe1f17b8cf3cf022b1f032298f2
# Parent  bce5daba05f23d87b3231c7dd28014f881b4de32
shelve: move rebasing logic to a separate function

Rebasing restored shelved commit onto the right destination is done
differently in traditional and obs-based unshelve:
- for traditional, we just rebase it
- for obs-based, we need to check whether a successor of
  the restored commit already exists in the destination (this
  might happen when unshelving twice on the same destination)
This is the reason why this piece of logic should be in its own
function: to not have excessive complexity in the main function.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -659,6 +659,44 @@ def _unshelverestorecommit(ui, repo, bas
 ui.quiet = oldquiet
 return repo, shelvectx
 
+def _rebaserestoredcommit(ui, repo, opts, tr, oldtiprev, basename, pctx,
+  tmpwctx, shelvectx, branchtorestore):
+"""Rebase restored commit from its original location to a destination"""
+# If the shelve is not immediately on top of the commit
+# we'll be merging with, rebase it to be on top.
+if tmpwctx.node() == shelvectx.parents()[0].node():
+return shelvectx
+
+ui.status(_('rebasing shelved changes\n'))
+try:
+rebase.rebase(ui, repo, **{
+'rev': [shelvectx.rev()],
+'dest': str(tmpwctx.rev()),
+'keep': True,
+'tool': opts.get('tool', ''),
+})
+except error.InterventionRequired:
+tr.close()
+
+stripnodes = [repo.changelog.node(rev)
+  for rev in xrange(oldtiprev, len(repo))]
+shelvedstate.save(repo, basename, pctx, tmpwctx, stripnodes,
+  branchtorestore)
+
+util.rename(repo.join('rebasestate'),
+repo.join('unshelverebasestate'))
+raise error.InterventionRequired(
+_("unresolved conflicts (see 'hg resolve', then "
+  "'hg unshelve --continue')"))
+
+# refresh ctx after rebase completes
+shelvectx = repo['tip']
+
+if not shelvectx in tmpwctx.children():
+# rebase was a no-op, so it produced no child commit
+shelvectx = tmpwctx
+return shelvectx
+
 @command('unshelve',
  [('a', 'abort', None,
_('abort an incomplete unshelve operation')),
@@ -790,38 +828,9 @@ def _dounshelve(ui, repo, *shelved, **op
 if shelvectx.branch() != shelvectx.p1().branch():
 branchtorestore = shelvectx.branch()
 
-# If the shelve is not immediately on top of the commit
-# we'll be merging with, rebase it to be on top.
-if tmpwctx.node() != shelvectx.parents()[0].node():
-ui.status(_('rebasing shelved changes\n'))
-try:
-rebase.rebase(ui, repo, **{
-'rev' : [shelvectx.rev()],
-'dest' : str(tmpwctx.rev()),
-'keep' : True,
-'tool' : opts.get('tool', ''),
-})
-except error.InterventionRequired:
-tr.close()
-
-stripnodes = [repo.changelog.node(rev)
-  for rev in xrange(oldtiprev, len(repo))]
-shelvedstate.save(repo, basename, pctx, tmpwctx, stripnodes,
-  branchtorestore)
-
-util.rename(repo.join('rebasestate'),
-repo.join('unshelverebasestate'))
-raise error.InterventionRequired(
-_("unresolved conflicts (see 'hg resolve', then "
-  "'hg unshelve --continue')"))
-
-# refresh ctx after rebase completes
-shelvectx = repo['tip']
-
-if not shelvectx in tmpwctx.children():
-# rebase was a no-op, so it produced no child commit
-shelvectx = tmpwctx
-
+shelvectx = _rebaserestoredcommit(ui, repo, opts, tr, oldtiprev,
+  basename, pctx, tmpwctx, shelvectx,
+  branchtorestore)
 mergefiles(ui, repo, pctx, shelvectx)
 restorebranch(ui, repo, branchtorestore)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 5 shelve-ext] shelve: move unshelve-finishing logic to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478876487 28800
#  Fri Nov 11 07:01:27 2016 -0800
# Node ID c34ee36f4e7fe6b04ea1b4e2f032e995fe65d6c5
# Parent  dce4581dcae386c3d3b420911350d176c0423520
shelve: move unshelve-finishing logic to a separate function

Finishing unshelve involves two steps now:
- stripping a changelog
- aborting a transaction
Obs-based shelve will not require these things, so isolating this logic
into a separate function where the normal/obs-shelve branching is
going to be implemented seems to be like a nice idea.

Behavior-wise this change moves 'unshelvecleanup' from being between
changelog stripping and transaction abortion to being after them.
I don't think this has any negative effects.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -708,6 +708,14 @@ def _forgetunknownfiles(repo, shelvectx,
 toforget = (addedafter & shelveunknown) - addedbefore
 repo[None].forget(toforget)
 
+def _finishunshelve(repo, oldtiprev, tr):
+# The transaction aborting will strip all the commits for us,
+# but it doesn't update the inmemory structures, so addchangegroup
+# hooks still fire and try to operate on the missing commits.
+# Clean up manually to prevent this.
+repo.unfiltered().changelog.strip(oldtiprev, tr)
+_aborttransaction(repo)
+
 @command('unshelve',
  [('a', 'abort', None,
_('abort an incomplete unshelve operation')),
@@ -847,16 +855,8 @@ def _dounshelve(ui, repo, *shelved, **op
 _forgetunknownfiles(repo, shelvectx, addedbefore)
 
 shelvedstate.clear(repo)
-
-# The transaction aborting will strip all the commits for us,
-# but it doesn't update the inmemory structures, so addchangegroup
-# hooks still fire and try to operate on the missing commits.
-# Clean up manually to prevent this.
-repo.unfiltered().changelog.strip(oldtiprev, tr)
-
+_finishunshelve(repo, oldtiprev, tr)
 unshelvecleanup(ui, repo, basename, opts)
-
-_aborttransaction(repo)
 finally:
 ui.quiet = oldquiet
 if tr:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 5 shelve-ext] shelve: move commit restoration logic to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478803866 28800
#  Thu Nov 10 10:51:06 2016 -0800
# Node ID bce5daba05f23d87b3231c7dd28014f881b4de32
# Parent  45dbc9a803375958310bced301227b02802372b0
shelve: move commit restoration logic to a separate function

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -651,6 +651,14 @@ def _commitworkingcopychanges(ui, repo, 
 tmpwctx = repo[node]
 return tmpwctx, addedbefore
 
+def _unshelverestorecommit(ui, repo, basename, oldquiet):
+"""Recreate commit in the repository during the unshelve"""
+ui.quiet = True
+shelvedfile(repo, basename, 'hg').applybundle()
+shelvectx = repo['tip']
+ui.quiet = oldquiet
+return repo, shelvectx
+
 @command('unshelve',
  [('a', 'abort', None,
_('abort an incomplete unshelve operation')),
@@ -776,12 +784,7 @@ def _dounshelve(ui, repo, *shelved, **op
 tmpwctx, addedbefore = _commitworkingcopychanges(ui, repo, opts,
  tmpwctx)
 
-ui.quiet = True
-shelvedfile(repo, basename, 'hg').applybundle()
-
-ui.quiet = oldquiet
-
-shelvectx = repo['tip']
+repo, shelvectx = _unshelverestorecommit(ui, repo, basename, oldquiet)
 
 branchtorestore = ''
 if shelvectx.branch() != shelvectx.p1().branch():
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 5 shelve-ext] shelve: move temporary commit creation to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1479036952 28800
#  Sun Nov 13 03:35:52 2016 -0800
# Node ID 45dbc9a803375958310bced301227b02802372b0
# Parent  2188194ca1ee86953855e0d2fb9396ec18636ed9
shelve: move temporary commit creation to a separate function

Committing working copy changes before rebasing a shelved commit
on top of them is an independent piece of behavior, which fits
into its own function.

Similar to the previous serier, this and a couple of following
patches are for unshelve refactoring.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -631,6 +631,26 @@ def unshelvecontinue(ui, repo, state, op
 unshelvecleanup(ui, repo, state.name, opts)
 ui.status(_("unshelve of '%s' complete\n") % state.name)
 
+def _commitworkingcopychanges(ui, repo, opts, tmpwctx):
+"""Temporarily commit working copy changes before moving unshelve commit"""
+# Store pending changes in a commit and remember added in case a shelve
+# contains unknown files that are part of the pending change
+s = repo.status()
+addedbefore = frozenset(s.added)
+if not (s.modified or s.added or s.removed or s.deleted):
+return tmpwctx, addedbefore
+ui.status(_("temporarily committing pending changes "
+"(restore with 'hg unshelve --abort')\n"))
+commitfunc = getcommitfunc(extra=None, interactive=False,
+   editor=False)
+tempopts = {}
+tempopts['message'] = "pending changes temporary commit"
+tempopts['date'] = opts.get('date')
+ui.quiet = True
+node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
+tmpwctx = repo[node]
+return tmpwctx, addedbefore
+
 @command('unshelve',
  [('a', 'abort', None,
_('abort an incomplete unshelve operation')),
@@ -753,21 +773,8 @@ def _dounshelve(ui, repo, *shelved, **op
 # and shelvectx is the unshelved changes. Then we merge it all down
 # to the original pctx.
 
-# Store pending changes in a commit and remember added in case a shelve
-# contains unknown files that are part of the pending change
-s = repo.status()
-addedbefore = frozenset(s.added)
-if s.modified or s.added or s.removed or s.deleted:
-ui.status(_("temporarily committing pending changes "
-"(restore with 'hg unshelve --abort')\n"))
-commitfunc = getcommitfunc(extra=None, interactive=False,
-   editor=False)
-tempopts = {}
-tempopts['message'] = "pending changes temporary commit"
-tempopts['date'] = opts.get('date')
-ui.quiet = True
-node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
-tmpwctx = repo[node]
+tmpwctx, addedbefore = _commitworkingcopychanges(ui, repo, opts,
+ tmpwctx)
 
 ui.quiet = True
 shelvedfile(repo, basename, 'hg').applybundle()
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 5 shelve-ext] shelve: move file-forgetting logic to a separate function

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1478804559 28800
#  Thu Nov 10 11:02:39 2016 -0800
# Node ID dce4581dcae386c3d3b420911350d176c0423520
# Parent  36d052f32bf4efe1f17b8cf3cf022b1f032298f2
shelve: move file-forgetting logic to a separate function

This is just a readability improvement.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -697,6 +697,17 @@ def _rebaserestoredcommit(ui, repo, opts
 shelvectx = tmpwctx
 return shelvectx
 
+def _forgetunknownfiles(repo, shelvectx, addedbefore):
+# Forget any files that were unknown before the shelve, unknown before
+# unshelve started, but are now added.
+shelveunknown = shelvectx.extra().get('shelve_unknown')
+if not shelveunknown:
+return
+shelveunknown = frozenset(shelveunknown.split('\0'))
+addedafter = frozenset(repo.status().added)
+toforget = (addedafter & shelveunknown) - addedbefore
+repo[None].forget(toforget)
+
 @command('unshelve',
  [('a', 'abort', None,
_('abort an incomplete unshelve operation')),
@@ -833,15 +844,7 @@ def _dounshelve(ui, repo, *shelved, **op
   branchtorestore)
 mergefiles(ui, repo, pctx, shelvectx)
 restorebranch(ui, repo, branchtorestore)
-
-# Forget any files that were unknown before the shelve, unknown before
-# unshelve started, but are now added.
-shelveunknown = shelvectx.extra().get('shelve_unknown')
-if shelveunknown:
-shelveunknown = frozenset(shelveunknown.split('\0'))
-addedafter = frozenset(repo.status().added)
-toforget = (addedafter & shelveunknown) - addedbefore
-repo[None].forget(toforget)
+_forgetunknownfiles(repo, shelvectx, addedbefore)
 
 shelvedstate.clear(repo)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] conflicts: make spacing consistent in conflict markers

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1479598897 28800
#  Sat Nov 19 15:41:37 2016 -0800
# Node ID 5f03a63d6def1cf3c34fa7f8979c34151a9ba660
# Parent  0c3a8b6239db2c0d6a0e9ddea2266445bca2d440
conflicts: make spacing consistent in conflict markers

The way default marker template was defined before this patch,
the spacing before dash in conflict markes was dependent on
whether changeset is a tip one or not. This is a relevant part
of template:
'{ifeq(tags, "tip", "", "{tags} "}'
If revision is a tip revision with no other tags, this would
resolve to an empty string, but for revisions which are not tip
and don't have any other tags, this would resolve to a single
space string. In the end this causes weirdnesses like the ones
you can see in the affected tests.

This is a not a big deal, but double spacing may be visually
less pleasant.

Please note that test changes where commit hashes change are
the result of marking files as resolved without removing markers.

diff --git a/mercurial/filemerge.py b/mercurial/filemerge.py
--- a/mercurial/filemerge.py
+++ b/mercurial/filemerge.py
@@ -518,7 +518,8 @@ def _formatconflictmarker(repo, ctx, tem
 return util.ellipsis(mark, 80 - 8)
 
 _defaultconflictmarker = ('{node|short} '
-  '{ifeq(tags, "tip", "", "{tags} ")}'
+  '{ifeq(tags, "tip", "", '
+   'ifeq(tags, "", "", "{tags} "))}'
   '{if(bookmarks, "{bookmarks} ")}'
   '{ifeq(branch, "default", "", "{branch} ")}'
   '- {author|user}: {desc|firstline}')
diff --git a/tests/test-commit-amend.t b/tests/test-commit-amend.t
--- a/tests/test-commit-amend.t
+++ b/tests/test-commit-amend.t
@@ -638,7 +638,7 @@ Amend a merge changeset (with renames an
   (no more unresolved files)
   $ hg ci -m 'merge bar'
   $ hg log --config diff.git=1 -pr .
-  changeset:   23:69c24fe01e35
+  changeset:   23:163cfd7219f7
   tag: tip
   parent:  22:30d96aeaf27b
   parent:  21:1aa437659d19
@@ -657,7 +657,7 @@ Amend a merge changeset (with renames an
dd
   +===
   +cc
-  +>>>>>>> merge rev:1aa437659d19  bar - test: aazzcc
+  +>>>>>>> merge rev:1aa437659d19 bar - test: aazzcc
   diff --git a/z b/zz
   rename from z
   rename to zz
@@ -671,7 +671,7 @@ Amend a merge changeset (with renames an
   $ HGEDITOR="sh .hg/checkeditform.sh" hg ci --amend -m 'merge bar (amend 
message)' --edit
   HGEDITFORM=commit.amend.merge
   $ hg log --config diff.git=1 -pr .
-  changeset:   24:cfa2fbef3169
+  changeset:   24:bca52d4ed186
   tag: tip
   parent:  22:30d96aeaf27b
   parent:  21:1aa437659d19
@@ -690,7 +690,7 @@ Amend a merge changeset (with renames an
dd
   +===
   +cc
-  +>>>>>>> merge rev:1aa437659d19  bar - test: aazzcc
+  +>>>>>>> merge rev:1aa437659d19 bar - test: aazzcc
   diff --git a/z b/zz
   rename from z
   rename to zz
@@ -704,7 +704,7 @@ Amend a merge changeset (with renames an
   $ hg mv zz z
   $ hg ci --amend -m 'merge bar (undo rename)'
   $ hg log --config diff.git=1 -pr .
-  changeset:   26:c34de68b014c
+  changeset:   26:12594a98ca3f
   tag: tip
   parent:  22:30d96aeaf27b
   parent:  21:1aa437659d19
@@ -723,7 +723,7 @@ Amend a merge changeset (with renames an
dd
   +===
   +cc
-  +>>>>>>> merge rev:1aa437659d19  bar - test: aazzcc
+  +>>>>>>> merge rev:1aa437659d19 bar - test: aazzcc
   
   $ hg debugrename z
   z not renamed
@@ -740,9 +740,9 @@ Amend a merge changeset (with renames du
   $ echo aa >> aaa
   $ hg ci -m 'merge bar again'
   $ hg log --config diff.git=1 -pr .
-  changeset:   28:37d40dcef03b
+  changeset:   28:dffde028b388
   tag: tip
-  parent:  26:c34de68b014c
+  parent:  26:12594a98ca3f
   parent:  27:4c94d5bc65f5
   user:test
   date:Thu Jan 01 00:00:00 1970 +
@@ -775,9 +775,9 @@ Amend a merge changeset (with renames du
   $ hg mv aaa aa
   $ hg ci --amend -m 'merge bar again (undo rename)'
   $ hg log --config diff.git=1 -pr .
-  changeset:   30:537c6d1b3633
+  changeset:   30:18e3ba160489
   tag: tip
-  parent:  26:c34de68b014c
+  parent:  26:12594a98ca3f
   parent:  27:4c94d5bc65f5
   user:test
   date:Thu Jan 01 00:00:00 1970 +
@@ -817,9 +817,9 @@ Amend a merge changeset (with manifest-l
   use (c)hanged version, (d)elete, or leave (u)nresolved? c
   $ hg ci -m 'merge bar (with conflicts)'
   $ hg log --config diff.git=1 -pr .
-  changeset:  

Re: [PATCH 1 of 5 shelve-ext] shelve: move temporary commit creation to a separate function

[Kostia Balytskyi]

On 11/19/16, 10:02 AM, "Yuya Nishihara"  wrote:

On Sun, 13 Nov 2016 03:39:34 -0800, Kostia Balytskyi wrote:
> # HG changeset patch
> # User Kostia Balytskyi 
> # Date 1479036952 28800
> #  Sun Nov 13 03:35:52 2016 -0800
> # Node ID 45dbc9a803375958310bced301227b02802372b0
> # Parent  2188194ca1ee86953855e0d2fb9396ec18636ed9
> shelve: move temporary commit creation to a separate function

Looks all good. Queued them, thanks.

> +def _commitworkingcopychanges(ui, repo, opts, tmpwctx):
> +"""Temporarily commit working copy changes before moving unshelve 
commit"""
> +# Store pending changes in a commit and remember added in case a 
shelve
> +# contains unknown files that are part of the pending change
> +s = repo.status()
> +addedbefore = frozenset(s.added)
> +if not (s.modified or s.added or s.removed or s.deleted):
> +return tmpwctx, addedbefore
> +ui.status(_("temporarily committing pending changes "
> +"(restore with 'hg unshelve --abort')\n"))
> +commitfunc = getcommitfunc(extra=None, interactive=False,
> +   editor=False)
> +tempopts = {}
> +tempopts['message'] = "pending changes temporary commit"
> +tempopts['date'] = opts.get('date')
> +ui.quiet = True
> +node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
> +tmpwctx = repo[node]
> +return tmpwctx, addedbefore

This and the next unbalanced ui.quiet hack can be a source of future bugs.
I hope they'll be fixed by a follow-up patch.
Can you explain please? I was trying to keep the ui.quiet behavior unchanged in 
this refactoring series.


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH RFC] ui: add configoverride context manager

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1479658212 28800
#  Sun Nov 20 08:10:12 2016 -0800
# Node ID b226d57bafe182f379b144ec4f6374fedfc124ef
# Parent  141b0d27e9e1e846215ead5314237536efc1a185
ui: add configoverride context manager

I feel like this idea might've been discussed before, so please
feel free to point me to the right mailing list entry to read
about why it should not be done.

We have a common pattern of the following code:
backup = ui.backupconfig(section, name)
try:
ui.setconfig(section, name, temporaryvalue, source)
do_something()
finally:
ui.restoreconfig(backup)

IMO, this looks better:
with ui.configoverride([(section, name, temporaryvalue)], source):
do_something()

Especially this becomes more convenient when one has to backup multiple
config values before doing something. In such case, adding a new value
to backup requires codemod in three places.

Note that this contextmanager does not have to be a member ui class,
but it felt like the right place to me.

I have tested it manually, but I am not sure how an automated test
can be written here.

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -7,6 +7,7 @@
 
 from __future__ import absolute_import
 
+import contextlib
 import errno
 import getpass
 import inspect
@@ -1193,6 +1194,26 @@ class ui(object):
 " update your code.)") % version
 self.develwarn(msg, stacklevel=2, config='deprec-warn')
 
+@contextlib.contextmanager
+def configoverride(self, overrides, source="", quiet=None):
+"""Context manager for temporary config overrides
+`overrides` must be a list of three-element tuples:
+[(section, name, value)].
+`quiet` is an optional override for `ui.quiet`"""
+backups = {}
+for override in overrides:
+section, name, value = override
+backups[(section, name)] = self.backupconfig(section, name)
+self.setconfig(section, name, value, source)
+if quiet is not None:
+backupquiet = self.quiet
+self.quiet = quiet
+yield
+for __, backup in backups.items():
+self.restoreconfig(backup)
+if quiet is not None:
+self.quiet = backupquiet
+
 class paths(dict):
 """Represents a collection of paths and their configs.
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH v2] ui: add configoverride context manager

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1479774146 28800
#  Mon Nov 21 16:22:26 2016 -0800
# Node ID a7c57b059190f02a450667411d92d9a4862f6375
# Parent  141b0d27e9e1e846215ead5314237536efc1a185
ui: add configoverride context manager

I feel like this idea might've been discussed before, so please
feel free to point me to the right mailing list entry to read
about why it should not be done.

We have a common pattern of the following code:
backup = ui.backupconfig(section, name)
try:
ui.setconfig(section, name, temporaryvalue, source)
do_something()
finally:
ui.restoreconfig(backup)

IMO, this looks better:
with ui.configoverride({(section, name): temporaryvalue}, source):
do_something()

Especially this becomes more convenient when one has to backup multiple
config values before doing something. In such case, adding a new value
to backup requires codemod in three places.

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -7,6 +7,7 @@
 
 from __future__ import absolute_import
 
+import contextlib
 import errno
 import getpass
 import inspect
@@ -1193,6 +1194,23 @@ class ui(object):
 " update your code.)") % version
 self.develwarn(msg, stacklevel=2, config='deprec-warn')
 
+@contextlib.contextmanager
+def configoverride(self, overrides, source=""):
+"""Context manager for temporary config overrides
+`overrides` must be a dict of the following structure:
+{(section, name) : value}"""
+backups = {}
+for (section, name), value in overrides.items():
+backups[(section, name)] = self.backupconfig(section, name)
+self.setconfig(section, name, value, source)
+yield
+for __, backup in backups.items():
+self.restoreconfig(backup)
+# just restoring ui.quiet config to the previous value is not enough
+# as it does not update ui.quiet class member
+if ('ui', 'quiet') in overrides:
+self.fixconfig(section='ui')
+
 class paths(dict):
 """Represents a collection of paths and their configs.
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH shelve-ext] shelve: make --keep option survive user intevention (issue5431)

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1479926825 28800
#  Wed Nov 23 10:47:05 2016 -0800
# Node ID 90b1e0d3e7daaf489428e0b9028cd7b32f37ea70
# Parent  db897ddf3a8ebb8df9556ce97de11f6380a9ef0b
shelve: make --keep option survive user intevention (issue5431)

Currently if user runs 'hg unshelve --keep' and merge conflicts
occur, the information about --keep provided by user is lost and
shelf is deleted after 'hg unshelve --continue'. This is obviously
not desired, so this patch fixes it.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -159,6 +159,8 @@ class shelvedstate(object):
 """
 _version = 1
 _filename = 'shelvedstate'
+_keep = 'keep'
+_nokeep = 'nokeep'
 
 @classmethod
 def load(cls, repo):
@@ -175,6 +177,7 @@ class shelvedstate(object):
 parents = [nodemod.bin(h) for h in fp.readline().split()]
 stripnodes = [nodemod.bin(h) for h in fp.readline().split()]
 branchtorestore = fp.readline().strip()
+keep = fp.readline().strip() == cls._keep
 except (ValueError, TypeError) as err:
 raise error.CorruptedState(str(err))
 finally:
@@ -188,6 +191,7 @@ class shelvedstate(object):
 obj.parents = parents
 obj.stripnodes = stripnodes
 obj.branchtorestore = branchtorestore
+obj.keep = keep
 except error.RepoLookupError as err:
 raise error.CorruptedState(str(err))
 
@@ -195,7 +199,7 @@ class shelvedstate(object):
 
 @classmethod
 def save(cls, repo, name, originalwctx, pendingctx, stripnodes,
- branchtorestore):
+ branchtorestore, keep=False):
 fp = repo.vfs(cls._filename, 'wb')
 fp.write('%i\n' % cls._version)
 fp.write('%s\n' % name)
@@ -206,6 +210,7 @@ class shelvedstate(object):
 fp.write('%s\n' %
  ' '.join([nodemod.hex(n) for n in stripnodes]))
 fp.write('%s\n' % branchtorestore)
+fp.write('%s\n' % (cls._keep if keep else cls._nokeep))
 fp.close()
 
 @classmethod
@@ -680,7 +685,7 @@ def _rebaserestoredcommit(ui, repo, opts
 stripnodes = [repo.changelog.node(rev)
   for rev in xrange(oldtiprev, len(repo))]
 shelvedstate.save(repo, basename, pctx, tmpwctx, stripnodes,
-  branchtorestore)
+  branchtorestore, opts.get('keep'))
 
 util.rename(repo.join('rebasestate'),
 repo.join('unshelverebasestate'))
@@ -782,6 +787,7 @@ def _dounshelve(ui, repo, *shelved, **op
 
 try:
 state = shelvedstate.load(repo)
+opts['keep'] = opts.get('keep') or state.keep
 except IOError as err:
 if err.errno != errno.ENOENT:
 raise
diff --git a/tests/test-shelve.t b/tests/test-shelve.t
--- a/tests/test-shelve.t
+++ b/tests/test-shelve.t
@@ -1622,3 +1622,31 @@ progress
   abort: no unshelve in progress
   [255]
   $ cd ..
+
+Unshelve respects --keep even if user intervention is needed
+  $ hg init unshelvekeep
+  $ echo 1 > file && hg ci -Am 1
+  adding file
+  $ echo 2 >> file
+  $ hg shelve
+  shelved as default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ echo 3 >> file && hg ci -Am 13
+  $ hg shelve --list
+  default (1s ago)changes to: 1
+  $ hg unshelve --keep
+  unshelving change 'default'
+  rebasing shelved changes
+  rebasing 3:1d24e58054c8 "changes to: 1" (tip)
+  merging file
+  warning: conflicts while merging file! (edit, then use 'hg resolve --mark')
+  unresolved conflicts (see 'hg resolve', then 'hg unshelve --continue')
+  [1]
+  $ hg resolve --mark file
+  (no more unresolved files)
+  continue: hg unshelve --continue
+  $ hg unshelve --continue
+  rebasing 3:1d24e58054c8 "changes to: 1" (tip)
+  unshelve of 'default' complete
+  $ hg shelve --list
+  default (1s ago)changes to: 1
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH shelve-ext v2] shelve: make --keep option survive user intevention (issue5431)

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1479941932 28800
#  Wed Nov 23 14:58:52 2016 -0800
# Node ID 00a022e44b62cb9340dfac94096b14f77407cefb
# Parent  db897ddf3a8ebb8df9556ce97de11f6380a9ef0b
shelve: make --keep option survive user intevention (issue5431)

Currently if user runs 'hg unshelve --keep' and merge conflicts
occur, the information about --keep provided by user is lost and
shelf is deleted after 'hg unshelve --continue'. This is obviously
not desired, so this patch fixes it.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -159,6 +159,8 @@ class shelvedstate(object):
 """
 _version = 1
 _filename = 'shelvedstate'
+_keep = 'keep'
+_nokeep = 'nokeep'
 
 @classmethod
 def load(cls, repo):
@@ -175,6 +177,7 @@ class shelvedstate(object):
 parents = [nodemod.bin(h) for h in fp.readline().split()]
 stripnodes = [nodemod.bin(h) for h in fp.readline().split()]
 branchtorestore = fp.readline().strip()
+keep = fp.readline().strip() == cls._keep
 except (ValueError, TypeError) as err:
 raise error.CorruptedState(str(err))
 finally:
@@ -188,6 +191,7 @@ class shelvedstate(object):
 obj.parents = parents
 obj.stripnodes = stripnodes
 obj.branchtorestore = branchtorestore
+obj.keep = keep
 except error.RepoLookupError as err:
 raise error.CorruptedState(str(err))
 
@@ -195,7 +199,7 @@ class shelvedstate(object):
 
 @classmethod
 def save(cls, repo, name, originalwctx, pendingctx, stripnodes,
- branchtorestore):
+ branchtorestore, keep=False):
 fp = repo.vfs(cls._filename, 'wb')
 fp.write('%i\n' % cls._version)
 fp.write('%s\n' % name)
@@ -206,6 +210,7 @@ class shelvedstate(object):
 fp.write('%s\n' %
  ' '.join([nodemod.hex(n) for n in stripnodes]))
 fp.write('%s\n' % branchtorestore)
+fp.write('%s\n' % (cls._keep if keep else cls._nokeep))
 fp.close()
 
 @classmethod
@@ -680,7 +685,7 @@ def _rebaserestoredcommit(ui, repo, opts
 stripnodes = [repo.changelog.node(rev)
   for rev in xrange(oldtiprev, len(repo))]
 shelvedstate.save(repo, basename, pctx, tmpwctx, stripnodes,
-  branchtorestore)
+  branchtorestore, opts.get('keep'))
 
 util.rename(repo.join('rebasestate'),
 repo.join('unshelverebasestate'))
@@ -782,6 +787,8 @@ def _dounshelve(ui, repo, *shelved, **op
 
 try:
 state = shelvedstate.load(repo)
+if opts['keep'] is None:
+opts['keep'] = state.keep
 except IOError as err:
 if err.errno != errno.ENOENT:
 raise
diff --git a/tests/test-shelve.t b/tests/test-shelve.t
--- a/tests/test-shelve.t
+++ b/tests/test-shelve.t
@@ -1622,3 +1622,31 @@ progress
   abort: no unshelve in progress
   [255]
   $ cd ..
+
+Unshelve respects --keep even if user intervention is needed
+  $ hg init unshelvekeep
+  $ echo 1 > file && hg ci -Am 1
+  adding file
+  $ echo 2 >> file
+  $ hg shelve
+  shelved as default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ echo 3 >> file && hg ci -Am 13
+  $ hg shelve --list
+  default (1s ago)changes to: 1
+  $ hg unshelve --keep
+  unshelving change 'default'
+  rebasing shelved changes
+  rebasing 3:1d24e58054c8 "changes to: 1" (tip)
+  merging file
+  warning: conflicts while merging file! (edit, then use 'hg resolve --mark')
+  unresolved conflicts (see 'hg resolve', then 'hg unshelve --continue')
+  [1]
+  $ hg resolve --mark file
+  (no more unresolved files)
+  continue: hg unshelve --continue
+  $ hg unshelve --continue
+  rebasing 3:1d24e58054c8 "changes to: 1" (tip)
+  unshelve of 'default' complete
+  $ hg shelve --list
+  default (1s ago)changes to: 1
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] ui: use try..finally in configoverride

[Kostia Balytskyi]

On 11/27/16, 9:10 AM, "Mercurial-devel on behalf of Yuya Nishihara" 
 wrote:

On Sat, 26 Nov 2016 09:14:46 -0800, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc 
> # Date 1480180481 28800
> #  Sat Nov 26 09:14:41 2016 -0800
> # Node ID ace32ef8bb5a16683e0312d38e4ff5aa9abfc911
> # Parent  906a7d8e969552536fffe0df7a5e63bf5d79b34b
> ui: use try..finally in configoverride
> 
> @contextmanager almost always have their "yield" inside a try..finally
> block. This is because if the calling code inside the activated
> context manager raises, the code after the "yield" won't get
> executed. A "finally" block, however, will get executed in this
> scenario.

Sure, queued, thanks.

I did not know about this, should’ve read the docs more carefully. Thanks!
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel&d=DgIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=ovbkX-mbuGLQiRqmUixTyjQmk3Y5x_gRDg067rRv02s&s=7mpE6rQbWryTUWe0KfxuoVfGNmRz-dnxccmbpo6fIDM&e=
 


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] formatter: add overview of API and example as doctest

[Kostia Balytskyi]

I like this and previous patch, but if the purpose of this one is to provide 
examlpes to developers, should we maybe also include debugformatter?

On 11/27/16, 1:45 PM, "Mercurial-devel on behalf of Yuya Nishihara" 
 wrote:

# HG changeset patch
# User Yuya Nishihara 
# Date 1477116131 -32400
#  Sat Oct 22 15:02:11 2016 +0900
# Node ID 4ee6e3d759bb847648dcab8c6af9a80a6cd56334
# Parent  a8627780e15c1382e11f1d5df0f33843497dbac7
formatter: add overview of API and example as doctest

diff --git a/mercurial/formatter.py b/mercurial/formatter.py
--- a/mercurial/formatter.py
+++ b/mercurial/formatter.py
@@ -5,6 +5,101 @@
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
+"""Generic output formatting for Mercurial
+
+The formatter provides API to show data in various ways. The following
+functions should be used in place of ui.write():
+
+- fm.write() for unconditional output
+- fm.condwrite() to show some extra data conditionally in plain output
+- fm.data() to provide extra data to JSON or template output
+- fm.plain() to show raw text that isn't provided to JSON or template 
output
+
+To show structured data (e.g. date tuples, dicts, lists), apply 
fm.format*()
+beforehand so the data is converted to the appropriate data type. Use
+fm.isplain() if you need to convert or format data conditionally which 
isn't
+supported by the formatter API.
+
+To build nested structure (i.e. a list of dicts), use fm.nested().
+
+See also 
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_wiki_GenericTemplatingPlan&d=DgIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=v1vFs5tNeWq6k6_Ofepm4RPrO70LtbAVADCkfqBEKzI&s=509gZDjWRL6U3ASP0G6CyKtiYJUxvBvvTXDsBB55TPo&e=
 
+
+fm.condwrite() vs 'if cond:':
+
+In most cases, use fm.condwrite() so users can selectively show the data
+in template output. If it's costly to build data, use plain 'if cond:' with
+fm.write().
+
+fm.nested() vs fm.formatdict() (or fm.formatlist()):
+
+fm.nested() should be used to form a tree structure (a list of dicts of
+lists of dicts...) which can be accessed through template keywords, e.g.
+"{foo % "{bar % {...}} {baz % {...}}"}". On the other hand, fm.formatdict()
+exports a dict-type object to template, which can be accessed by e.g.
+"{get(foo, key)}" function.
+
+Doctest helper:
+
+>>> def show(fn, verbose=False, **opts):
+... import sys
+... from . import ui as uimod
+... ui = uimod.ui()
+... ui.fout = sys.stdout  # redirect to doctest
+... ui.verbose = verbose
+... return fn(ui, ui.formatter(fn.__name__, opts))
+
+Basic example:
+
+>>> def files(ui, fm):
+... files = [('foo', 123, (0, 0)), ('bar', 456, (1, 0))]
+... for f in files:
+... fm.startitem()
+... fm.write('path', '%s', f[0])
+... fm.condwrite(ui.verbose, 'date', '  %s',
+...  fm.formatdate(f[2], '%Y-%m-%d %H:%M:%S'))
+... fm.data(size=f[1])
+... fm.plain('\\n')
+... fm.end()
+>>> show(files)
+foo
+bar
+>>> show(files, verbose=True)
+foo  1970-01-01 00:00:00
+bar  1970-01-01 00:00:01
+>>> show(files, template='json')
+[
+ {
+  "date": [0, 0],
+  "path": "foo",
+  "size": 123
+ },
+ {
+  "date": [1, 0],
+  "path": "bar",
+  "size": 456
+ }
+]
+>>> show(files, template='path: {path}\\ndate: {date|rfc3339date}\\n')
+path: foo
+date: 1970-01-01T00:00:00+00:00
+path: bar
+date: 1970-01-01T00:00:01+00:00
+
+Nested example:
+
+>>> def subrepos(ui, fm):
+... fm.startitem()
+... fm.write('repo', '[%s]\\n', 'baz')
+... files(ui, fm.nested('files'))
+... fm.end()
+>>> show(subrepos)
+[baz]
+foo
+bar
+>>> show(subrepos, template='{repo}: {join(files % "{path}", ", ")}\\n')
+baz: foo, bar
+"""
+
 from __future__ import absolute_import
 
 import os
diff --git a/tests/test-doctest.py b/tests/test-doctest.py
--- a/tests/test-doctest.py
+++ b/tests/test-doctest.py
@@ -20,6 +20,7 @@ testmod('mercurial.changelog')
 testmod('mercurial.dagparser', optionflags=doctest.NORMALIZE_WHITESPACE)
 testmod('mercurial.dispatch')
 testmod('mercurial.encoding')
+testmod('mercurial.formatter')
 testmod('mercurial.hg')
 testmod('mercurial.hgweb.hgwebdir_mod')
 testmod('mercurial.match')
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercu

[PATCH shelve-ext] shelve: fix use of unexpected working dirs in test-shelve.t

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480421465 28800
#  Tue Nov 29 04:11:05 2016 -0800
# Node ID b4e28d31201c6af4c677c53b42c140b3cfb3d352
# Parent  6d69c3708cf3b9a70403c3d3cb3c44ff1e8faf53
shelve: fix use of unexpected working dirs in test-shelve.t

Fixing some clowniness where we created ~four levels of nested repos
and once (my test case :( ) did not even cd into a created repo.

diff --git a/tests/test-shelve.t b/tests/test-shelve.t
--- a/tests/test-shelve.t
+++ b/tests/test-shelve.t
@@ -1383,6 +1383,7 @@ We expect that bare-shelve will not keep
   0 files updated, 0 files merged, 1 files removed, 0 files unresolved
   $ hg branch
   default
+  $ cd ..
 
 When i shelve commit on newly created branch i expect
 that after unshelve newly created branch will be preserved.
@@ -1416,6 +1417,7 @@ that after unshelve newly created branch
   ? b
   $ hg branch
   test
+  $ cd ..
 
 When i shelve commit on newly created branch, make
 some changes, unshelve it and running into merge
@@ -1489,6 +1491,7 @@ test branch.
   A b
   $ hg branch
   default
+  $ cd ..
 
 When i unshelve resulting in merge conflicts and makes saved
 file shelvedstate looks like in previous versions in
@@ -1551,6 +1554,7 @@ in previous versions) and running unshel
   M a
   $ hg branch
   default
+  $ cd ..
 
 On non bare shelve the branch information shouldn't be restored
 
@@ -1624,7 +1628,7 @@ progress
   $ cd ..
 
 Unshelve respects --keep even if user intervention is needed
-  $ hg init unshelvekeep
+  $ hg init unshelvekeep && cd unshelvekeep
   $ echo 1 > file && hg ci -Am 1
   adding file
   $ echo 2 >> file
@@ -1637,7 +1641,7 @@ Unshelve respects --keep even if user in
   $ hg unshelve --keep
   unshelving change 'default'
   rebasing shelved changes
-  rebasing 3:1d24e58054c8 "changes to: 1" (tip)
+  rebasing 2:3fbe6fbb0bef "changes to: 1" (tip)
   merging file
   warning: conflicts while merging file! (edit, then use 'hg resolve --mark')
   unresolved conflicts (see 'hg resolve', then 'hg unshelve --continue')
@@ -1646,7 +1650,7 @@ Unshelve respects --keep even if user in
   (no more unresolved files)
   continue: hg unshelve --continue
   $ hg unshelve --continue
-  rebasing 3:1d24e58054c8 "changes to: 1" (tip)
+  rebasing 2:3fbe6fbb0bef "changes to: 1" (tip)
   unshelve of 'default' complete
   $ hg shelve --list
-  default (1s ago)changes to: 1
+  default (*s ago)changes to: 1 (glob)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 09 of 10 shelve-ext] shelve: add obs-based unshelve functionality

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480431173 28800
#  Tue Nov 29 06:52:53 2016 -0800
# Node ID 533d99eca3bf11c4aac869674e0abb16b74ed670
# Parent  85c9c651887915733feb3d385866955741f28ec0
shelve: add obs-based unshelve functionality

Obsolescense-based unshelve works as follows:
1. Instead of stripping temporary nodes, markers are created to
obsolete them.
2. Restoring commit is just finding it in an unfiltered repo.
3. '--keep' is only passed to rebase on traditional unshelves
(and thus traditional rebases), becuase we want markers to be
created fro obsolete-based rebases.
4. 'hg unshelve' uses unfiltered repo to perform rebases
because we want rebase to be able to create markers between original
and new commits. 'rebaseskipobsolete' is disabled to make rebase not
skip the commit altogether.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -26,6 +26,7 @@ import collections
 import errno
 import itertools
 import json
+import time
 
 from mercurial.i18n import _
 from mercurial import (
@@ -264,8 +265,13 @@ class shelvedstate(object):
 
 def prunenodes(self):
 """Cleanup temporary nodes from the repo"""
-repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
- topic='shelve')
+if self.obsshelve:
+unfi = self.repo.unfiltered()
+relations = [(unfi[n], ()) for n in self.nodestoprune]
+obsolete.createmarkers(self.repo, relations)
+else:
+repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
+ topic='shelve')
 
 def cleanupoldbackups(repo):
 vfs = scmutil.vfs(repo.join(backupdir))
@@ -670,9 +676,14 @@ def unshelvecontinue(ui, repo, state, op
 util.rename(repo.join('unshelverebasestate'),
 repo.join('rebasestate'))
 try:
-rebase.rebase(ui, repo, **{
-'continue' : True
-})
+# if shelve is obs-based, we want rebase to be able
+# to create markers to already-obsoleted commits
+_repo = repo.unfiltered() if state.obsshelve else repo
+with ui.configoverride({('experimental', 'rebaseskipobsolete'):
+'off'}, 'unshelve'):
+rebase.rebase(ui, _repo, **{
+'continue' : True,
+})
 except Exception:
 util.rename(repo.join('rebasestate'),
 repo.join('unshelverebasestate'))
@@ -712,30 +723,54 @@ def _commitworkingcopychanges(ui, repo, 
 with ui.configoverride({('ui', 'quiet'): True}):
 node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
 tmpwctx = repo[node]
+ui.debug("temporary working copy commit: %s:%s\n" %
+ (tmpwctx.rev(), nodemod.short(node)))
 return tmpwctx, addedbefore
 
-def _unshelverestorecommit(ui, repo, basename):
+def _unshelverestorecommit(ui, repo, basename, obsshelve, shfile):
 """Recreate commit in the repository during the unshelve"""
 with ui.configoverride({('ui', 'quiet'): True}):
-shelvedfile(repo, basename, 'hg').applybundle()
-shelvectx = repo['tip']
+if obsshelve:
+md = shfile.readjson()
+shelvenode = nodemod.bin(md['node'])
+repo = repo.unfiltered()
+shelvectx = repo[shelvenode]
+else:
+shelvedfile(repo, basename, 'hg').applybundle()
+shelvectx = repo['tip']
 return repo, shelvectx
 
 def _rebaserestoredcommit(ui, repo, opts, tr, oldtiprev, basename, pctx,
-  tmpwctx, shelvectx, branchtorestore):
+  tmpwctx, shelvectx, branchtorestore, obsshelve):
 """Rebase restored commit from its original location to a destination"""
 # If the shelve is not immediately on top of the commit
 # we'll be merging with, rebase it to be on top.
 if tmpwctx.node() == shelvectx.parents()[0].node():
+# shelvectx is immediately on top of the tmpwctx
 return shelvectx
 
+# we need a new commit extra every time we perform a rebase to ensure
+# that "nothing to rebase" does not happen with obs-based shelve
+# "nothing to rebase" means that tip does not point to a "successor"
+# commit after a rebase and we have no way to learn which commit
+# should be a "shelvectx". this is a dirty hack until we implement
+# some way to learn the results of rebase operation, other than
+# text output and return code
+

[PATCH 04 of 10 shelve-ext] shelve: move node-pruning functionality to be member of shelvedstate

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480425731 28800
#  Tue Nov 29 05:22:11 2016 -0800
# Node ID 4166a4cfdc14629e169f12e83607b065887594b7
# Parent  93bb69fa1f1abbe0a42455ad8df691fdf29b2e3a
shelve: move node-pruning functionality to be member of shelvedstate

Node-pruning can be node stripping or marker creation, depending on
whether shelve is traditional or obs-based. Thus it makes sense to
move it to a separate function. Also, since we already have
shelvedstate object and this functionality operates on that object,
it makes sense to make it a method.

Having shelvedstate object contain repo and ui as members allows for
calling 'state.prunenodes()' instead of 'state.prunenodes(repo, ui)'
which is better IMO.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -185,8 +185,12 @@ class shelvedstate(object):
 _keep = 'keep'
 _nokeep = 'nokeep'
 
+def __init__(self, ui, repo):
+self.ui = ui
+self.repo = repo
+
 @classmethod
-def load(cls, repo):
+def load(cls, ui, repo):
 fp = repo.vfs(cls._filename)
 try:
 version = int(fp.readline().strip())
@@ -207,7 +211,7 @@ class shelvedstate(object):
 fp.close()
 
 try:
-obj = cls()
+obj = cls(ui, repo)
 obj.name = name
 obj.wctx = repo[wctx]
 obj.pendingctx = repo[pendingctx]
@@ -240,6 +244,11 @@ class shelvedstate(object):
 def clear(cls, repo):
 util.unlinkpath(repo.join(cls._filename), ignoremissing=True)
 
+def prunenodes(self):
+"""Cleanup temporary nodes from the repo"""
+repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
+ topic='shelve')
+
 def cleanupoldbackups(repo):
 vfs = scmutil.vfs(repo.join(backupdir))
 maxbackups = repo.ui.configint('shelve', 'maxbackups', 10)
@@ -576,8 +585,7 @@ def unshelveabort(ui, repo, state, opts)
 raise
 
 mergefiles(ui, repo, state.wctx, state.pendingctx)
-repair.strip(ui, repo, state.nodestoprune, backup=False,
- topic='shelve')
+state.prunenodes()
 finally:
 shelvedstate.clear(repo)
 ui.warn(_("unshelve of '%s' aborted\n") % state.name)
@@ -654,7 +662,7 @@ def unshelvecontinue(ui, repo, state, op
 mergefiles(ui, repo, state.wctx, shelvectx)
 restorebranch(ui, repo, state.branchtorestore)
 
-repair.strip(ui, repo, state.nodestoprune, backup=False, 
topic='shelve')
+state.prunenodes()
 shelvedstate.clear(repo)
 unshelvecleanup(ui, repo, state.name, opts)
 ui.status(_("unshelve of '%s' complete\n") % state.name)
@@ -810,7 +818,7 @@ def _dounshelve(ui, repo, *shelved, **op
 ui.warn(_('tool option will be ignored\n'))
 
 try:
-state = shelvedstate.load(repo)
+state = shelvedstate.load(ui, repo)
 if opts.get('keep') is None:
 opts['keep'] = state.keep
 except IOError as err:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 05 of 10 shelve-ext] shelve: add a function to check whether obs-based shelve is enabled

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480426659 28800
#  Tue Nov 29 05:37:39 2016 -0800
# Node ID 7f225df4c58b4f62a7883f95349cdb9e9c86e15e
# Parent  4166a4cfdc14629e169f12e83607b065887594b7
shelve: add a function to check whether obs-based shelve is enabled

A central place to check whether code should use traditional or
obsolescense-based shelve behavior.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -41,6 +41,7 @@ from mercurial import (
 mdiff,
 merge,
 node as nodemod,
+obsolete,
 patch,
 phases,
 repair,
@@ -71,6 +72,18 @@ patchextension = 'patch'
 # generic user for all shelve operations
 shelveuser = 'shelve@localhost'
 
+def isobsshelve(repo, ui):
+"""Check whether obsolescense-based shelve is enabled"""
+obsshelve = ui.configbool('experimental', 'obsshelve')
+if not obsshelve:
+return False
+if not obsolete.isenabled(repo, obsolete.createmarkersopt):
+w = _('ignoring experimental.obsshelve because createmarkers option '
+  'is disabled')
+ui.warn(w)
+return False
+return True
+
 class shelvedfile(object):
 """Helper for the file storing a single shelve
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 10 shelve-ext] shelve: add an ability to write json to a new type of shelve files

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480426193 28800
#  Tue Nov 29 05:29:53 2016 -0800
# Node ID 37119e028c699d9fabd220086e08c754827e709f
# Parent  f6f0ab3f7b0ea0e05cfdcd7afd4994ea21988fd9
shelve: add an ability to write json to a new type of shelve files

Obsolescense-based shelve only needs metadata stored in .hg/shelved
and I think that this metadata should be stored in json for
potential extensibility purposes. JSON is not critical here, but
I want to avoid storing it in an unstructured text file where
order of lines determines their semantical meanings (as now
happens in .hg/shelvedstate. .hg/rebasestate and I suspect other
state files as well).

If we want, we can refactor it to something else later.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -25,6 +25,7 @@ from __future__ import absolute_import
 import collections
 import errno
 import itertools
+import json
 
 from mercurial.i18n import _
 from mercurial import (
@@ -62,7 +63,7 @@ testedwith = 'ships-with-hg-core'
 
 backupdir = 'shelve-backup'
 shelvedir = 'shelved'
-shelvefileextensions = ['hg', 'patch']
+shelvefileextensions = ['hg', 'patch', 'oshelve']
 # universal extension is present in all types of shelves
 patchextension = 'patch'
 
@@ -153,6 +154,26 @@ class shelvedfile(object):
 bundle2.writebundle(self.ui, cg, self.fname, btype, self.vfs,
 compression=compression)
 
+def writejson(self, jsn):
+fp = self.opener('wb')
+try:
+fp.write(json.dumps(jsn))
+finally:
+fp.close()
+
+def readjson(self):
+fp = self.opener()
+contents = None
+try:
+contents = fp.read()
+finally:
+fp.close()
+try:
+jsn = json.loads(contents)
+except (TypeError, ValueError):
+raise error.abort(_('could not read obsolescense-based shelve'))
+return jsn
+
 class shelvedstate(object):
 """Handle persistence during unshelving operations.
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 08 of 10 shelve-ext] shelve: migrate config overrides to ui.configoverride

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480427488 28800
#  Tue Nov 29 05:51:28 2016 -0800
# Node ID 85c9c651887915733feb3d385866955741f28ec0
# Parent  bcf8d603cc8b678f875ceca24dd2b14eda09bce7
shelve: migrate config overrides to ui.configoverride

This patch also makes ui.quiet manipulations much more explicit
and readable, addressing previous Yuya's concerns.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -355,17 +355,16 @@ def getcommitfunc(extra, interactive, ed
 hasmq = util.safehasattr(repo, 'mq')
 if hasmq:
 saved, repo.mq.checkapplied = repo.mq.checkapplied, False
-backup = repo.ui.backupconfig('phases', 'new-commit')
 try:
-repo.ui.setconfig('phases', 'new-commit', phases.secret)
-editor_ = False
-if editor:
-editor_ = cmdutil.getcommiteditor(editform='shelve.shelve',
-  **opts)
-return repo.commit(message, shelveuser, opts.get('date'), match,
-   editor=editor_, extra=extra)
+overrides = {('phases', 'new-commit'): phases.secret}
+with repo.ui.configoverride(overrides):
+editor_ = False
+if editor:
+editor_ = cmdutil.getcommiteditor(editform='shelve.shelve',
+  **opts)
+return repo.commit(message, shelveuser, opts.get('date'),
+   match, editor=editor_, extra=extra)
 finally:
-repo.ui.restoreconfig(backup)
 if hasmq:
 repo.mq.checkapplied = saved
 
@@ -625,9 +624,7 @@ def unshelveabort(ui, repo, state, opts)
 def mergefiles(ui, repo, wctx, shelvectx):
 """updates to wctx and merges the changes from shelvectx into the
 dirstate."""
-oldquiet = ui.quiet
-try:
-ui.quiet = True
+with ui.configoverride({('ui', 'quiet'): True}):
 hg.update(repo, wctx.node())
 files = []
 files.extend(shelvectx.files())
@@ -642,8 +639,6 @@ def mergefiles(ui, repo, wctx, shelvectx
*pathtofiles(repo, files),
**{'no_backup': True})
 ui.popbuffer()
-finally:
-ui.quiet = oldquiet
 
 def restorebranch(ui, repo, branchtorestore):
 if branchtorestore and branchtorestore != repo.dirstate.branch():
@@ -714,17 +709,16 @@ def _commitworkingcopychanges(ui, repo, 
 tempopts = {}
 tempopts['message'] = "pending changes temporary commit"
 tempopts['date'] = opts.get('date')
-ui.quiet = True
-node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
+with ui.configoverride({('ui', 'quiet'): True}):
+node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
 tmpwctx = repo[node]
 return tmpwctx, addedbefore
 
-def _unshelverestorecommit(ui, repo, basename, oldquiet):
+def _unshelverestorecommit(ui, repo, basename):
 """Recreate commit in the repository during the unshelve"""
-ui.quiet = True
-shelvedfile(repo, basename, 'hg').applybundle()
-shelvectx = repo['tip']
-ui.quiet = oldquiet
+with ui.configoverride({('ui', 'quiet'): True}):
+shelvedfile(repo, basename, 'hg').applybundle()
+shelvectx = repo['tip']
 return repo, shelvectx
 
 def _rebaserestoredcommit(ui, repo, opts, tr, oldtiprev, basename, pctx,
@@ -890,13 +884,9 @@ def _dounshelve(ui, repo, *shelved, **op
 if not shelvedfile(repo, basename, patchextension).exists():
 raise error.Abort(_("shelved change '%s' not found") % basename)
 
-oldquiet = ui.quiet
 lock = tr = None
-forcemerge = ui.backupconfig('ui', 'forcemerge')
 try:
-ui.setconfig('ui', 'forcemerge', opts.get('tool', ''), 'unshelve')
 lock = repo.lock()
-
 tr = repo.transaction('unshelve', report=lambda x: None)
 oldtiprev = len(repo)
 
@@ -911,16 +901,18 @@ def _dounshelve(ui, repo, *shelved, **op
 tmpwctx, addedbefore = _commitworkingcopychanges(ui, repo, opts,
  tmpwctx)
 
-repo, shelvectx = _unshelverestorecommit(ui, repo, basename, oldquiet)
+repo, shelvectx = _unshelverestorecommit(ui, repo, basename)
 
 branchtorestore = ''
 if shelvectx.branch() != shelvectx.p1().branch():
 branchtorestore = shelvectx.branch()
 
-shelvectx = _re

[PATCH 01 of 10 shelve-ext] shelve: move patch extension to a string constant

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480432803 28800
#  Tue Nov 29 07:20:03 2016 -0800
# Node ID f6f0ab3f7b0ea0e05cfdcd7afd4994ea21988fd9
# Parent  64b55bffc1c059eb4c11ca195b561ca8a287f59e
shelve: move patch extension to a string constant

We are using 'name + ".patch"' pattern throughout the shelve code to
identify the existence of a shelve with a particular name. In two
cases however we use 'name + ".hg"' instead. This commit makes
'patch' be used in all places and "emphasizes" it by moving
'patch' to live in a constant. Also, this allows to extract file
name without extension like this:
f[:-(1 + len(patchextension))]
instead of:
f[:-6]
which is good IMO.

This is a first patch from this initial "obsshelve" series. This
series does not include tests, although locally I have all of
test-shelve.t ported to test obs-shelve as well. I will send tests
later as a separate series.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -63,6 +63,8 @@ testedwith = 'ships-with-hg-core'
 backupdir = 'shelve-backup'
 shelvedir = 'shelved'
 shelvefileextensions = ['hg', 'patch']
+# universal extension is present in all types of shelves
+patchextension = 'patch'
 
 # we never need the user, so we use a
 # generic user for all shelve operations
@@ -220,7 +222,8 @@ class shelvedstate(object):
 def cleanupoldbackups(repo):
 vfs = scmutil.vfs(repo.join(backupdir))
 maxbackups = repo.ui.configint('shelve', 'maxbackups', 10)
-hgfiles = [f for f in vfs.listdir() if f.endswith('.hg')]
+hgfiles = [f for f in vfs.listdir()
+   if f.endswith('.' + patchextension)]
 hgfiles = sorted([(vfs.stat(f).st_mtime, f) for f in hgfiles])
 if 0 < maxbackups and maxbackups < len(hgfiles):
 bordermtime = hgfiles[-maxbackups][0]
@@ -230,7 +233,7 @@ def cleanupoldbackups(repo):
 if mtime == bordermtime:
 # keep it, because timestamp can't decide exact order of backups
 continue
-base = f[:-3]
+base = f[:-(1 + len(patchextension))]
 for ext in shelvefileextensions:
 try:
 vfs.unlink(base + '.' + ext)
@@ -264,12 +267,12 @@ def getshelvename(repo, parent, opts):
 label = label.replace('/', '_')
 
 if name:
-if shelvedfile(repo, name, 'hg').exists():
+if shelvedfile(repo, name, patchextension).exists():
 e = _("a shelved change named '%s' already exists") % name
 raise error.Abort(e)
 else:
 for n in gennames():
-if not shelvedfile(repo, n, 'hg').exists():
+if not shelvedfile(repo, n, patchextension).exists():
 name = n
 break
 else:
@@ -337,7 +340,7 @@ def _shelvecreatedcommit(repo, node, nam
 bases = list(mutableancestors(repo[node]))
 shelvedfile(repo, name, 'hg').writebundle(bases, node)
 cmdutil.export(repo, [node],
-   fp=shelvedfile(repo, name, 'patch').opener('wb'),
+   fp=shelvedfile(repo, name, patchextension).opener('wb'),
opts=mdiff.diffopts(git=True))
 
 def _includeunknownfiles(repo, pats, opts, extra):
@@ -444,7 +447,7 @@ def deletecmd(ui, repo, pats):
 # but the .hg file is optional as in future we
 # will add obsolete shelve with does not create a
 # bundle
-if shfile.exists() or suffix == 'patch':
+if shfile.exists() or suffix == patchextension:
 shfile.movetobackup()
 cleanupoldbackups(repo)
 except OSError as err:
@@ -463,7 +466,7 @@ def listshelves(repo):
 info = []
 for (name, _type) in names:
 pfx, sfx = name.rsplit('.', 1)
-if not pfx or sfx != 'patch':
+if not pfx or sfx != patchextension:
 continue
 st = shelvedfile(repo, name).stat()
 info.append((st.st_mtime, shelvedfile(repo, pfx).filename()))
@@ -491,7 +494,7 @@ def listcmd(ui, repo, pats, opts):
 ui.write(age, label='shelve.age')
 ui.write(' ' * (12 - len(age)))
 used += 12
-with open(name + '.patch', 'rb') as fp:
+with open(name + '.' + patchextension, 'rb') as fp:
 while True:
 line = fp.readline()
 if not line:
@@ -519,7 +522,7 @@ def singlepatchcmds(ui, repo, pats, opts
 raise error.Abort(_("--%s expects a single shelf") % subcommand)
 shelfname = pats[0]
 
-if not shelvedfile(repo, s

[PATCH 03 of 10 shelve-ext] shelve: rename stripnodes to nodestoprune

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480425731 28800
#  Tue Nov 29 05:22:11 2016 -0800
# Node ID 93bb69fa1f1abbe0a42455ad8df691fdf29b2e3a
# Parent  37119e028c699d9fabd220086e08c754827e709f
shelve: rename stripnodes to nodestoprune

Since we are introducing obs-based shelve, we are no longer
stripping temporary nodes, we are obsoleting them. Therefore
it looks like stipnodes would be a misleading name, while
prune has a connotaion of "strip but with obsolescense", so
nodestoprune seems like a good rename.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -198,7 +198,7 @@ class shelvedstate(object):
 wctx = nodemod.bin(fp.readline().strip())
 pendingctx = nodemod.bin(fp.readline().strip())
 parents = [nodemod.bin(h) for h in fp.readline().split()]
-stripnodes = [nodemod.bin(h) for h in fp.readline().split()]
+nodestoprune = [nodemod.bin(h) for h in fp.readline().split()]
 branchtorestore = fp.readline().strip()
 keep = fp.readline().strip() == cls._keep
 except (ValueError, TypeError) as err:
@@ -212,7 +212,7 @@ class shelvedstate(object):
 obj.wctx = repo[wctx]
 obj.pendingctx = repo[pendingctx]
 obj.parents = parents
-obj.stripnodes = stripnodes
+obj.nodestoprune = nodestoprune
 obj.branchtorestore = branchtorestore
 obj.keep = keep
 except error.RepoLookupError as err:
@@ -221,7 +221,7 @@ class shelvedstate(object):
 return obj
 
 @classmethod
-def save(cls, repo, name, originalwctx, pendingctx, stripnodes,
+def save(cls, repo, name, originalwctx, pendingctx, nodestoprune,
  branchtorestore, keep=False):
 fp = repo.vfs(cls._filename, 'wb')
 fp.write('%i\n' % cls._version)
@@ -231,7 +231,7 @@ class shelvedstate(object):
 fp.write('%s\n' %
  ' '.join([nodemod.hex(p) for p in repo.dirstate.parents()]))
 fp.write('%s\n' %
- ' '.join([nodemod.hex(n) for n in stripnodes]))
+ ' '.join([nodemod.hex(n) for n in nodestoprune]))
 fp.write('%s\n' % branchtorestore)
 fp.write('%s\n' % (cls._keep if keep else cls._nokeep))
 fp.close()
@@ -576,7 +576,7 @@ def unshelveabort(ui, repo, state, opts)
 raise
 
 mergefiles(ui, repo, state.wctx, state.pendingctx)
-repair.strip(ui, repo, state.stripnodes, backup=False,
+repair.strip(ui, repo, state.nodestoprune, backup=False,
  topic='shelve')
 finally:
 shelvedstate.clear(repo)
@@ -649,12 +649,12 @@ def unshelvecontinue(ui, repo, state, op
 shelvectx = state.pendingctx
 else:
 # only strip the shelvectx if the rebase produced it
-state.stripnodes.append(shelvectx.node())
+state.nodestoprune.append(shelvectx.node())
 
 mergefiles(ui, repo, state.wctx, shelvectx)
 restorebranch(ui, repo, state.branchtorestore)
 
-repair.strip(ui, repo, state.stripnodes, backup=False, topic='shelve')
+repair.strip(ui, repo, state.nodestoprune, backup=False, 
topic='shelve')
 shelvedstate.clear(repo)
 unshelvecleanup(ui, repo, state.name, opts)
 ui.status(_("unshelve of '%s' complete\n") % state.name)
@@ -706,9 +706,9 @@ def _rebaserestoredcommit(ui, repo, opts
 except error.InterventionRequired:
 tr.close()
 
-stripnodes = [repo.changelog.node(rev)
-  for rev in xrange(oldtiprev, len(repo))]
-shelvedstate.save(repo, basename, pctx, tmpwctx, stripnodes,
+nodestoprune = [repo.changelog.node(rev)
+for rev in xrange(oldtiprev, len(repo))]
+shelvedstate.save(repo, basename, pctx, tmpwctx, nodestoprune,
   branchtorestore, opts.get('keep'))
 
 util.rename(repo.join('rebasestate'),
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 10 shelve-ext] shelve: add obs-based shelve functionality

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480427179 28800
#  Tue Nov 29 05:46:19 2016 -0800
# Node ID d1356cbb72cfe91a2b427098c66bc00937912d79
# Parent  7f225df4c58b4f62a7883f95349cdb9e9c86e15e
shelve: add obs-based shelve functionality

Obsolescense-based shelve works in a following way:
1. In order to shelve some changes, it creates a commit, records its
node into a .oshelve file and prunes created commit.
2. In order to finish a shelve operation, transaction is just
closed and not aborted.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -379,9 +379,16 @@ def _nothingtoshelvemessaging(ui, repo, 
 else:
 ui.status(_("nothing changed\n"))
 
-def _shelvecreatedcommit(repo, node, name):
-bases = list(mutableancestors(repo[node]))
-shelvedfile(repo, name, 'hg').writebundle(bases, node)
+def _shelvecreatedcommit(ui, repo, node, name, tr):
+if isobsshelve(repo, ui):
+unfi = repo.unfiltered()
+obsolete.createmarkers(repo, [(unfi[node], ())])
+shelvedfile(repo, name, 'oshelve').writejson({
+'node': nodemod.hex(node)
+})
+else:
+bases = list(mutableancestors(repo[node]))
+shelvedfile(repo, name, 'hg').writebundle(bases, node)
 cmdutil.export(repo, [node],
fp=shelvedfile(repo, name, patchextension).opener('wb'),
opts=mdiff.diffopts(git=True))
@@ -393,8 +400,12 @@ def _includeunknownfiles(repo, pats, opt
 extra['shelve_unknown'] = '\0'.join(s.unknown)
 repo[None].add(s.unknown)
 
-def _finishshelve(repo):
-_aborttransaction(repo)
+def _finishshelve(ui, repo, tr):
+if isobsshelve(repo, ui):
+tr.close()
+tr.release()
+else:
+_aborttransaction(repo)
 
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
@@ -416,9 +427,12 @@ def _docreatecmd(ui, repo, pats, opts):
 try:
 lock = repo.lock()
 
-# use an uncommitted transaction to generate the bundle to avoid
-# pull races. ensure we don't print the abort message to stderr.
-tr = repo.transaction('commit', report=lambda x: None)
+# depending on whether shelve is traditional or
+# obsolescense-based, we either abort or commit this
+# transaction in the end. If we abort it, we don't
+# want to print anything to stderr
+report = None if isobsshelve(repo, ui) else (lambda x: None)
+tr = repo.transaction('commit', report=report)
 
 interactive = opts.get('interactive', False)
 includeunknown = (opts.get('unknown', False) and
@@ -444,7 +458,7 @@ def _docreatecmd(ui, repo, pats, opts):
 _nothingtoshelvemessaging(ui, repo, pats, opts)
 return 1
 
-_shelvecreatedcommit(repo, node, name)
+_shelvecreatedcommit(ui, repo, node, name, tr)
 
 if ui.formatted():
 desc = util.ellipsis(desc, ui.termwidth())
@@ -453,7 +467,7 @@ def _docreatecmd(ui, repo, pats, opts):
 if origbranch != repo['.'].branch() and not _isbareshelve(pats, opts):
 repo.dirstate.setbranch(origbranch)
 
-_finishshelve(repo)
+_finishshelve(ui, repo, tr)
 finally:
 lockmod.release(tr, lock)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 07 of 10 shelve-ext] shelve: add shelve type saving and loading

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480425731 28800
#  Tue Nov 29 05:22:11 2016 -0800
# Node ID bcf8d603cc8b678f875ceca24dd2b14eda09bce7
# Parent  d1356cbb72cfe91a2b427098c66bc00937912d79
shelve: add shelve type saving and loading

We need shelve type to be stored in .hg/shelvedstate in order
to be able to run abort or continue action properly. If the shelve
is obsbased, those actions should create markes, if it is traditional,
the actions should strip commits.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -197,6 +197,8 @@ class shelvedstate(object):
 _filename = 'shelvedstate'
 _keep = 'keep'
 _nokeep = 'nokeep'
+_obsbased = 'obsbased'
+_traditional = 'traditional'
 
 def __init__(self, ui, repo):
 self.ui = ui
@@ -218,6 +220,7 @@ class shelvedstate(object):
 nodestoprune = [nodemod.bin(h) for h in fp.readline().split()]
 branchtorestore = fp.readline().strip()
 keep = fp.readline().strip() == cls._keep
+obsshelve = fp.readline().strip() == cls._obsbased
 except (ValueError, TypeError) as err:
 raise error.CorruptedState(str(err))
 finally:
@@ -232,6 +235,7 @@ class shelvedstate(object):
 obj.nodestoprune = nodestoprune
 obj.branchtorestore = branchtorestore
 obj.keep = keep
+obj.obsshelve = obsshelve
 except error.RepoLookupError as err:
 raise error.CorruptedState(str(err))
 
@@ -239,7 +243,7 @@ class shelvedstate(object):
 
 @classmethod
 def save(cls, repo, name, originalwctx, pendingctx, nodestoprune,
- branchtorestore, keep=False):
+ branchtorestore, keep=False, obsshelve=False):
 fp = repo.vfs(cls._filename, 'wb')
 fp.write('%i\n' % cls._version)
 fp.write('%s\n' % name)
@@ -251,6 +255,7 @@ class shelvedstate(object):
  ' '.join([nodemod.hex(n) for n in nodestoprune]))
 fp.write('%s\n' % branchtorestore)
 fp.write('%s\n' % (cls._keep if keep else cls._nokeep))
+fp.write('%s\n' % (cls._obsbased if obsshelve else cls._traditional))
 fp.close()
 
 @classmethod
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 10 of 10 shelve-ext] shelve: add logic to preserve active bookmarks

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480431519 28800
#  Tue Nov 29 06:58:39 2016 -0800
# Node ID 5c55e23c067998adf36b9a2c6eb028ba1a7fc643
# Parent  533d99eca3bf11c4aac869674e0abb16b74ed670
shelve: add logic to preserve active bookmarks

This adds an explicit active-bookmark-handling logic
to *both* traditional and obs-based shelve. Although it
is possible to only add it to obs-based, I think it would
be ugly and I see no harm in explicitly handling bookmarks
in addition to reliance on trasnactions.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -30,6 +30,7 @@ import time
 
 from mercurial.i18n import _
 from mercurial import (
+bookmarks,
 bundle2,
 bundlerepo,
 changegroup,
@@ -200,6 +201,8 @@ class shelvedstate(object):
 _nokeep = 'nokeep'
 _obsbased = 'obsbased'
 _traditional = 'traditional'
+# colon is essential to differentiate from a real bookmark name
+_noactivebook = ':no-active-bookmark'
 
 def __init__(self, ui, repo):
 self.ui = ui
@@ -222,6 +225,7 @@ class shelvedstate(object):
 branchtorestore = fp.readline().strip()
 keep = fp.readline().strip() == cls._keep
 obsshelve = fp.readline().strip() == cls._obsbased
+activebook = fp.readline().strip()
 except (ValueError, TypeError) as err:
 raise error.CorruptedState(str(err))
 finally:
@@ -237,6 +241,9 @@ class shelvedstate(object):
 obj.branchtorestore = branchtorestore
 obj.keep = keep
 obj.obsshelve = obsshelve
+obj.activebookmark = ''
+if activebook != cls._noactivebook:
+obj.activebookmark = activebook
 except error.RepoLookupError as err:
 raise error.CorruptedState(str(err))
 
@@ -244,7 +251,7 @@ class shelvedstate(object):
 
 @classmethod
 def save(cls, repo, name, originalwctx, pendingctx, nodestoprune,
- branchtorestore, keep=False, obsshelve=False):
+ branchtorestore, keep=False, obsshelve=False, activebook=''):
 fp = repo.vfs(cls._filename, 'wb')
 fp.write('%i\n' % cls._version)
 fp.write('%s\n' % name)
@@ -257,6 +264,7 @@ class shelvedstate(object):
 fp.write('%s\n' % branchtorestore)
 fp.write('%s\n' % (cls._keep if keep else cls._nokeep))
 fp.write('%s\n' % (cls._obsbased if obsshelve else cls._traditional))
+fp.write('%s\n' % (activebook or cls._noactivebook))
 fp.close()
 
 @classmethod
@@ -295,6 +303,16 @@ def cleanupoldbackups(repo):
 if err.errno != errno.ENOENT:
 raise
 
+def _backupactivebookmark(repo):
+activebookmark = repo._activebookmark
+if activebookmark:
+bookmarks.deactivate(repo)
+return activebookmark
+
+def _restoreactivebookmark(repo, mark):
+if mark:
+bookmarks.activate(repo, mark)
+
 def _aborttransaction(repo):
 '''Abort current transaction for shelve/unshelve, but keep dirstate
 '''
@@ -410,7 +428,7 @@ def _includeunknownfiles(repo, pats, opt
 extra['shelve_unknown'] = '\0'.join(s.unknown)
 repo[None].add(s.unknown)
 
-def _finishshelve(ui, repo, tr):
+def _finishshelve(ui, repo, tr, activebookmark):
 if isobsshelve(repo, ui):
 tr.close()
 tr.release()
@@ -433,7 +451,7 @@ def _docreatecmd(ui, repo, pats, opts):
 if not opts.get('message'):
 opts['message'] = desc
 
-lock = tr = None
+lock = tr = activebookmark = None
 try:
 lock = repo.lock()
 
@@ -449,6 +467,7 @@ def _docreatecmd(ui, repo, pats, opts):
   not opts.get('addremove', False))
 
 name = getshelvename(repo, parent, opts)
+activebookmark = _backupactivebookmark(repo)
 extra = {}
 if includeunknown:
 _includeunknownfiles(repo, pats, opts, extra)
@@ -463,7 +482,8 @@ def _docreatecmd(ui, repo, pats, opts):
 node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
 else:
 node = cmdutil.dorecord(ui, repo, commitfunc, None,
-False, cmdutil.recordfilter, *pats, **opts)
+False, cmdutil.recordfilter, *pats,
+**opts)
 if not node:
 _nothingtoshelvemessaging(ui, repo, pats, opts)
 return 1
@@ -477,8 +497,9 @@ def _docreatecmd(ui, repo, pats, opts):
 if origbranch != repo['.'].branch() and not _isbareshelve(pats, opts):
 repo.dirstate.setbranch(origbranch)
 
-_finishshelve(ui, repo, tr)
+_finishshelve(ui, repo, tr, activebookmark)
 fina

Re: mercurial@30541: 92 new changesets

[Kostia Balytskyi]

On 11/28/16, 12:16 AM, "Mercurial-devel on behalf of Mercurial Commits" 
 
wrote:

92 new changesets in mercurial:

http://selenic.com/repo/hg//rev/0acf3fd718f1
changeset:   30450:0acf3fd718f1
user:Gregory Szorc 
date:Thu Nov 17 20:09:10 2016 -0800
summary: setup: add flag to build_ext to control building zstd

http://selenic.com/repo/hg//rev/94ca0e13d1fc
changeset:   30451:94ca0e13d1fc
user:Gregory Szorc 
date:Thu Nov 17 20:17:51 2016 -0800
summary: perf: add command for measuring revlog chunk operations

http://selenic.com/repo/hg//rev/932b18c95e11
changeset:   30452:932b18c95e11
user:Gregory Szorc 
date:Thu Nov 17 20:30:00 2016 -0800
summary: commands: print chunk type in debugrevlog

http://selenic.com/repo/hg//rev/2e736f01a710
changeset:   30453:2e736f01a710
user:    Kostia Balytskyi 
date:Sun Nov 13 03:35:52 2016 -0800
summary: shelve: move temporary commit creation to a separate function

http://selenic.com/repo/hg//rev/672026aece64
changeset:   30454:672026aece64
user:    Kostia Balytskyi 
date:Thu Nov 10 10:51:06 2016 -0800
summary: shelve: move commit restoration logic to a separate function

http://selenic.com/repo/hg//rev/10684a298973
changeset:   30455:10684a298973
user:    Kostia Balytskyi 
date:Thu Nov 10 10:57:10 2016 -0800
summary: shelve: move rebasing logic to a separate function

http://selenic.com/repo/hg//rev/b924375cce3a
changeset:   30456:b924375cce3a
user:    Kostia Balytskyi 
date:Thu Nov 10 11:02:39 2016 -0800
summary: shelve: move file-forgetting logic to a separate function

http://selenic.com/repo/hg//rev/893be22cdb38
changeset:   30457:893be22cdb38
user:    Kostia Balytskyi 
date:Fri Nov 11 07:01:27 2016 -0800
summary: shelve: move unshelve-finishing logic to a separate function

http://selenic.com/repo/hg//rev/0df215fba6cf
changeset:   30458:0df215fba6cf
user:Gábor Stefanik 
date:Fri Oct 28 17:44:28 2016 +0200
summary: setup: include a dummy $PATH in the custom environment used by 
build.py

http://selenic.com/repo/hg//rev/bccd89b46cbf
changeset:   30459:bccd89b46cbf
user:Durham Goode 
date:Thu Nov 10 09:21:41 2016 -0800
summary: rebase: move bookmark update to before rebase clearing

http://selenic.com/repo/hg//rev/ce3a133f71b3
changeset:   30460:ce3a133f71b3
user:    Kostia Balytskyi 
date:Sat Nov 19 15:41:37 2016 -0800
summary: conflicts: make spacing consistent in conflict markers

http://selenic.com/repo/hg//rev/d195fa651b51
changeset:   30461:d195fa651b51
user:Gregory Szorc 
date:Sun Nov 20 16:56:21 2016 -0800
summary: bdiff: don't check border condition in loop

http://selenic.com/repo/hg//rev/356406ac454f
changeset:   30462:356406ac454f
user:Gregory Szorc 
date:Sat Nov 19 10:54:21 2016 -0800
summary: debuginstall: print compression engine support

http://selenic.com/repo/hg//rev/bc0def54c17d
changeset:   30463:bc0def54c17d
user:Gregory Szorc 
date:Sat Nov 19 17:11:12 2016 -0800
summary: keepalive: reorder header precedence

http://selenic.com/repo/hg//rev/e16e234b9ca3
changeset:   30464:e16e234b9ca3
user:Gregory Szorc 
date:Sat Nov 19 18:31:40 2016 -0800
summary: httppeer: do decompression inside _callstream

http://selenic.com/repo/hg//rev/40a1871eea5e
changeset:   30465:40a1871eea5e
user:Gregory Szorc 
date:Sun Nov 20 13:55:53 2016 -0800
summary: httppeer: use compression engine API for decompressing 
responses

http://selenic.com/repo/hg//rev/2add671bf55b
changeset:   30466:2add671bf55b
user:Gregory Szorc 
date:Sun Nov 20 13:50:45 2016 -0800
summary: wireproto: perform chunking and compression at protocol layer 
(API)

http://selenic.com/repo/hg//rev/5b0baa9f3362
changeset:   30467:5b0baa9f3362
user:Pulkit Goyal <7895pul...@gmail.com>
date:Mon Nov 21 15:26:47 2016 +0530
summary: py3: use pycompat.sysargv in scmposix.systemrcpath()

http://selenic.com/repo/hg//rev/7f2b18c34c02
changeset:   30468:7f2b18c34c02
user:Pulkit Goyal <7895pul...@gmail.com>
date:Mon Nov 21 15:35:22 2016 +0530
summary: py3: use pycompat.sysargv in dispatch.run()

http://selenic.com/repo/hg//rev/8b3ad0252344
changeset:   30469:8b3ad0252344
user:Pulkit Goyal <7895pul...@gmail.com>
date:Mon Nov 21 15:38:56 2016 +0530
su

Re: [PATCH evolve-ext] wireproto: chunking and compression is forthwith to be handled by hgweb

[Kostia Balytskyi]

This looks good to me, although I would maybe perform a more thorough check 
than just catching a TypeError. For example, something like check:
if “v1compressible” in wireproto.streamres.__init__.__code__.co_varnames:
pass this arg
else:
don’t pass it


On 11/29/16, 4:10 PM, "Mercurial-devel on behalf of Martijn Pieters" 
 
wrote:

# HG changeset patch
# User Martijn Pieters 
# Date 1480435818 0
#  Tue Nov 29 16:10:18 2016 +
# Node ID 28a092ed406e930894c59eb88d645221abddc307
# Parent  cb2bac3253fbd52894ffcb4719a148fe6a3da38b
wireproto: chunking and compression is forthwith to be handled by hgweb

Various functions disappeared in the process. Use the new streamres API but 
fall back to the old way if the keyword arguments are not accepted.

See 
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_repo_hg_rev_2add671bf55b&d=DgIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=itsfqjECZoIFxlvcK1F6-xHmasBtG5DrNpHJ5crQ8Qk&s=L9z5Oyy5EDFtbdleAKmWKcYdz4ScmzGLZznUUuk5k_8&e=
 

diff --git a/hgext/evolve.py b/hgext/evolve.py
--- a/hgext/evolve.py
+++ b/hgext/evolve.py
@@ -3848,7 +3848,12 @@
 finaldata.write('%20i' % len(obsdata))
 finaldata.write(obsdata)
 finaldata.seek(0)
-return wireproto.streamres(proto.groupchunks(finaldata))
+try:
+return wireproto.streamres(reader=finaldata, v1compressible=True)
+except TypeError:
+# older mercurial version, expected to do our own compression
+return wireproto.streamres(proto.groupchunks(finaldata))
+
 
 def _obsrelsethashtreefm0(repo):
 return _obsrelsethashtree(repo, obsolete._fm0encodeonemarker)
diff --git a/hgext/simple4server.py b/hgext/simple4server.py
--- a/hgext/simple4server.py
+++ b/hgext/simple4server.py
@@ -175,7 +175,11 @@
 finaldata.write('%20i' % len(obsdata))
 finaldata.write(obsdata)
 finaldata.seek(0)
-return wireproto.streamres(proto.groupchunks(finaldata))
+try:
+return wireproto.streamres(reader=finaldata, v1compressible=True)
+except TypeError:
+# older mercurial version, expected to do our own compression
+return wireproto.streamres(proto.groupchunks(finaldata))
 
 
 # from evolve extension: 3249814dabd1
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel&d=DgIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=itsfqjECZoIFxlvcK1F6-xHmasBtG5DrNpHJ5crQ8Qk&s=Ja50nSg2ysFrAQC62ZhOs8UsS5w1-omMglM6HYBbfjg&e=
 


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 02 of 10 shelve-ext] shelve: add an ability to write json to a new type of shelve files

[Kostia Balytskyi]

On 11/29/16, 3:55 PM, "Jun Wu"  wrote:

Excerpts from Kostia Balytskyi's message of 2016-11-29 07:22:56 -0800:
> # HG changeset patch
    > # User Kostia Balytskyi 
> # Date 1480426193 28800
> #  Tue Nov 29 05:29:53 2016 -0800
> # Node ID 37119e028c699d9fabd220086e08c754827e709f
> # Parent  f6f0ab3f7b0ea0e05cfdcd7afd4994ea21988fd9
> shelve: add an ability to write json to a new type of shelve files
> 
> Obsolescense-based shelve only needs metadata stored in .hg/shelved
> and I think that this metadata should be stored in json for
> potential extensibility purposes. JSON is not critical here, but
> I want to avoid storing it in an unstructured text file where
> order of lines determines their semantical meanings (as now
> happens in .hg/shelvedstate. .hg/rebasestate and I suspect other
> state files as well).
> 
> If we want, we can refactor it to something else later.
> 
> diff --git a/hgext/shelve.py b/hgext/shelve.py
> --- a/hgext/shelve.py
> +++ b/hgext/shelve.py
> @@ -25,6 +25,7 @@ from __future__ import absolute_import
>  import collections
>  import errno
>  import itertools
> +import json

I think we avoid using "import json" for some reason (encoding?). There is
no "import json" in the code base yet.

>  from mercurial.i18n import _
>  from mercurial import (
> @@ -62,7 +63,7 @@ testedwith = 'ships-with-hg-core'
>  
>  backupdir = 'shelve-backup'
>  shelvedir = 'shelved'
> -shelvefileextensions = ['hg', 'patch']
> +shelvefileextensions = ['hg', 'patch', 'oshelve']
>  # universal extension is present in all types of shelves
>  patchextension = 'patch'
>  
> @@ -153,6 +154,26 @@ class shelvedfile(object):
>  bundle2.writebundle(self.ui, cg, self.fname, btype, self.vfs,
>  compression=compression)
>  
> +def writejson(self, jsn):
> +fp = self.opener('wb')
> +try:
> +fp.write(json.dumps(jsn))
> +finally:
> +fp.close()
> +
> +def readjson(self):
> +fp = self.opener()
> +contents = None
> +try:
> +contents = fp.read()
> +finally:
> +fp.close()
> +try:
> +jsn = json.loads(contents)
> +except (TypeError, ValueError):
> +raise error.abort(_('could not read obsolescense-based 
shelve'))

error.Abort

The method does not seem to be related to "obsolescense" just from the name
and logic. So the error message would be inaccurate if callers use
"writejson" to write other things.

Given the fact "json" is a too generic name and we may replace it using
other (lightweight) format in the future, it may be better to rename those
methods to something more specific, and avoid exposing the "json" format
to the caller, like:

def writeobsinfo(self, node):


def readobsinfo(self):

Renaming it makes sense, I will do that in a following patch. I did not get 
what to do with json though: should I try to replace it with some other format 
right away or is it fine to leave json for now?

> +return jsn
> +
>  class shelvedstate(object):
>  """Handle persistence during unshelving operations.
>  


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH V2] evolve: improve error message if unstable changes are disallowed

[Kostia Balytskyi]

Shouldn’t we mention the config in a hint message?

On 11/25/16, 12:28 PM, "Mercurial-devel on behalf of Mateusz Kwapich" 
 wrote:

LGTM, I suppose that timeless may also want to review it before it's
queued.

Excerpts from Pulkit Goyal's message of 2016-11-24 23:13:48 +0530:
> # HG changeset patch
> # User Pulkit Goyal <7895pul...@gmail.com>
> # Date 1479915042 -19800
> #  Wed Nov 23 21:00:42 2016 +0530
> # Node ID 920d5946d13339d9cf4828f678fb55063cd8
> # Parent  cb2bac3253fbd52894ffcb4719a148fe6a3da38b
> evolve: improve error message if unstable changes are disallowed
> 
> I saw a question on stackoverflow why evolve reports something like cannot
> fold chain not ending with head. Even I was confused the first time about 
the
> behavior. The error message can be improved to avoid confusion to people 
who
> are unaware about the config in future.
> 
> diff -r cb2bac3253fb -r 920d5946d133 hgext/evolve.py
> --- a/hgext/evolve.pyWed Nov 02 18:56:44 2016 +0100
> +++ b/hgext/evolve.pyWed Nov 23 21:00:42 2016 +0530
> @@ -2514,7 +2514,8 @@
>  raise error.Abort('nothing to prune')
>  
>  if _disallowednewunstable(repo, revs):
> -raise error.Abort(_("cannot prune in the middle of a stack"))
> +raise error.Abort(_("cannot prune in the middle of a stack"),
> +hint = _("new unstable changesets are not 
allowed"))
>  
>  # defines successors changesets
>  sucs = scmutil.revrange(repo, succs)
> @@ -3234,8 +3235,9 @@
>  newunstable = _disallowednewunstable(repo, revs)
>  if newunstable:
>  raise error.Abort(
> -_('cannot edit commit information in the middle of a 
stack'),
> -hint=_('%s will be affected') % 
repo[newunstable.first()])
> +_('cannot edit commit information in the middle of a 
'\
> +'stack'), hint=_('%s will become unstable and new 
unstable'\
> +' changes are not allowed') % 
repo[newunstable.first()])
>  root = head = repo[revs.first()]
>  
>  wctx = repo[None]
> @@ -3299,7 +3301,8 @@
>  head = repo[heads.first()]
>  if _disallowednewunstable(repo, revs):
>  raise error.Abort(_("cannot fold chain not ending with a head "\
> -"or with branching"))
> +"or with branching"), hint = _("new 
unstable"\
> +" changesets are not allowed"))
>  return root, head
>  
>  def _disallowednewunstable(repo, revs):
> diff -r cb2bac3253fb -r 920d5946d133 tests/test-evolve.t
> --- a/tests/test-evolve.tWed Nov 02 18:56:44 2016 +0100
> +++ b/tests/test-evolve.tWed Nov 23 21:00:42 2016 +0530
> @@ -1301,9 +1301,11 @@
>created new head
>$ hg prune '26 + 27'
>abort: cannot prune in the middle of a stack
> +  (new unstable changesets are not allowed)
>[255]
>$ hg prune '19::28'
>abort: cannot prune in the middle of a stack
> +  (new unstable changesets are not allowed)
>[255]
>$ hg prune '26::'
>3 changesets pruned
> @@ -1338,9 +1340,11 @@
>  
>$ hg fold --exact "19 + 18"
>abort: cannot fold chain not ending with a head or with branching
> +  (new unstable changesets are not allowed)
>[255]
>$ hg fold --exact "18::29"
>abort: cannot fold chain not ending with a head or with branching
> +  (new unstable changesets are not allowed)
>[255]
>$ hg fold --exact "19::"
>2 changesets folded
> @@ -1483,10 +1487,11 @@
>  check that metaedit respects allowunstable
>$ hg metaedit '.^' --config 'experimental.evolution=createmarkers, 
allnewcommands'
>abort: cannot edit commit information in the middle of a stack
> -  (c904da5245b0 will be affected)
> +  (c904da5245b0 will become unstable and new unstable changes are not 
allowed)
>[255]
>$ hg metaedit '18::20' --fold --config 
'experimental.evolution=createmarkers, allnewcommands'
>abort: cannot fold chain not ending with a head or with branching
> +  (new unstable changesets are not allowed)
>[255]
>$ hg metaedit --user foobar
>0 files updated, 0 files merged, 0 files removed, 0 files unresolved

-- 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel&d=DgIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=QukFSh3cja_kc0KmG_KvCqtbq77q1UqiBomG5m3Nr5I&s=ywLFcN2fvyXbzsLQIBB1

Re: [PATCH V2 evolve-ext] wireproto: chunking and compression is forthwith to be handled by hgweb

[Kostia Balytskyi]

This looks good to me.

On 11/30/16, 11:26 AM, "Mercurial-devel on behalf of Martijn Pieters" 
 
wrote:

# HG changeset patch
# User Martijn Pieters 
# Date 1480504675 0
#  Wed Nov 30 11:17:55 2016 +
# Node ID 5bfbd771d6348b10edabfbd2c61f412bb66c87ae
# Parent  cb2bac3253fbd52894ffcb4719a148fe6a3da38b
wireproto: chunking and compression is forthwith to be handled by hgweb

Various functions disappeared in the process. Use the new streamres API but 
fall back to the old way if the keyword arguments are not accepted.

See 
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_repo_hg_rev_2add671bf55b&d=DgIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=eh7UoQJLleKIlXLG7SxQZ0wZ7Anq0z7vSMVdUR1zfx0&s=_sho53Fr0LJ0WryHbP2N8y4mPGKLmSEAPheNoV5ad54&e=
 

diff --git a/hgext/evolve.py b/hgext/evolve.py
--- a/hgext/evolve.py
+++ b/hgext/evolve.py
@@ -3837,6 +3837,17 @@
 def local_pullobsmarkers(self, heads=None, common=None):
 return _getobsmarkersstream(self._repo, heads=heads, common=common)
 
+# The wireproto.streamres API changed, handling chunking and compression
+# directly. Handle either case.
+if util.safehasattr(wireproto.abstractserverproto, 'groupchunks'):
+# We need to handle chunking and compression directly
+def streamres(d, proto):
+wireproto.streamres(proto.groupchunks(d))
+else:
+# Leave chunking and compression to streamres
+def streamres(d, proto):
+wireproto.streamres(reader=d, v1compressible=True)
+
 def srv_pullobsmarkers(repo, proto, others):
 opts = wireproto.options('', ['heads', 'common'], others)
 for k, v in opts.iteritems():
@@ -3848,7 +3859,7 @@
 finaldata.write('%20i' % len(obsdata))
 finaldata.write(obsdata)
 finaldata.seek(0)
-return wireproto.streamres(proto.groupchunks(finaldata))
+return streamres(finaldata, proto)
 
 def _obsrelsethashtreefm0(repo):
 return _obsrelsethashtree(repo, obsolete._fm0encodeonemarker)
diff --git a/hgext/simple4server.py b/hgext/simple4server.py
--- a/hgext/simple4server.py
+++ b/hgext/simple4server.py
@@ -158,6 +158,17 @@
 seennodes |= pendingnodes
 return seenmarkers
 
+# The wireproto.streamres API changed, handling chunking and compression
+# directly. Handle either case.
+if util.safehasattr(wireproto.abstractserverproto, 'groupchunks'):
+# We need to handle chunking and compression directly
+def streamres(d, proto):
+return wireproto.streamres(proto.groupchunks(d))
+else:
+# Leave chunking and compression to streamres
+def streamres(d, proto):
+return wireproto.streamres(reader=d, v1compressible=True)
+
 # from evolve extension: cf35f38d6a10
 def srv_pullobsmarkers(repo, proto, others):
 """serves a binary stream of markers.
@@ -175,7 +186,7 @@
 finaldata.write('%20i' % len(obsdata))
 finaldata.write(obsdata)
 finaldata.seek(0)
-return wireproto.streamres(proto.groupchunks(finaldata))
+return streamres(finaldata, proto)
 
 
 # from evolve extension: 3249814dabd1
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mercurial-2Dscm.org_mailman_listinfo_mercurial-2Ddevel&d=DgIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=Pp-gQYFgs4tKlSFPF5kfCw&m=eh7UoQJLleKIlXLG7SxQZ0wZ7Anq0z7vSMVdUR1zfx0&s=yT9FG-ApJKXTq23h612kHxLfohTTYnkwwFOcWvd7jys&e=
 


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 09 of 10 shelve-ext] shelve: add obs-based unshelve functionality

[Kostia Balytskyi]

From: Martin von Zweigbergk 
Date: Wednesday, November 30, 2016 at 6:24 AM
To: Kostia Balytskyi , "mercurial-devel@mercurial-scm.org" 

Subject: Re: [PATCH 09 of 10 shelve-ext] shelve: add obs-based unshelve 
functionality


On Tue, Nov 29, 2016, 08:23 Kostia Balytskyi 
mailto:ikos...@fb.com>> wrote:
# HG changeset patch
# User Kostia Balytskyi mailto:ikos...@fb.com>>
# Date 1480431173 28800
#  Tue Nov 29 06:52:53 2016 -0800
# Node ID 533d99eca3bf11c4aac869674e0abb16b74ed670
# Parent  85c9c651887915733feb3d385866955741f28ec0
shelve: add obs-based unshelve functionality

Obsolescense-based unshelve works as follows:
1. Instead of stripping temporary nodes, markers are created to
obsolete them.
2. Restoring commit is just finding it in an unfiltered repo.

What happens if it's been stripped? Speaking of that, I didn't see any tests 
added in this series. Seems like there should be some. Did I just miss them?

I have an entire test-shelve.t ported to test obs-based shelve (+ some more new 
tests), I am going to send out tests in a separate series as there will be many 
of them, I think I mentioned that in the first patch of this series.

I did not handle the stripped case because we usually don’t strip things if 
createmarkers is enabled, thus it should be a rare enough case which can only 
be caused by shooting yourself in the foot. Having that said, it’s not hard to 
add a nicer error message. Will do that in v2.

3. '--keep' is only passed to rebase on traditional unshelves
(and thus traditional rebases), becuase we want markers to be
created fro obsolete-based rebases.
4. 'hg unshelve' uses unfiltered repo to perform rebases
because we want rebase to be able to create markers between original
and new commits. 'rebaseskipobsolete' is disabled to make rebase not
skip the commit altogether.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -26,6 +26,7 @@ import collections
 import errno
 import itertools
 import json
+import time

 from mercurial.i18n import _
 from mercurial import (
@@ -264,8 +265,13 @@ class shelvedstate(object):

 def prunenodes(self):
 """Cleanup temporary nodes from the repo"""
-repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
- topic='shelve')
+if self.obsshelve:
+unfi = self.repo.unfiltered()
+relations = [(unfi[n], ()) for n in self.nodestoprune]
+obsolete.createmarkers(self.repo, relations)
+else:
+repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
+ topic='shelve')

 def cleanupoldbackups(repo):
 vfs = scmutil.vfs(repo.join(backupdir))
@@ -670,9 +676,14 @@ def unshelvecontinue(ui, repo, state, op
 util.rename(repo.join('unshelverebasestate'),
 repo.join('rebasestate'))
 try:
-rebase.rebase(ui, repo, **{
-'continue' : True
-})
+# if shelve is obs-based, we want rebase to be able
+# to create markers to already-obsoleted commits
+_repo = repo.unfiltered() if state.obsshelve else repo
+with ui.configoverride({('experimental', 'rebaseskipobsolete'):
+'off'}, 'unshelve'):
+rebase.rebase(ui, _repo, **{
+'continue' : True,
+})
 except Exception:
 util.rename(repo.join('rebasestate'),
 repo.join('unshelverebasestate'))
@@ -712,30 +723,54 @@ def _commitworkingcopychanges(ui, repo,
 with ui.configoverride({('ui', 'quiet'): True}):
 node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
 tmpwctx = repo[node]
+ui.debug("temporary working copy commit: %s:%s\n" %
+ (tmpwctx.rev(), nodemod.short(node)))
 return tmpwctx, addedbefore

-def _unshelverestorecommit(ui, repo, basename):
+def _unshelverestorecommit(ui, repo, basename, obsshelve, shfile):
 """Recreate commit in the repository during the unshelve"""
 with ui.configoverride({('ui', 'quiet'): True}):
-shelvedfile(repo, basename, 'hg').applybundle()
-shelvectx = repo['tip']
+if obsshelve:
+md = shfile.readjson()
+shelvenode = nodemod.bin(md['node'])
+repo = repo.unfiltered()
+shelvectx = repo[shelvenode]
+else:
+shelvedfile(repo, basename, 'hg').applybundle()
+shelvectx = repo['tip']
 return repo, shelvectx

 def _rebaserestoredcommit(ui, repo, opts, tr, oldtiprev, basename, pctx,

Re: [PATCH 09 of 10 shelve-ext] shelve: add obs-based unshelve functionality

[Kostia Balytskyi]

Replying one more time to make stuff readable.

On 11/30/16 1:46 PM, Kostia Balytskyi wrote:

From: Martin von Zweigbergk 
<mailto:martinv...@google.com>
Date: Wednesday, November 30, 2016 at 6:24 AM
To: Kostia Balytskyi <mailto:ikos...@fb.com>, 
"mercurial-devel@mercurial-scm.org"<mailto:mercurial-devel@mercurial-scm.org> 
<mailto:mercurial-devel@mercurial-scm.org>
Subject: Re: [PATCH 09 of 10 shelve-ext] shelve: add obs-based unshelve 
functionality


On Tue, Nov 29, 2016, 08:23 Kostia Balytskyi 
mailto:ikos...@fb.com><mailto:ikos...@fb.com><mailto:ikos...@fb.com>>
 wrote:
# HG changeset patch
# User Kostia Balytskyi 
mailto:ikos...@fb.com><mailto:ikos...@fb.com><mailto:ikos...@fb.com>>
# Date 1480431173 28800
#  Tue Nov 29 06:52:53 2016 -0800
# Node ID 533d99eca3bf11c4aac869674e0abb16b74ed670
# Parent  85c9c651887915733feb3d385866955741f28ec0
shelve: add obs-based unshelve functionality

Obsolescense-based unshelve works as follows:
1. Instead of stripping temporary nodes, markers are created to
obsolete them.
2. Restoring commit is just finding it in an unfiltered repo.

What happens if it's been stripped? Speaking of that, I didn't see any tests 
added in this series. Seems like there should be some. Did I just miss them?

I have an entire test-shelve.t ported to test obs-based shelve (+ some more new 
tests), I am going to send out tests in a separate series as there will be many 
of them, I think I mentioned that in the first patch of this series.

I did not handle the stripped case because we usually don’t strip things if 
createmarkers is enabled, thus it should be a rare enough case which can only 
be caused by shooting yourself in the foot. Having that said, it’s not hard to 
add a nicer error message. Will do that in v2.



3. '--keep' is only passed to rebase on traditional unshelves
(and thus traditional rebases), becuase we want markers to be
created fro obsolete-based rebases.
4. 'hg unshelve' uses unfiltered repo to perform rebases
because we want rebase to be able to create markers between original
and new commits. 'rebaseskipobsolete' is disabled to make rebase not
skip the commit altogether.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -26,6 +26,7 @@ import collections
 import errno
 import itertools
 import json
+import time

 from mercurial.i18n import _
 from mercurial import (
@@ -264,8 +265,13 @@ class shelvedstate(object):

 def prunenodes(self):
 """Cleanup temporary nodes from the repo"""
-repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
- topic='shelve')
+if self.obsshelve:
+unfi = self.repo.unfiltered()
+relations = [(unfi[n], ()) for n in self.nodestoprune]
+obsolete.createmarkers(self.repo, relations)
+else:
+repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
+ topic='shelve')

 def cleanupoldbackups(repo):
 vfs = scmutil.vfs(repo.join(backupdir))
@@ -670,9 +676,14 @@ def unshelvecontinue(ui, repo, state, op
 util.rename(repo.join('unshelverebasestate'),
 repo.join('rebasestate'))
 try:
-rebase.rebase(ui, repo, **{
-'continue' : True
-})
+# if shelve is obs-based, we want rebase to be able
+# to create markers to already-obsoleted commits
+_repo = repo.unfiltered() if state.obsshelve else repo
+with ui.configoverride({('experimental', 'rebaseskipobsolete'):
+'off'}, 'unshelve'):
+rebase.rebase(ui, _repo, **{
+'continue' : True,
+})
 except Exception:
 util.rename(repo.join('rebasestate'),
 repo.join('unshelverebasestate'))
@@ -712,30 +723,54 @@ def _commitworkingcopychanges(ui, repo,
 with ui.configoverride({('ui', 'quiet'): True}):
 node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
 tmpwctx = repo[node]
+ui.debug("temporary working copy commit: %s:%s\n" %
+ (tmpwctx.rev(), nodemod.short(node)))
 return tmpwctx, addedbefore

-def _unshelverestorecommit(ui, repo, basename):
+def _unshelverestorecommit(ui, repo, basename, obsshelve, shfile):
 """Recreate commit in the repository during the unshelve"""
 with ui.configoverride({('ui', 'quiet'): True}):
-shelvedfile(repo, basename, 'hg').applybundle()
-shelvectx = repo['tip']
+if obsshelve:
+md = sh

Re: [PATCH 1 of 2] tests: use cp -R instead of cp -r

[Kostia Balytskyi]

Good catch!

On 11/30/16 5:34 PM, Jun Wu wrote:
> # HG changeset patch
> # User Jun Wu 
> # Date 1480526736 0
> #  Wed Nov 30 17:25:36 2016 +
> # Node ID a4f46a03e5f8e33f32c36326c6b14b84d9cb03b4
> # Parent  9e29d4e4e08b5996adda49cdd0b497d89e2b16ee
> # Available At https://bitbucket.org/quark-zju/hg-draft
> #  hg pull https://bitbucket.org/quark-zju/hg-draft -r 
> a4f46a03e5f8
> tests: use cp -R instead of cp -r
>
> This makes OS X happy for some tests after 0d87b1caed92. The default
> behavior of -R (-P, not follow symlinks) is what we want.
>
> "cp -r" is not documented in the OSX cp man page, and is not defined by
> POSIX. The next patch will add a check-code rule to forbid it.
>
> diff --git a/tests/test-bookmarks.t b/tests/test-bookmarks.t
> --- a/tests/test-bookmarks.t
> +++ b/tests/test-bookmarks.t
> @@ -573,6 +573,6 @@ pull --update works the same as pull &&
> $ hg bookmark -r3 Y
> moving bookmark 'Y' forward from db815d6d32e6
> -  $ cp -r  ../cloned-bookmarks-update ../cloned-bookmarks-manual-update
> -  $ cp -r  ../cloned-bookmarks-update 
> ../cloned-bookmarks-manual-update-with-divergence
> +  $ cp -R ../cloned-bookmarks-update ../cloned-bookmarks-manual-update
> +  $ cp -R ../cloned-bookmarks-update 
> ../cloned-bookmarks-manual-update-with-divergence
>   
>   (manual version)
> diff --git a/tests/test-histedit-obsolete.t b/tests/test-histedit-obsolete.t
> --- a/tests/test-histedit-obsolete.t
> +++ b/tests/test-histedit-obsolete.t
> @@ -340,5 +340,5 @@ phases.new-commit option is.
>   New-commit as draft (default)
>   
> -  $ cp -r base simple-draft
> +  $ cp -R base simple-draft
> $ cd simple-draft
> $ hg histedit -r 'b449568bf7fc' --commands - << EOF
> @@ -379,5 +379,5 @@ New-commit as draft (default)
>   New-commit as secret (config)
>   
> -  $ cp -r base simple-secret
> +  $ cp -R base simple-secret
> $ cd simple-secret
> $ cat >> .hg/hgrc << EOF
> @@ -426,5 +426,5 @@ If a secret changeset is put before a dr
>   It seems more important to present the secret phase.
>   
> -  $ cp -r base reorder
> +  $ cp -R base reorder
> $ cd reorder
> $ hg histedit -r 'b449568bf7fc' --commands - << EOF
> @@ -463,5 +463,5 @@ better safe than sorry). Folding between
>   Note that there is a few reordering in this series for more extensive test
>   
> -  $ cp -r base folding
> +  $ cp -R base folding
> $ cd folding
> $ cat >> .hg/hgrc << EOF
> diff --git a/tests/test-obsolete-checkheads.t 
> b/tests/test-obsolete-checkheads.t
> --- a/tests/test-obsolete-checkheads.t
> +++ b/tests/test-obsolete-checkheads.t
> @@ -24,5 +24,5 @@ Check that obsolete properly strip heads
> $ hg phase --public .
> $ cd ..
> -  $ cp -r remote base
> +  $ cp -R remote base
> $ hg clone remote local
> updating to branch default
> @@ -55,5 +55,5 @@ setup
> o  b4952fcf48cf (public) add base
> 
> -  $ cp -r ../remote ../backup1
> +  $ cp -R ../remote ../backup1
>   
>   old exists remotely as draft. It is obsoleted by new that we now push.
> @@ -74,5 +74,5 @@ setup
>   
> $ rm -fr ../remote
> -  $ cp -r ../backup1 ../remote
> +  $ cp -R ../backup1 ../remote
> $ hg -R ../remote phase --public c70b08862e08
> $ hg pull -v
> @@ -105,5 +105,5 @@ TODO: Not implemented yet.
>   #
>   #   $ rm -fr ../remote
> -#   $ cp -r ../backup1 ../remote
> +#   $ cp -R ../backup1 ../remote
>   #   $ hg -R ../remote phase --public c70b08862e08
>   #   $ hg phase --draft --force c70b08862e08
> @@ -132,5 +132,5 @@ setup
>   
> $ rm -fr ../remote
> -  $ cp -r ../backup1 ../remote
> +  $ cp -R ../backup1 ../remote
> $ hg phase --draft --force '(0::) - 0'
> $ hg up -q '.^'
> @@ -207,5 +207,5 @@ setup. (The obsolete marker is known loc
> @  b4952fcf48cf (public) add base
> 
> -  $ cp -r ../remote ../backup2
> +  $ cp -R ../remote ../backup2
>   
>   Push should not warn about adding new heads. We create one, but we'll delete
> @@ -227,5 +227,5 @@ setup
>   
> $ rm -fr ../remote
> -  $ cp -r ../backup1 ../remote
> +  $ cp -R ../backup1 ../remote
> $ cd ..
> $ rm -rf local
> diff --git a/tests/test-revert.t b/tests/test-revert.t
> --- a/tests/test-revert.t
> +++ b/tests/test-revert.t
> @@ -785,5 +785,5 @@ Test revert --all to parent content
>   (setup from reference repo)
>   
> -  $ cp -r revert-ref revert-parent-all
> +  $ cp -R revert-ref revert-parent-all
> $ cd revert-parent-all
>   
> @@ -842,5 +842,5 @@ Test revert --all to "base" content
>   (setup from reference repo)
>   
> -  $ cp -r revert-ref revert-base-all
> +  $ cp -R revert-ref revert-base-all
> $ cd revert-base-all
>   
> @@ -897,5 +897,5 @@ Test revert to parent content with expli
>   (setup from reference repo)
>   
> -  $ cp -r revert-ref revert-parent-explicit
> +  $ cp -R revert-ref revert-parent-explicit
> $ cd revert-parent-explicit
>   
> @@ -990,5 +990,5 @@ Test revert to "base" content with expli
>   (setup from reference repo)
>   
> -  $ cp -r rev

Re: [PATCH 09 of 10 shelve-ext] shelve: add obs-based unshelve functionality

[Kostia Balytskyi]

On 11/29/16 4:26 PM, Jun Wu wrote:
> Excerpts from Kostia Balytskyi's message of 2016-11-29 07:23:03 -0800:
>> # HG changeset patch
>> # User Kostia Balytskyi 
>> # Date 1480431173 28800
>> #  Tue Nov 29 06:52:53 2016 -0800
>> # Node ID 533d99eca3bf11c4aac869674e0abb16b74ed670
>> # Parent  85c9c651887915733feb3d385866955741f28ec0
>> shelve: add obs-based unshelve functionality
>>
>> Obsolescense-based unshelve works as follows:
>> 1. Instead of stripping temporary nodes, markers are created to
>> obsolete them.
>> 2. Restoring commit is just finding it in an unfiltered repo.
>> 3. '--keep' is only passed to rebase on traditional unshelves
>> (and thus traditional rebases), becuase we want markers to be
>> created fro obsolete-based rebases.
>> 4. 'hg unshelve' uses unfiltered repo to perform rebases
>> because we want rebase to be able to create markers between original
>> and new commits. 'rebaseskipobsolete' is disabled to make rebase not
>> skip the commit altogether.
>>
>> diff --git a/hgext/shelve.py b/hgext/shelve.py
>> --- a/hgext/shelve.py
>> +++ b/hgext/shelve.py
>> @@ -26,6 +26,7 @@ import collections
>>   import errno
>>   import itertools
>>   import json
>> +import time
>>   
>>   from mercurial.i18n import _
>>   from mercurial import (
>> @@ -264,8 +265,13 @@ class shelvedstate(object):
>>   
>>   def prunenodes(self):
>>   """Cleanup temporary nodes from the repo"""
>> -repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
>> - topic='shelve')
>> +if self.obsshelve:
>> +unfi = self.repo.unfiltered()
>> +relations = [(unfi[n], ()) for n in self.nodestoprune]
>> +obsolete.createmarkers(self.repo, relations)
>> +else:
>> +repair.strip(self.ui, self.repo, self.nodestoprune, 
>> backup=False,
>> + topic='shelve')
>>   
>>   def cleanupoldbackups(repo):
>>   vfs = scmutil.vfs(repo.join(backupdir))
>> @@ -670,9 +676,14 @@ def unshelvecontinue(ui, repo, state, op
>>   util.rename(repo.join('unshelverebasestate'),
>>   repo.join('rebasestate'))
>>   try:
>> -rebase.rebase(ui, repo, **{
>> -'continue' : True
>> -})
>> +# if shelve is obs-based, we want rebase to be able
>> +# to create markers to already-obsoleted commits
>> +_repo = repo.unfiltered() if state.obsshelve else repo
>> +with ui.configoverride({('experimental', 'rebaseskipobsolete'):
>> +'off'}, 'unshelve'):
>> +rebase.rebase(ui, _repo, **{
>> +'continue' : True,
>> +})
>>   except Exception:
>>   util.rename(repo.join('rebasestate'),
>>   repo.join('unshelverebasestate'))
>> @@ -712,30 +723,54 @@ def _commitworkingcopychanges(ui, repo,
>>   with ui.configoverride({('ui', 'quiet'): True}):
>>   node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
>>   tmpwctx = repo[node]
>> +ui.debug("temporary working copy commit: %s:%s\n" %
>> + (tmpwctx.rev(), nodemod.short(node)))
>>   return tmpwctx, addedbefore
>>   
>> -def _unshelverestorecommit(ui, repo, basename):
>> +def _unshelverestorecommit(ui, repo, basename, obsshelve, shfile):
> "basename" and "shfile" looks duplicated. Maybe just keep one of them.
> (pass "file=shelvedfile(repo, basename, 'hg')", or just pass basename)
Will fix.
>>   """Recreate commit in the repository during the unshelve"""
>>   with ui.configoverride({('ui', 'quiet'): True}):
>> -shelvedfile(repo, basename, 'hg').applybundle()
>> -shelvectx = repo['tip']
>> +if obsshelve:
>> +md = shfile.readjson()
>> +shelvenode = nodemod.bin(md['node'])
>> +repo = repo.unfiltered()
>> +shelvectx = repo[shelvenode]
>> +else:
>> +shelvedfile(repo, basename, 'hg').applybundle()
>> +shelvectx = repo['tip'

[PATCH RFC] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480808065 28800
#  Sat Dec 03 15:34:25 2016 -0800
# Node ID c4312906ef7d06dc3be1575ee5cde574b915c9c4
# Parent  715232cf9182f904d5153862a1ec4905faaeee6e
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,51 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Kyes must be alphanumerics and start with a letter, values might not
+contain '\n' characters
+"""
+class InvalidKeyInFileException(Exception): pass
+class InvalidValueInFileException(Exception): pass
+class MissingRequiredKeyInFileException(Exception): pass
+
+# if KEYS is non-empty, read values are validated against it:
+# - if field name starts with an uppercase letter, this FIELDS
+#   is considered required and its absense is an Exception
+# - if field name starts with a lowercase letter, it is optional
+#   and serves mainly as a reference for code reader
+KEYS = []
+
+def __init__(self, repo, path):
+self.vfs = repo.vfs
+self.path = path
+
+def validate(self, d):
+for k in self.KEYS:
+if k[0].isupper() and k not in d:
+e = _("Missing a required key: '%s'" % k)
+raise self.MissingRequiredKeyInFileException(e)
+
+def read(self):
+lines = self.vfs.readlines(self.path)
+d = dict(line[:-1].split('=', 1) for line in lines)
+self.validate(d)
+return d
+
+def write(self, data):
+lines = []
+for k, v in data.items():
+if not k[0].isalpha():
+e = _("Keys must start with a letter in a key-value file")
+raise self.InvalidKeyInFileException(e)
+if not k.isalnum():
+e = _("Invalid key name in a simple key-value file")
+raise self.InvalidKeyInFileException(e)
+if '\n' in v:
+e = _("Invalid value in a simple key-value file")
+raise self.InvalidValueInFileException(e)
+lines.append("%s=%s\n" % (k, v))
+self.vfs.writelines(self.path, lines)
diff --git a/tests/test-other.t b/tests/test-other.t
new file mode 100644
--- /dev/null
+++ b/tests/test-other.t
@@ -0,0 +1,95 @@
+Test simple key-value files
+  $ cd $TESTTMP
+  $ hg init repo
+  $ cd $TESTTMP/repo
+
+Test simple key-value file creation
+  $ cat < keyvalwriter.py
+  > from mercurial import ui, hg
+  > from mercurial.scmutil import simplekeyvaluefile
+  > ui = ui.ui()
+  > repo = hg.repository(ui, '$TESTTMP/repo')
+  > d = {'key1': 'value1', 'Key2': 'value2'}
+  > kvf = simplekeyvaluefile(repo, 'kvfile').write(d)
+  > EOF
+  $ python keyvalwriter.py
+  $ cat .hg/kvfile | sort
+  Key2=value2
+  key1=value1
+
+Test simple key-value file reading with invalid keys or values
+  $ cat < keyvalwriter.py
+  > from mercurial import ui, hg
+  > from mercurial.scmutil import simplekeyvaluefile
+  > ui = ui.ui()
+  > repo = hg.repository(ui, '$TESTTMP/repo')
+  > d = {'0key1': 'value1', 'Key2': 'value2'}
+  > kvf = simplekeyvaluefile(repo, 'kvfile').write(d)
+  > EOF
+  $ python keyvalwriter.py 2>&1 | tail -1
+  mercurial.scmutil.InvalidKeyInFileException: Keys must start with a letter 
in a key-value file
+  $ cat < keyvalwriter.py
+  > from mercurial import ui, hg
+  > from mercurial.scmutil import simplekeyvaluefile
+  > ui = ui.ui()
+  > repo = hg.repository(ui, '$TESTTMP/r

[PATCH RFC v2] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480967781 28800
#  Mon Dec 05 11:56:21 2016 -0800
# Node ID eceaa3ab833141da00291a236a20ca0be0b4b3b0
# Parent  243ecbd4f5c9f452275d4435866359cf84dc03ff
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,55 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Kyes must be alphanumerics and start with a letter, values might not
+contain '\n' characters
+"""
+class InvalidKeyValueFile(Exception): pass
+class InvalidKeyInFileException(Exception): pass
+class InvalidValueInFileException(Exception): pass
+class MissingRequiredKeyInFileException(Exception): pass
+
+# if KEYS is non-empty, read values are validated against it:
+# - if field name starts with an uppercase letter, this FIELDS
+#   is considered required and its absense is an Exception
+# - if field name starts with a lowercase letter, it is optional
+#   and serves mainly as a reference for code reader
+KEYS = []
+
+def __init__(self, vfs, path):
+self.vfs = vfs
+self.path = path
+
+def validate(self, d):
+for k in self.KEYS:
+if k[0].isupper() and k not in d:
+e = _("Missing a required key: '%s'" % k)
+raise self.MissingRequiredKeyInFileException(e)
+
+def read(self):
+lines = self.vfs.readlines(self.path)
+try:
+d = dict(line[:-1].split('=', 1) for line in lines)
+except ValueError as e:
+raise self.InvalidKeyValueFile(str(e))
+self.validate(d)
+return d
+
+def write(self, data):
+lines = []
+for k, v in data.items():
+if not k[0].isalpha():
+e = _("Keys must start with a letter in a key-value file")
+raise self.InvalidKeyInFileException(e)
+if not k.isalnum():
+e = _("Invalid key name in a simple key-value file")
+raise self.InvalidKeyInFileException(e)
+if '\n' in v:
+e = _("Invalid value in a simple key-value file")
+raise self.InvalidValueInFileException(e)
+lines.append("%s=%s\n" % (k, v))
+self.vfs.writelines(self.path, lines)
diff --git a/tests/test-other.t b/tests/test-other.t
new file mode 100644
--- /dev/null
+++ b/tests/test-other.t
@@ -0,0 +1,102 @@
+Test simple key-value files
+  $ cd $TESTTMP
+  $ hg init repo
+  $ cd $TESTTMP/repo
+
+Test simple key-value file creation
+  $ cat < keyvalwriter.py
+  > from mercurial import ui, hg
+  > from mercurial.scmutil import simplekeyvaluefile
+  > ui = ui.ui()
+  > repo = hg.repository(ui, '$TESTTMP/repo')
+  > d = {'key1': 'value1', 'Key2': 'value2'}
+  > kvf = simplekeyvaluefile(repo.vfs, 'kvfile').write(d)
+  > EOF
+  $ python keyvalwriter.py
+  $ cat .hg/kvfile | sort
+  Key2=value2
+  key1=value1
+
+Test simple key-value file reading with invalid keys or values
+  $ cat < keyvalwriter.py
+  > from mercurial import ui, hg
+  > from mercurial.scmutil import simplekeyvaluefile
+  > ui = ui.ui()
+  > repo = hg.repository(ui, '$TESTTMP/repo')
+  > d = {'0key1': 'value1', 'Key2': 'value2'}
+  > kvf = simplekeyvaluefile(repo.vfs, 'kvfile').write(d)
+  > EOF
+  $ python keyvalwriter.py 2>&1 | tail -1
+  mercurial.scmutil.InvalidKeyInFileException: Keys must start with a letter 
in a key-value file
+  $ cat < keyvalwriter.py
+  > from mercurial imp

Re: [PATCH] chgserver: call "load" for new ui objects

[Kostia Balytskyi]

LGTM.

On 12/5/16 9:37 PM, Jun Wu wrote:
> # HG changeset patch
> # User Jun Wu 
> # Date 1480973795 0
> #  Mon Dec 05 21:36:35 2016 +
> # Node ID b1877dc4bd741a4855c3fda0d473e5d821f5c4e2
> # Parent  2463b16bbd3710b619e0e948651db9948341f990
> # Available At https://bitbucket.org/quark-zju/hg-draft
> #  hg pull https://bitbucket.org/quark-zju/hg-draft -r 
> b1877dc4bd74
> chgserver: call "load" for new ui objects
>
> After d83ca854fa21, we need to call "ui.load" explicitly to load config
> files.
>
> diff --git a/mercurial/chgserver.py b/mercurial/chgserver.py
> --- a/mercurial/chgserver.py
> +++ b/mercurial/chgserver.py
> @@ -256,5 +256,5 @@ def _loadnewui(srcui, args):
>   from . import dispatch  # avoid cycle
>   
> -newui = srcui.__class__()
> +newui = srcui.__class__.load()
>   for a in ['fin', 'fout', 'ferr', 'environ']:
>   setattr(newui, a, getattr(srcui, a))
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH RFC] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/5/16 9:54 PM, Jun Wu wrote:
> Excerpts from Kostia Balytskyi's message of 2016-12-03 15:37:55 -0800:
>> # HG changeset patch
>> # User Kostia Balytskyi 
>> # Date 1480808065 28800
>> #  Sat Dec 03 15:34:25 2016 -0800
>> # Node ID c4312906ef7d06dc3be1575ee5cde574b915c9c4
>> # Parent  715232cf9182f904d5153862a1ec4905faaeee6e
>> scmutil: add a simple key-value file helper
>>
>> The purpose of the added class is to serve purposes like save files of shelve
>> or state files of shelve, rebase and histedit. Keys of these files can be
>> alphanumeric and start with letters, while values must not contain newlines.
>> Keys which start with an uppercase letter are required, while other keys
>> are optional.
>>
>> In light of Mercurial's reluctancy to use Python's json module, this tries
>> to provide a reasonable alternative for a non-nested named data.
>> Comparing to current approach of storing state in plain text files, where
>> semantic meaning of lines of text is only determined by their oreder,
>> simple key-value file allows for reordering lines and thus helps handle
>> optional values.
>>
>> Initial use-case I see for this is obs-shelve's shelve files. Later we
>> can possibly migrate state files to this approach.
>>
>> The test is in a new file beause I did not figure out where to put it
>> within existing test suite. If you give me a better idea, I will gladly
>> follow it.
>>
>> diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
>> --- a/mercurial/scmutil.py
>> +++ b/mercurial/scmutil.py
>> @@ -1571,3 +1571,51 @@ class checkambigatclosing(closewrapbase)
>>   def close(self):
>>   self._origfh.close()
>>   self._checkambig()
>> +
>> +class simplekeyvaluefile(object):
>> +"""A simple file with key=value lines
>> +
>> +Kyes must be alphanumerics and start with a letter, values might not
> Keys.
Good catch.
>
>> +contain '\n' characters
>> +"""
>> +class InvalidKeyInFileException(Exception): pass
>> +class InvalidValueInFileException(Exception): pass
>> +class MissingRequiredKeyInFileException(Exception): pass
> This enforces the caller to do
>
>excpet simplekeyvaluefile.InvalidKeyInFileException
>
> I guess we put every exception classes inside error.py?
> (See below, I think the exceptions used in "write" should be just
>   "RuntimeError"s)
How strong is our habit of storing exceptions in error.py? I feel like
having exceptions namespaced in simplekeyvaluefile reduces the need
to have really verbose names. In short, I am for keeping exceptions in
the class, but I don't feel too strongly about it.
>
>> +
>> +# if KEYS is non-empty, read values are validated against it:
>> +# - if field name starts with an uppercase letter, this FIELDS
>> +#   is considered required and its absense is an Exception
>> +# - if field name starts with a lowercase letter, it is optional
>> +#   and serves mainly as a reference for code reader
> This may make the file look a bit strange when editing directly (mixed of
> upper case and lower cases). As the coding style is basically "lowercase
> everywhere". I'd suggest underscore or an explicit boolean flag instead.
Will do.
>
>> +KEYS = []
>> +
>> +def __init__(self, repo, path):
>> +self.vfs = repo.vfs
>> +self.path = path
>> +
>> +def validate(self, d):
>> +for k in self.KEYS:
>> +if k[0].isupper() and k not in d:
>> +e = _("Missing a required key: '%s'" % k)
> I think we use lower-case in exception message ("missing" vs "Missing").
Will fix.
>
>> +raise self.MissingRequiredKeyInFileException(e)
>> +
>> +def read(self):
>> +lines = self.vfs.readlines(self.path)
>> +d = dict(line[:-1].split('=', 1) for line in lines)
>> +self.validate(d)
>> +return d
>> +
>> +def write(self, data):
> data deserves a docstring about its type.
Good idea, will fix.
>
>> +lines = []
>> +for k, v in data.items():
>> +if not k[0].isalpha():
>> +e = _("Keys must start with a letter in a key-value file")
>> +raise self.InvalidKeyInFileException(e)
>> +if not k.isalnum():
>> +e = _("Invalid key name in a simple key-value file")
>&g

[PATCH RFC v3] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1480978778 28800
#  Mon Dec 05 14:59:38 2016 -0800
# Node ID bdc49209b0b926214e865f9a98ea987866f64d68
# Parent  243ecbd4f5c9f452275d4435866359cf84dc03ff
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,56 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Keys must be alphanumerics and start with a letter, values might not
+contain '\n' characters
+"""
+class InvalidKeyValueFile(Exception): pass
+class InvalidKeyInFileException(Exception): pass
+class InvalidValueInFileException(Exception): pass
+class MissingRequiredKeyInFileException(Exception): pass
+
+# if KEYS is non-empty, read values are validated against it:
+# each key is a tuple (keyname, required)
+KEYS = []
+
+def __init__(self, vfs, path):
+self.vfs = vfs
+self.path = path
+
+def validate(self, d):
+for key, req in self.KEYS:
+if req and key not in d:
+e = _("missing a required key: '%s'") % key
+raise self.MissingRequiredKeyInFileException(e)
+
+def read(self):
+lines = self.vfs.readlines(self.path)
+try:
+d = dict(line[:-1].split('=', 1) for line in lines if line)
+except ValueError as e:
+raise self.InvalidKeyValueFile(str(e))
+self.validate(d)
+return d
+
+def write(self, data):
+"""Write key=>value mapping to a file
+data is a dict. Keys should be alphanumerical and start with a letter.
+Values should not contain newline characters."""
+lines = []
+for k, v in data.items():
+if not k[0].isalpha():
+e = _("keys must start with a letter in a key-value file")
+raise self.InvalidKeyInFileException(e)
+if not k.isalnum():
+e = _("invalid key name in a simple key-value file")
+raise self.InvalidKeyInFileException(e)
+if '\n' in v:
+e = _("invalid value in a simple key-value file")
+raise self.InvalidValueInFileException(e)
+lines.append("%s=%s\n" % (k, v))
+with self.vfs(self.path, mode='wb', atomictemp=True) as fp:
+fp.write(''.join(lines))
diff --git a/tests/test-other.t b/tests/test-other.t
new file mode 100644
--- /dev/null
+++ b/tests/test-other.t
@@ -0,0 +1,102 @@
+Test simple key-value files
+  $ cd $TESTTMP
+  $ hg init repo
+  $ cd $TESTTMP/repo
+
+Test simple key-value file creation
+  $ cat < keyvalwriter.py
+  > from mercurial import ui, hg
+  > from mercurial.scmutil import simplekeyvaluefile
+  > ui = ui.ui()
+  > repo = hg.repository(ui, '$TESTTMP/repo')
+  > d = {'key1': 'value1', 'Key2': 'value2'}
+  > kvf = simplekeyvaluefile(repo.vfs, 'kvfile').write(d)
+  > EOF
+  $ python keyvalwriter.py
+  $ cat .hg/kvfile | sort
+  Key2=value2
+  key1=value1
+
+Test simple key-value file reading with invalid keys or values
+  $ cat < keyvalwriter.py
+  > from mercurial import ui, hg
+  > from mercurial.scmutil import simplekeyvaluefile
+  > ui = ui.ui()
+  > repo = hg.repository(ui, '$TESTTMP/repo')
+  > d = {'0key1': 'value1', 'Key2': 'value2'}
+  > kvf = simplekeyvaluefile(repo.vfs, 'kvfile').write(d)
+  > EOF
+  $ python keyvalwriter.py 2>&1 | tail -1
+  mercurial.scmutil.InvalidKeyInFileException: keys mu

Re: [PATCH RFC v3] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/6/16 1:04 PM, Yuya Nishihara wrote:
> On Mon, 5 Dec 2016 15:03:35 -0800, Kostia Balytskyi wrote:
>> # HG changeset patch
>> # User Kostia Balytskyi 
>> # Date 1480978778 28800
>> #  Mon Dec 05 14:59:38 2016 -0800
>> # Node ID bdc49209b0b926214e865f9a98ea987866f64d68
>> # Parent  243ecbd4f5c9f452275d4435866359cf84dc03ff
>> scmutil: add a simple key-value file helper
>>
>> The purpose of the added class is to serve purposes like save files of shelve
>> or state files of shelve, rebase and histedit. Keys of these files can be
>> alphanumeric and start with letters, while values must not contain newlines.
>> Keys which start with an uppercase letter are required, while other keys
>> are optional.
>> +class simplekeyvaluefile(object):
>> +"""A simple file with key=value lines
>> +
>> +Keys must be alphanumerics and start with a letter, values might not
>> +contain '\n' characters
>> +"""
>> +class InvalidKeyValueFile(Exception): pass
>> +class InvalidKeyInFileException(Exception): pass
>> +class InvalidValueInFileException(Exception): pass
>> +class MissingRequiredKeyInFileException(Exception): pass
>> +
>> +# if KEYS is non-empty, read values are validated against it:
>> +# each key is a tuple (keyname, required)
>> +KEYS = []
> If we want to define a required key back to older Mercurial versions (like
> mergestate), we'll need a static rule such as "uppercase is required" seen in
> your V1 patch.
I am not sure I understand this one. If you meant that 
simplekeyvaluefile should be able
to read state files created by older mercurail versions, I don't think 
it's possible. I don't think
simplekeyvaluefile itself should provide a backwards-compatible layer. 
Rather, when we
migrate a state file, we should first try to parse it into a 
simlpekeyvaluefile and if this fails,
fallback to a traditional file format. New fields should only be added 
to a simplekeyvaluefile
code path, while traditional parsing should replace them with some sort 
of empty/default values.

Can you clarify what you meant by "define a required key back to older 
Mercurial versions"?
>
>> +def __init__(self, vfs, path):
>> +self.vfs = vfs
>> +self.path = path
>> +
>> +def validate(self, d):
>> +for key, req in self.KEYS:
>> +if req and key not in d:
>> +e = _("missing a required key: '%s'") % key
>> +raise self.MissingRequiredKeyInFileException(e)
> _() is necessary only for user-facing errors.
>
> I don't think nested classes are useful unless we want to give a room for
> sub classes to override them.
>
>> +lines.append("%s=%s\n" % (k, v))
>> +with self.vfs(self.path, mode='wb', atomictemp=True) as fp:
>> +fp.write(''.join(lines))
> BTW, the syntax of this file seems somewhat similar to config.config.
I wanted something explicitly much simpler than config.config (also with 
possibility to further limit
what can be values if needed).
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH RFC v3] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/6/16 1:28 PM, Jun Wu wrote:
> Excerpts from Kostia Balytskyi's message of 2016-12-06 13:22:40 +:
>> On 12/6/16 1:04 PM, Yuya Nishihara wrote:
>>> On Mon, 5 Dec 2016 15:03:35 -0800, Kostia Balytskyi wrote:
>>>> # HG changeset patch
>>>> # User Kostia Balytskyi 
>>>> # Date 1480978778 28800
>>>> #  Mon Dec 05 14:59:38 2016 -0800
>>>> # Node ID bdc49209b0b926214e865f9a98ea987866f64d68
>>>> # Parent  243ecbd4f5c9f452275d4435866359cf84dc03ff
>>>> scmutil: add a simple key-value file helper
>>>>
>>>> The purpose of the added class is to serve purposes like save files of 
>>>> shelve
>>>> or state files of shelve, rebase and histedit. Keys of these files can be
>>>> alphanumeric and start with letters, while values must not contain 
>>>> newlines.
>>>> Keys which start with an uppercase letter are required, while other keys
>>>> are optional.
>>>> +class simplekeyvaluefile(object):
>>>> +"""A simple file with key=value lines
>>>> +
>>>> +Keys must be alphanumerics and start with a letter, values might not
>>>> +contain '\n' characters
>>>> +"""
>>>> +class InvalidKeyValueFile(Exception): pass
>>>> +class InvalidKeyInFileException(Exception): pass
>>>> +class InvalidValueInFileException(Exception): pass
>>>> +class MissingRequiredKeyInFileException(Exception): pass
>>>> +
>>>> +# if KEYS is non-empty, read values are validated against it:
>>>> +# each key is a tuple (keyname, required)
>>>> +KEYS = []
>>> If we want to define a required key back to older Mercurial versions (like
>>> mergestate), we'll need a static rule such as "uppercase is required" seen 
>>> in
>>> your V1 patch.
>> I am not sure I understand this one. If you meant that
>> simplekeyvaluefile should be able
>> to read state files created by older mercurail versions, I don't think
>> it's possible. I don't think
>> simplekeyvaluefile itself should provide a backwards-compatible layer.
>> Rather, when we
>> migrate a state file, we should first try to parse it into a
>> simlpekeyvaluefile and if this fails,
>> fallback to a traditional file format. New fields should only be added
>> to a simplekeyvaluefile
>> code path, while traditional parsing should replace them with some sort
>> of empty/default values.
>>
>> Can you clarify what you meant by "define a required key back to older
>> Mercurial versions"?
> It means, for example, in hg v5.0, we have keys "a" and "b" as "required".
> And we add another key "c" as required in v5.1. What will v5.1 do with the
> simplekeyvaluefile written by v5.0 ? (and what will we do if we want to
> remove required keys?)
Hm, I don't see how upper/lower-case instead of explicit boolean flag 
handles that...
But to answer your question:
1. When you add a new field and you mark it as required, you really have 
only two options:
   a. Breaking backwards compatibility. I guess this is wanted rarely.
   b. Adding a logic to replace the missing required field with some 
default value. My opinion is that such logic should exist outside of 
simplekeyvaluefile implementation, so from this class' perspective that 
field is still optional
2. When you remove a required field or make field which was required 
optional. Because we don't limit the optional
fields in any way, this should just work - unexpected field in the file 
is just read and stored in the dictionary.
>
>>>> +def __init__(self, vfs, path):
>>>> +self.vfs = vfs
>>>> +self.path = path
>>>> +
>>>> +def validate(self, d):
>>>> +for key, req in self.KEYS:
>>>> +if req and key not in d:
>>>> +e = _("missing a required key: '%s'") % key
>>>> +raise self.MissingRequiredKeyInFileException(e)
>>> _() is necessary only for user-facing errors.
>>>
>>> I don't think nested classes are useful unless we want to give a room for
>>> sub classes to override them.
>>>
>>>> +lines.append("%s=%s\n" % (k, v))
>>>> +with self.vfs(self.path, mode='wb', atomictemp=True) as fp:
>>>> +fp.write(''.join(lines))
>>> BTW, the syntax of this file seems somewhat similar to config.config.
>> I wanted something explicitly much simpler than config.config (also with
>> possibility to further limit
>> what can be values if needed).
>>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH RFC v3] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/6/16 1:48 PM, Jun Wu wrote:
> Excerpts from Kostia Balytskyi's message of 2016-12-06 13:39:31 +:
>> Hm, I don't see how upper/lower-case instead of explicit boolean flag
>> handles that...
> I didn't realize the issue either, before Yuya's reply.
>
>> But to answer your question:
>> 1. When you add a new field and you mark it as required, you really have
>> only two options:
>> a. Breaking backwards compatibility. I guess this is wanted rarely.
>> b. Adding a logic to replace the missing required field with some
>> default value. My opinion is that such logic should exist outside of
>> simplekeyvaluefile implementation, so from this class' perspective that
>> field is still optional
>> 2. When you remove a required field or make field which was required
>> optional. Because we don't limit the optional
>> fields in any way, this should just work - unexpected field in the file
>> is just read and stored in the dictionary.
> It seems that the user of simplekeyvaluefile have to deal with "required" /
> "optional" etc.
>
> How about just removing the "required" check in simplekeyvaluefile?
> Leave them to the user to handle. That seems cleaner to me.
This is what happens if you don't inherit the class or don't override 
the empty KEYS list in the inherited class.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH RFC v5] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1481032921 28800
#  Tue Dec 06 06:02:01 2016 -0800
# Node ID 79336a94b7709f8680fd6975c39bac41759ec6f4
# Parent  243ecbd4f5c9f452275d4435866359cf84dc03ff
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/error.py b/mercurial/error.py
--- a/mercurial/error.py
+++ b/mercurial/error.py
@@ -243,3 +243,19 @@ class UnsupportedBundleSpecification(Exc
 
 class CorruptedState(Exception):
 """error raised when a command is not able to read its state from file"""
+
+class InvalidKeyValueFile(Exception):
+"""error raised when the file can't be parsed as simple key-value file"""
+
+class InvalidKeyInFileException(Exception):
+"""error raised when invalid key is attempted to be written
+
+This is used in simple key-value file implementation"""
+
+class InvalidValueInFileException(Exception):
+"""error raisesd when invalid value is attempted to be written
+
+This is used in simple key-value file implementation"""
+
+class MissingRequiredKeyInFileException(Exception):
+"""error raised when simple key-value file misses a required key"""
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,123 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Keys must be alphanumerics and start with a letter, values might not
+contain '\n' characters
+
+>>> contents = {}
+>>> class fileobj(object):
+... def __init__(self, name):
+... self.name = name
+... def __enter__(self):
+... return self
+... def __exit__(self, *args, **kwargs):
+... pass
+... def write(self, text):
+... contents[self.name] = text
+... def read(self):
+... return contents[self.name]
+>>> class mockvfs(object):
+... def read(self, path):
+... return fileobj(path).read()
+... def readlines(self, path):
+... return fileobj(path).read().split('\\n')
+... def __call__(self, path, mode, atomictemp):
+... return fileobj(path)
+>>> vfs = mockvfs()
+
+Basic testing of whether simple key-value file works:
+>>> d = {'key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+>>> print sorted(vfs.read('kvfile').split('\\n'))
+['', 'Key2=value2', 'key1=value1']
+
+Testing of whether invalid keys are detected:
+>>> d = {'0key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidKeyInFileException: keys must start with a letter ...
+>>> d = {'key1@': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidKeyInFileException: invalid key name in a simple key-value file
+
+Testing of whether invalid values are detected:
+>>> d = {'key1': 'value1', 'Key2': 'value2\\n'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidValueInFileException: invalid value in a simple key-value file
+
+Test cases when necessary keys are present
+>>> d = {'

[PATCH RFC v4] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1481032446 28800
#  Tue Dec 06 05:54:06 2016 -0800
# Node ID 008bf4590cd2bed2329ad5eb8f2ad3e2bd121f2d
# Parent  243ecbd4f5c9f452275d4435866359cf84dc03ff
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/error.py b/mercurial/error.py
--- a/mercurial/error.py
+++ b/mercurial/error.py
@@ -243,3 +243,19 @@ class UnsupportedBundleSpecification(Exc
 
 class CorruptedState(Exception):
 """error raised when a command is not able to read its state from file"""
+
+class InvalidKeyValueFile(Exception):
+"""error raised when the file can't be parsed as simple key-value file"""
+
+class InvalidKeyInFileException(Exception):
+"""error raised when invalid key is attempted to be written
+
+This is used in simple key-value file implementation"""
+
+class InvalidValueInFileException(Exception):
+"""error raisesd when invalid value is attempted to be written
+
+This is used in simple key-value file implementation"""
+
+class MissingRequiredKeyInFileException(Exception):
+"""error raised when simple key-value file misses a required key"""
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,123 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Keys must be alphanumerics and start with a letter, values might not
+contain '\n' characters
+
+>>> contents = {}
+>>> class fileobj(object):
+... def __init__(self, name):
+... self.name = name
+... def __enter__(self):
+... return self
+... def __exit__(self, *args, **kwargs):
+... pass
+... def write(self, text):
+... contents[self.name] = text
+... def read(self):
+... return contents[self.name]
+>>> class mockvfs(object):
+... def read(self, path):
+... return fileobj(path).read()
+... def readlines(self, path):
+... return fileobj(path).read().split('\\n')
+... def __call__(self, path, mode, atomictemp):
+... return fileobj(path)
+>>> vfs = mockvfs()
+
+Basic testing of whether simple key-value file works:
+>>> d = {'key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+>>> print sorted(vfs.read('kvfile').split('\\n'))
+['', 'Key2=value2', 'key1=value1']
+
+Testing of whether invalid keys are detected:
+>>> d = {'0key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidKeyInFileException: keys must start with a letter ...
+>>> d = {'key1@': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidKeyInFileException: invalid key name in a simple key-value file
+
+Testing of whether invalid values are detected:
+>>> d = {'key1': 'value1', 'Key2': 'value2\\n'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidValueInFileException: invalid value in a simple key-value file
+
+Test cases when necessary keys are present
+>>&

Re: [PATCH RFC v4] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

Please disregard v4, see v5.

On 12/6/16 2:01 PM, Kostia Balytskyi wrote:
> # HG changeset patch
> # User Kostia Balytskyi 
> # Date 1481032446 28800
> #  Tue Dec 06 05:54:06 2016 -0800
> # Node ID 008bf4590cd2bed2329ad5eb8f2ad3e2bd121f2d
> # Parent  243ecbd4f5c9f452275d4435866359cf84dc03ff
> scmutil: add a simple key-value file helper
>
> The purpose of the added class is to serve purposes like save files of shelve
> or state files of shelve, rebase and histedit. Keys of these files can be
> alphanumeric and start with letters, while values must not contain newlines.
> Keys which start with an uppercase letter are required, while other keys
> are optional.
>
> In light of Mercurial's reluctancy to use Python's json module, this tries
> to provide a reasonable alternative for a non-nested named data.
> Comparing to current approach of storing state in plain text files, where
> semantic meaning of lines of text is only determined by their oreder,
> simple key-value file allows for reordering lines and thus helps handle
> optional values.
>
> Initial use-case I see for this is obs-shelve's shelve files. Later we
> can possibly migrate state files to this approach.
>
> The test is in a new file beause I did not figure out where to put it
> within existing test suite. If you give me a better idea, I will gladly
> follow it.
>
> diff --git a/mercurial/error.py b/mercurial/error.py
> --- a/mercurial/error.py
> +++ b/mercurial/error.py
> @@ -243,3 +243,19 @@ class UnsupportedBundleSpecification(Exc
>   
>   class CorruptedState(Exception):
>   """error raised when a command is not able to read its state from 
> file"""
> +
> +class InvalidKeyValueFile(Exception):
> +"""error raised when the file can't be parsed as simple key-value file"""
> +
> +class InvalidKeyInFileException(Exception):
> +"""error raised when invalid key is attempted to be written
> +
> +This is used in simple key-value file implementation"""
> +
> +class InvalidValueInFileException(Exception):
> +"""error raisesd when invalid value is attempted to be written
> +
> +This is used in simple key-value file implementation"""
> +
> +class MissingRequiredKeyInFileException(Exception):
> +"""error raised when simple key-value file misses a required key"""
> diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
> --- a/mercurial/scmutil.py
> +++ b/mercurial/scmutil.py
> @@ -1571,3 +1571,123 @@ class checkambigatclosing(closewrapbase)
>   def close(self):
>   self._origfh.close()
>   self._checkambig()
> +
> +class simplekeyvaluefile(object):
> +"""A simple file with key=value lines
> +
> +Keys must be alphanumerics and start with a letter, values might not
> +contain '\n' characters
> +
> +>>> contents = {}
> +>>> class fileobj(object):
> +... def __init__(self, name):
> +... self.name = name
> +... def __enter__(self):
> +... return self
> +... def __exit__(self, *args, **kwargs):
> +... pass
> +... def write(self, text):
> +... contents[self.name] = text
> +... def read(self):
> +... return contents[self.name]
> +>>> class mockvfs(object):
> +... def read(self, path):
> +... return fileobj(path).read()
> +... def readlines(self, path):
> +... return fileobj(path).read().split('\\n')
> +... def __call__(self, path, mode, atomictemp):
> +... return fileobj(path)
> +>>> vfs = mockvfs()
> +
> +Basic testing of whether simple key-value file works:
> +>>> d = {'key1': 'value1', 'Key2': 'value2'}
> +>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
> +>>> print sorted(vfs.read('kvfile').split('\\n'))
> +['', 'Key2=value2', 'key1=value1']
> +
> +Testing of whether invalid keys are detected:
> +>>> d = {'0key1': 'value1', 'Key2': 'value2'}
> +>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
> +Traceback (most recent call last):
> +...
> +InvalidKeyInFileException: keys must start with a letter ...
> +>>> d = {'key1@': 'value1', 'Key2': 'value2'}
> +>>> simplekeyvaluefile(vfs, 'kvf

Re: [PATCH RFC v3] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/6/16 2:08 PM, Jun Wu wrote:
> Excerpts from Kostia Balytskyi's message of 2016-12-06 13:54:02 +:
>> This is what happens if you don't inherit the class or don't override
>> the empty KEYS list in the inherited class.
> If KEYS is used to verify existence, how about just making it a list of
> strings, instead of tuples? The "required" boolean flag seems to be
> unnecessary.
That is true and I've thought about it: just store the list of required 
keys in KEYS.
The reason I want optional keys to be in this list is verbosity. I 
someone else wrote
the subclass for some key-value file, I as a maintainer have a single 
place to see all
the fields "desired" (required and optional) by current version of code. 
So the purpose
of optional keys there is documentation and readability.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH RFC v6] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1481034715 28800
#  Tue Dec 06 06:31:55 2016 -0800
# Node ID 99bf22661062bb7d28ece67d94ebf421e3440731
# Parent  243ecbd4f5c9f452275d4435866359cf84dc03ff
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/error.py b/mercurial/error.py
--- a/mercurial/error.py
+++ b/mercurial/error.py
@@ -243,3 +243,19 @@ class UnsupportedBundleSpecification(Exc
 
 class CorruptedState(Exception):
 """error raised when a command is not able to read its state from file"""
+
+class InvalidKeyValueFile(Exception):
+"""error raised when the file can't be parsed as simple key-value file"""
+
+class InvalidKeyInFileException(Exception):
+"""error raised when invalid key is attempted to be written
+
+This is used in simple key-value file implementation"""
+
+class InvalidValueInFileException(Exception):
+"""error raisesd when invalid value is attempted to be written
+
+This is used in simple key-value file implementation"""
+
+class MissingRequiredKeyInFileException(Exception):
+"""error raised when simple key-value file misses a required key"""
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,123 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Keys must be alphanumerics and start with a letter, values must not
+contain '\n' characters
+
+>>> contents = {}
+>>> class fileobj(object):
+... def __init__(self, name):
+... self.name = name
+... def __enter__(self):
+... return self
+... def __exit__(self, *args, **kwargs):
+... pass
+... def write(self, text):
+... contents[self.name] = text
+... def read(self):
+... return contents[self.name]
+>>> class mockvfs(object):
+... def read(self, path):
+... return fileobj(path).read()
+... def readlines(self, path):
+... return fileobj(path).read().split('\\n')
+... def __call__(self, path, mode, atomictemp):
+... return fileobj(path)
+>>> vfs = mockvfs()
+
+Basic testing of whether simple key-value file works:
+>>> d = {'key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+>>> print sorted(vfs.read('kvfile').split('\\n'))
+['', 'Key2=value2', 'key1=value1']
+
+Testing of whether invalid keys are detected:
+>>> d = {'0key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidKeyInFileException: keys must start with a letter ...
+>>> d = {'key1@': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidKeyInFileException: invalid key name in a simple key-value file
+
+Testing of whether invalid values are detected:
+>>> d = {'key1': 'value1', 'Key2': 'value2\\n'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+InvalidValueInFileException: invalid value in a simple key-value file
+
+Test cases when necessary keys are present
+>>> d = {'

Re: [PATCH 2 of 2] localrepo: use ProgrammingError

[Kostia Balytskyi]

Both patches look good to me.

On 12/6/16 5:13 PM, Jun Wu wrote:
> # HG changeset patch
> # User Jun Wu 
> # Date 1481043999 0
> #  Tue Dec 06 17:06:39 2016 +
> # Node ID 5197933cb8b90a976be3d298b5f51b154fd8e733
> # Parent  67bcd43e64ca03f486a817fccf38e83020b06793
> # Available At https://bitbucket.org/quark-zju/hg-draft
> #  hg pull https://bitbucket.org/quark-zju/hg-draft -r 
> 5197933cb8b9
> localrepo: use ProgrammingError
>
> This is an example usage of ProgrammingError. Let's start migrating
> RuntimeError to ProgrammingError.
>
> The code only runs when devel.all-warnings or devel.check-locks is set, so
> it does not affect the end-user experience.
>
> diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
> --- a/mercurial/localrepo.py
> +++ b/mercurial/localrepo.py
> @@ -1017,6 +1017,5 @@ class localrepository(object):
>   or self.ui.configbool('devel', 'check-locks')):
>   if self._currentlock(self._lockref) is None:
> -raise RuntimeError('programming error: transaction requires '
> -   'locking')
> +raise error.ProgrammingError('transaction requires locking')
>   tr = self.currenttransaction()
>   if tr is not None:
> diff --git a/tests/test-devel-warnings.t b/tests/test-devel-warnings.t
> --- a/tests/test-devel-warnings.t
> +++ b/tests/test-devel-warnings.t
> @@ -175,5 +175,5 @@ Test programming error failure:
> ** Extensions loaded: * (glob)
> Traceback (most recent call last):
> -  RuntimeError: programming error: transaction requires locking
> +  mercurial.error.ProgrammingError: transaction requires locking
>   
> $ cd ..
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH RFC v3] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/6/16 2:45 PM, Jun Wu wrote:
> Excerpts from Kostia Balytskyi's message of 2016-12-06 14:30:18 +:
>> On 12/6/16 2:08 PM, Jun Wu wrote:
>>> Excerpts from Kostia Balytskyi's message of 2016-12-06 13:54:02 +:
 This is what happens if you don't inherit the class or don't override
 the empty KEYS list in the inherited class.
>>> If KEYS is used to verify existence, how about just making it a list of
>>> strings, instead of tuples? The "required" boolean flag seems to be
>>> unnecessary.
>> That is true and I've thought about it: just store the list of required
>> keys in KEYS.
>> The reason I want optional keys to be in this list is verbosity. I
>> someone else wrote
>> the subclass for some key-value file, I as a maintainer have a single
>> place to see all
>> the fields "desired" (required and optional) by current version of code.
>> So the purpose
>> of optional keys there is documentation and readability.
> Then how about explicit "REQUIREDKEYS" and "OPTIONALKEYS"?
>
> It seems sub-optimal to me that the "verify" method has to go through keys
> unnecessarily.
Well, I don't think performance is an issue here, these classes will 
rarely store more
than 20 fields if written by people and for people. If however someone 
uses it store
some large number of values - like some actual data - I don't think that 
KEYS is going
to be of any value in that case. I don't have a strong opinion about 
this, can change
if it matters.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH RFC v7] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1481060343 28800
#  Tue Dec 06 13:39:03 2016 -0800
# Node ID 211cfe5a24d18218720c38a601dfb2b88373b59a
# Parent  831d29deed083618bddc2fade7d2a6fe297c896d
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/error.py b/mercurial/error.py
--- a/mercurial/error.py
+++ b/mercurial/error.py
@@ -246,3 +246,6 @@ class UnsupportedBundleSpecification(Exc
 
 class CorruptedState(Exception):
 """error raised when a command is not able to read its state from file"""
+
+class MissingRequiredKeyInFileException(Exception):
+"""error raised when simple key-value file misses a required key"""
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,123 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Keys must be alphanumerics and start with a letter, values must not
+contain '\n' characters
+
+>>> contents = {}
+>>> class fileobj(object):
+... def __init__(self, name):
+... self.name = name
+... def __enter__(self):
+... return self
+... def __exit__(self, *args, **kwargs):
+... pass
+... def write(self, text):
+... contents[self.name] = text
+... def read(self):
+... return contents[self.name]
+>>> class mockvfs(object):
+... def read(self, path):
+... return fileobj(path).read()
+... def readlines(self, path):
+... return fileobj(path).read().split('\\n')
+... def __call__(self, path, mode, atomictemp):
+... return fileobj(path)
+>>> vfs = mockvfs()
+
+Basic testing of whether simple key-value file works:
+>>> d = {'key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+>>> print sorted(vfs.read('kvfile').split('\\n'))
+['', 'Key2=value2', 'key1=value1']
+
+Testing of whether invalid keys are detected:
+>>> d = {'0key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+ProgrammingError: keys must start with a letter ...
+>>> d = {'key1@': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+ProgrammingError: invalid key name in a simple key-value file
+
+Testing of whether invalid values are detected:
+>>> d = {'key1': 'value1', 'Key2': 'value2\\n'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+ProgrammingError: invalid value in a simple key-value file
+
+Test cases when necessary keys are present
+>>> d = {'key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'allkeyshere').write(d)
+>>> class kvf(simplekeyvaluefile):
+... KEYS = [('key3', False), ('Key2', True)]
+>>> print sorted(kvf(vfs, 'allkeyshere').read().items())
+[('Key2', 'value'), ('key1', 'value')]
+
+Test cases when necessary keys are absent
+>>> d = {'key1': 'value1', 'Key3': 'value2'}
+>>&g

Re: [PATCH RFC v7] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/6/16 9:53 PM, Jun Wu wrote:
> Excerpts from Kostia Balytskyi's message of 2016-12-06 13:41:07 -0800:
>>   class CorruptedState(Exception):
>>   """error raised when a command is not able to read its state from 
>> file"""
>> +
>> +class MissingRequiredKeyInFileException(Exception):
>> +"""error raised when simple key-value file misses a required key"""
> I still think "CorruptedState" is better. It fits the use-case, is short and
> concise. 6-word exception sounds strange to me, partially because everything
> else in error.py is at most 4 words. If we have to use a new exception,
> maybe just "MissingRequiredKey", or just "KeyError".
I agree that the name I chose is too long. But CorruptedState is a 
significantly different thing than missing key and gut feeling tells me 
it should sometimes be treated differently. I can't think of an example 
right now though.
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 6 v2] [convert] Use convert_revision for P4 imports

[Kostia Balytskyi]

On 12/7/16 9:30 PM, David Soria Parra wrote:
> # HG changeset patch
> # User David Soria Parra 
> # Date 1480895549 28800
> #  Sun Dec 04 15:52:29 2016 -0800
> # Node ID ade3103f5cf939007ca73af5e9e5853643dc8264
> # Parent  48bf03a8f718f870256f79cbdf10550ca92c7f25
> [convert] Use convert_revision for P4 imports
This does not follow our naming scheme "keyword: some short description"
>
> We are using convert_revisions in other importers. In order to unify this
> we are also using convert_revision for Perforce in addition to the original
> 'p4'.
>
> diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
> --- a/hgext/convert/p4.py
> +++ b/hgext/convert/p4.py
> @@ -151,7 +151,7 @@
>   c = common.commit(author=self.recode(d["user"]),
> date=util.datestr(date, '%Y-%m-%d %H:%M:%S 
> %1%2'),
> parents=parents, desc=desc, branch=None,
> -  extra={"p4": change})
> +  extra={"p4": change, "convert_revision": 
> change})
>   
>   files = []
>   copies = {}
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 4 of 6 v3] convert: Parse perforce changes when needed, not in the constructor

[Kostia Balytskyi]

On 12/7/16 9:51 PM, David Soria Parra wrote:
> # HG changeset patch
> # User David Soria Parra 
> # Date 1481139471 28800
> #  Wed Dec 07 11:37:51 2016 -0800
> # Node ID e3c34b2fafd7ce1a78d23e35888a33ef10629fbf
> # Parent  897be730ab884aa8f7373a34eeb6a288cc1dabc7
> convert: Parse perforce changes when needed, not in the constructor
>
> The perforce source is parsing all changes in the constructor and not 
> deferring
> it to when needed. This leads to the source parsing changes before the revmap
> is available. Let's move the parsing to when we need it the first time.
>
> diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
> --- a/hgext/convert/p4.py
> +++ b/hgext/convert/p4.py
> @@ -79,7 +79,6 @@
>   if revs and len(revs) > 1:
>   raise error.Abort(_("p4 source does not support specifying "
>  "multiple revisions"))
> -self._parse(ui, path)
>   
>   def setrevmap(self, revmap):
>   self.revmap = revmap
> @@ -224,6 +223,8 @@
>   self.heads = [lastid]
>   
>   def getheads(self):
> +if len(self.p4changes) == 0:
> +self._parse(self.ui, self.path)
I think this could be implemented as a lazy property for less duplication.
>   return self.heads
>   
>   def getfile(self, name, rev):
> @@ -299,7 +300,11 @@
>   return self.changeset[rev]
>   
>   def gettags(self):
> +if len(self.p4changes) == 0:
> +self._parse(self.ui, self.path)
>   return self.tags
>   
>   def getchangedfiles(self, rev, i):
> +if len(self.p4changes) == 0:
> +self._parse(self.ui, self.path)
>   return sorted([x[0] for x in self.files[rev]])
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 6 of 6 v3] convert: Create commits from revmap list if needed

[Kostia Balytskyi]

On 12/7/16 9:51 PM, David Soria Parra wrote:
> # HG changeset patch
> # User David Soria Parra 
> # Date 1481143876 28800
> #  Wed Dec 07 12:51:16 2016 -0800
> # Node ID 109de539306c5bc49d38d6f1c802c4a8d092b485
> # Parent  be68e4436851b7e20f3b8cb34666418f5840dd66
> convert: Create commits from revmap list if needed
This commit demonstrates the need to add some sort of comment to 
self.revmap = {} in constructor. Otherwise, it's very confusing IMO.
>
> diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
> --- a/hgext/convert/p4.py
> +++ b/hgext/convert/p4.py
> @@ -314,6 +314,12 @@
>   return marshal.load(stdout)
>   
>   def getcommit(self, rev):
> +if rev not in self.changeset and rev not in self.revmap:
> +raise error.Abort(
> +_("cannot find %s in the revmap or parsed changesets") % rev)
> +if rev not in self.changeset:
> +d = self._fetch_revision(rev)
> +return self._construct_commit(d, parents=None)
>   return self.changeset[rev]
>   
>   def gettags(self):
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 6 of 6 v3] convert: Create commits from revmap list if needed

[Kostia Balytskyi]

On 12/8/16 12:18 AM, David Soria Parra wrote:
> On Wed, Dec 07, 2016 at 10:23:08PM +0000, Kostia Balytskyi wrote:
>> On 12/7/16 9:51 PM, David Soria Parra wrote:
>>> # HG changeset patch
>>> # User David Soria Parra 
>>> # Date 1481143876 28800
>>> #  Wed Dec 07 12:51:16 2016 -0800
>>> # Node ID 109de539306c5bc49d38d6f1c802c4a8d092b485
>>> # Parent  be68e4436851b7e20f3b8cb34666418f5840dd66
>>> convert: Create commits from revmap list if needed
>> This commit demonstrates the need to add some sort of comment to
>> self.revmap = {} in constructor. Otherwise, it's very confusing IMO.
> I honestly don't undersetand what is so confusing about it. It's a
> revmap and it seems straight forward to me that it's a dictionary
> that maps commits, at least thats what all revmaps in convert ever do.
The confusing (at least to me) part is what revmap maps commits to. 
There's no example
of setrevmap usage, so it's unclear how it is used. Feel free to ignore 
this if you think this is unreasonable though.
>
>>> diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
>>> --- a/hgext/convert/p4.py
>>> +++ b/hgext/convert/p4.py
>>> @@ -314,6 +314,12 @@
>>>return marshal.load(stdout)
>>>
>>>def getcommit(self, rev):
>>> +if rev not in self.changeset and rev not in self.revmap:
>>> +raise error.Abort(
>>> +_("cannot find %s in the revmap or parsed changesets") % 
>>> rev)
>>> +if rev not in self.changeset:
>>> +d = self._fetch_revision(rev)
>>> +return self._construct_commit(d, parents=None)
>>>return self.changeset[rev]
>>>
>>>def gettags(self):
>>> ___
>>> Mercurial-devel mailing list
>>> Mercurial-devel@mercurial-scm.org
>>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>> ___
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 6 of 8 v4] convert: do not provide head revisions if we have no changests to import

[Kostia Balytskyi]

On 12/14/16 9:46 AM, David Soria Parra wrote:
> # HG changeset patch
> # User David Soria Parra 
> # Date 1481694598 28800
> #  Tue Dec 13 21:49:58 2016 -0800
> # Node ID c6c26e238617600d0523b44a9043d17f963f9eda
> # Parent  4b9d267d2a05970d9320b83f8bf29ec99f8ab40f
> convert: do not provide head revisions if we have no changests to import
>
> Don't set a head revision in cases where we have a revmap but no
> changesets to import, as convertcmd.convert() treats them as heads of
> to-imported revisions.
>
> diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
> --- a/hgext/convert/p4.py
> +++ b/hgext/convert/p4.py
> @@ -228,7 +228,7 @@
>   self.copies[change] = copies
>   lastid = change
>   
> -if lastid:
> +if lastid and len(self.changeset) > 0:
>   self.heads = [lastid]
So previously self.heads was initialized as a dict but later changed to 
be a list? Wow.
>   
>   def getheads(self):
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 8 of 8 v4] convert: return commit objects for revisions in the revmap

[Kostia Balytskyi]

On 12/14/16 9:46 AM, David Soria Parra wrote:
> # HG changeset patch
> # User David Soria Parra 
> # Date 1481704712 28800
> #  Wed Dec 14 00:38:32 2016 -0800
> # Node ID eb4a6df3de20deb31e681aa7eba408dc352440c3
> # Parent  26ec2474ab06fdaf1aca034c54309245ded267b0
> convert: return commit objects for revisions in the revmap
>
> Source revision data that exists in the revmap are ignored when pulling
> data from Perforce as we consider them already imported. In case where
> the `convertcmd.convert` algorithm requests a commit object for such
> a revision we are creating it.  This is usually the case for parent of
> the first imported revision.
>
> diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
> --- a/hgext/convert/p4.py
> +++ b/hgext/convert/p4.py
> @@ -321,6 +321,12 @@
>   return marshal.load(stdout)
>   
>   def getcommit(self, rev):
> +if rev not in self.changeset and rev not in self.revmap:
> +raise error.Abort(
> +_("cannot find %s in the revmap or parsed changesets") % rev)
> +if rev not in self.changeset:
I think this would be more clear if you used 'if rev in self.revmap'. 
But I think whoever queues it can change it in flight.
> +d = self._fetch_revision(rev)
> +return self._construct_commit(d, parents=None)
>   return self.changeset[rev]
>   
>   def gettags(self):
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 8 of 8 v4] convert: return commit objects for revisions in the revmap

[Kostia Balytskyi]

Other than the nit below, this series LGTM.

On 12/14/16 4:53 PM, Kostia Balytskyi wrote:
>
> On 12/14/16 9:46 AM, David Soria Parra wrote:
>> # HG changeset patch
>> # User David Soria Parra 
>> # Date 1481704712 28800
>> #  Wed Dec 14 00:38:32 2016 -0800
>> # Node ID eb4a6df3de20deb31e681aa7eba408dc352440c3
>> # Parent  26ec2474ab06fdaf1aca034c54309245ded267b0
>> convert: return commit objects for revisions in the revmap
>>
>> Source revision data that exists in the revmap are ignored when pulling
>> data from Perforce as we consider them already imported. In case where
>> the `convertcmd.convert` algorithm requests a commit object for such
>> a revision we are creating it.  This is usually the case for parent of
>> the first imported revision.
>>
>> diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
>> --- a/hgext/convert/p4.py
>> +++ b/hgext/convert/p4.py
>> @@ -321,6 +321,12 @@
>>return marshal.load(stdout)
>>
>>def getcommit(self, rev):
>> +if rev not in self.changeset and rev not in self.revmap:
>> +raise error.Abort(
>> +_("cannot find %s in the revmap or parsed changesets") % 
>> rev)
>> +if rev not in self.changeset:
> I think this would be more clear if you used 'if rev in self.revmap'.
> But I think whoever queues it can change it in flight.
>> +d = self._fetch_revision(rev)
>> +return self._construct_commit(d, parents=None)
>>return self.changeset[rev]
>>
>>def gettags(self):
>> ___
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH RFC v8] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1481734993 28800
#  Wed Dec 14 09:03:13 2016 -0800
# Node ID 0973a68a3dcdfb7154e8ab27b7bb7028e97dd516
# Parent  95b076f5ddee5ab7bd208b8c2f42121ce9907129
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/error.py b/mercurial/error.py
--- a/mercurial/error.py
+++ b/mercurial/error.py
@@ -246,3 +246,6 @@ class UnsupportedBundleSpecification(Exc
 
 class CorruptedState(Exception):
 """error raised when a command is not able to read its state from file"""
+
+class MissingRequiredKey(Exception):
+"""error raised when simple key-value file misses a required key"""
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,123 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Keys must be alphanumerics and start with a letter, values must not
+contain '\n' characters
+
+>>> contents = {}
+>>> class fileobj(object):
+... def __init__(self, name):
+... self.name = name
+... def __enter__(self):
+... return self
+... def __exit__(self, *args, **kwargs):
+... pass
+... def write(self, text):
+... contents[self.name] = text
+... def read(self):
+... return contents[self.name]
+>>> class mockvfs(object):
+... def read(self, path):
+... return fileobj(path).read()
+... def readlines(self, path):
+... return fileobj(path).read().split('\\n')
+... def __call__(self, path, mode, atomictemp):
+... return fileobj(path)
+>>> vfs = mockvfs()
+
+Basic testing of whether simple key-value file works:
+>>> d = {'key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+>>> print sorted(vfs.read('kvfile').split('\\n'))
+['', 'Key2=value2', 'key1=value1']
+
+Testing of whether invalid keys are detected:
+>>> d = {'0key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+ProgrammingError: keys must start with a letter ...
+>>> d = {'key1@': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+ProgrammingError: invalid key name in a simple key-value file
+
+Testing of whether invalid values are detected:
+>>> d = {'key1': 'value1', 'Key2': 'value2\\n'}
+>>> simplekeyvaluefile(vfs, 'kvfile').write(d)
+Traceback (most recent call last):
+...
+ProgrammingError: invalid value in a simple key-value file
+
+Test cases when necessary keys are present
+>>> d = {'key1': 'value1', 'Key2': 'value2'}
+>>> simplekeyvaluefile(vfs, 'allkeyshere').write(d)
+>>> class kvf(simplekeyvaluefile):
+... KEYS = [('key3', False), ('Key2', True)]
+>>> print sorted(kvf(vfs, 'allkeyshere').read().items())
+[('Key2', 'value'), ('key1', 'value')]
+
+Test cases when necessary keys are absent
+>>> d = {'key1': 'value1', 'Key3': 'value2'}
+>>> simplekeyvaluef

Re: [PATCH 8 of 8 v4] convert: return commit objects for revisions in the revmap

[Kostia Balytskyi]

On 12/14/16 5:55 PM, David Soria Parra wrote:
> On Wed, Dec 14, 2016 at 04:53:24PM +0000, Kostia Balytskyi wrote:
>>
>> On 12/14/16 9:46 AM, David Soria Parra wrote:
>>> # HG changeset patch
>>> # User David Soria Parra 
>>> # Date 1481704712 28800
>>> #  Wed Dec 14 00:38:32 2016 -0800
>>> # Node ID eb4a6df3de20deb31e681aa7eba408dc352440c3
>>> # Parent  26ec2474ab06fdaf1aca034c54309245ded267b0
>>> convert: return commit objects for revisions in the revmap
>>>
>>> Source revision data that exists in the revmap are ignored when pulling
>>> data from Perforce as we consider them already imported. In case where
>>> the `convertcmd.convert` algorithm requests a commit object for such
>>> a revision we are creating it.  This is usually the case for parent of
>>> the first imported revision.
>>>
>>> diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
>>> --- a/hgext/convert/p4.py
>>> +++ b/hgext/convert/p4.py
>>> @@ -321,6 +321,12 @@
>>>return marshal.load(stdout)
>>>
>>>def getcommit(self, rev):
>>> +if rev not in self.changeset and rev not in self.revmap:
>>> +raise error.Abort(
>>> +_("cannot find %s in the revmap or parsed changesets") % 
>>> rev)
>>> +if rev not in self.changeset:
>> I think this would be more clear if you used 'if rev in self.revmap'.
>> But I think whoever queues it can change it in flight.
> that would not be sufficient. We want to fallback to revmap but use
> changeset if we have it.
>   
> So it would be:
>   if rev in self.revmap and rev not in self.changeset:
> or:
>   if rev in self.changeset:
>   return self.changeset[rev]
>   elif rev in self.revmap:
>   ...
>   else:
>   ...
>
> I can sent a v5 later.
Right, I missed the case when it's in both sets.
>
>>> +d = self._fetch_revision(rev)
>>> +return self._construct_commit(d, parents=None)
>>>return self.changeset[rev]
>>>
>>>def gettags(self):
>>> ___
>>> Mercurial-devel mailing list
>>> Mercurial-devel@mercurial-scm.org
>>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>> ___
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH RFC v7] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/15/16 4:49 PM, Augie Fackler wrote:
> On Wed, Dec 14, 2016 at 03:17:42AM +0100, Pierre-Yves David wrote:
>>
>> On 12/06/2016 10:53 PM, Jun Wu wrote:
>>> Excerpts from Kostia Balytskyi's message of 2016-12-06 13:41:07 -0800:
 class CorruptedState(Exception):
  """error raised when a command is not able to read its state from 
 file"""
 +
 +class MissingRequiredKeyInFileException(Exception):
 +"""error raised when simple key-value file misses a required key"""
>>> I still think "CorruptedState" is better. It fits the use-case, is short and
>>> concise. 6-word exception sounds strange to me, partially because everything
>>> else in error.py is at most 4 words. If we have to use a new exception,
>>> maybe just "MissingRequiredKey", or just "KeyError".
>> I've not looked at any logic or any context for this, but
>> "MissingRequiredKeyInFileException" is most certainly a
>> TooLongNameToBeAccepted
> +1 to MissingRequiredKey (let's avoid KeyError since this is slightly
> more domain-specific and that's a builtin name)
That's what I've sent in v8.
>
>> See https://lwn.net/Articles/455265/ for details ;-)
>>
>> Cheers,
>>
>> --
>> Pierre-Yves David
>> ___
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH RFC v8] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

On 12/16/2016 6:45 PM, Augie Fackler wrote:
> On Fri, Dec 16, 2016 at 1:35 PM, Augie Fackler  wrote:
>>> scmutil: add a simple key-value file helper
>>>
>> At this point, I think I'd like to see obs-shelve too. Is there a
>> place I can do that?
> (Specifically: before we do a resend of this, I'd like to see the
> client, so a link to a pastebin or repo where I could see a client on
> this thread would be outstanding. THanks!)
Here's a pastebin of my shelve.py: http://pastebin.com/3CYmqf2J
Please note that I haven't worked on storing the actual shelved state in 
this file yet: that is a low-pri for me at the moment, but I want it to 
be available for when I have time.
See obsshelvefile on line 88. The use case in shelve is very primitive - 
I could just store the node in a text file and it would've been ok. But 
I wanted to have a potential for growth and a way to move states of 
other commands as well.

> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH v9 RFC] scmutil: add a simple key-value file helper

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484824655 28800
#  Thu Jan 19 03:17:35 2017 -0800
# Node ID 19a449c91ef14e691cf1347748473e0094fedc86
# Parent  9f264adbe75bfae8551dc0e6e0fce8d43fc7b43a
scmutil: add a simple key-value file helper

The purpose of the added class is to serve purposes like save files of shelve
or state files of shelve, rebase and histedit. Keys of these files can be
alphanumeric and start with letters, while values must not contain newlines.
Keys which start with an uppercase letter are required, while other keys
are optional.

In light of Mercurial's reluctancy to use Python's json module, this tries
to provide a reasonable alternative for a non-nested named data.
Comparing to current approach of storing state in plain text files, where
semantic meaning of lines of text is only determined by their oreder,
simple key-value file allows for reordering lines and thus helps handle
optional values.

Initial use-case I see for this is obs-shelve's shelve files. Later we
can possibly migrate state files to this approach.

The test is in a new file beause I did not figure out where to put it
within existing test suite. If you give me a better idea, I will gladly
follow it.

diff --git a/mercurial/error.py b/mercurial/error.py
--- a/mercurial/error.py
+++ b/mercurial/error.py
@@ -246,3 +246,6 @@ class UnsupportedBundleSpecification(Exc
 
 class CorruptedState(Exception):
 """error raised when a command is not able to read its state from file"""
+
+class MissingRequiredKey(Exception):
+"""error raised when simple key-value file misses a required key"""
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1571,3 +1571,51 @@ class checkambigatclosing(closewrapbase)
 def close(self):
 self._origfh.close()
 self._checkambig()
+
+class simplekeyvaluefile(object):
+"""A simple file with key=value lines
+
+Keys must be alphanumerics and start with a letter, values must not
+contain '\n' characters"""
+
+# if KEYS is non-empty, read values are validated against it:
+# each key is a tuple (keyname, required)
+KEYS = []
+
+def __init__(self, vfs, path):
+self.vfs = vfs
+self.path = path
+
+def validate(self, d):
+for key, req in self.KEYS:
+if req and key not in d:
+e = "missing a required key: '%s'" % key
+raise error.MissingRequiredKey(e)
+
+def read(self):
+lines = self.vfs.readlines(self.path)
+try:
+d = dict(line[:-1].split('=', 1) for line in lines if line)
+except ValueError as e:
+raise error.CorruptedState(str(e))
+self.validate(d)
+return d
+
+def write(self, data):
+"""Write key=>value mapping to a file
+data is a dict. Keys must be alphanumerical and start with a letter.
+Values must not contain newline characters."""
+lines = []
+for k, v in data.items():
+if not k[0].isalpha():
+e = "keys must start with a letter in a key-value file"
+raise error.ProgrammingError(e)
+if not k.isalnum():
+e = "invalid key name in a simple key-value file"
+raise error.ProgrammingError(e)
+if '\n' in v:
+e = "invalid value in a simple key-value file"
+raise error.ProgrammingError(e)
+lines.append("%s=%s\n" % (k, v))
+with self.vfs(self.path, mode='wb', atomictemp=True) as fp:
+fp.write(''.join(lines))
diff --git a/tests/test-simplekeyvaluefile.py b/tests/test-simplekeyvaluefile.py
new file mode 100644
--- /dev/null
+++ b/tests/test-simplekeyvaluefile.py
@@ -0,0 +1,87 @@
+from __future__ import absolute_import
+
+import unittest
+import silenttestrunner
+
+from mercurial import (
+error,
+scmutil,
+)
+
+contents = {}
+
+class fileobj(object):
+def __init__(self, name):
+self.name = name
+
+def __enter__(self):
+return self
+
+def __exit__(self, *args, **kwargs):
+pass
+
+def write(self, text):
+contents[self.name] = text
+
+def read(self):
+return contents[self.name]
+
+class mockvfs(object):
+def read(self, path):
+return fileobj(path).read()
+
+def readlines(self, path):
+return fileobj(path).read().split('\n')
+
+def __call__(self, path, mode, atomictemp):
+return fileobj(path)
+
+class testsimplekeyvaluefile(unittest.TestCase):
+def setUp(self):
+self.vfs = mockvfs()
+
+def testbasicwriting(self):
+d = {'key1': 'value1', 'Key2

[PATCH 01 of 10 shelve-ext v2] shelve: add an ability to write key-val data to a new type of shelve files

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484835841 28800
#  Thu Jan 19 06:24:01 2017 -0800
# Node ID d904df83e9ead56f65104e10d765c0157d647401
# Parent  19a449c91ef14e691cf1347748473e0094fedc86
shelve: add an ability to write key-val data to a new type of shelve files

Obsolescense-based shelve only needs metadata stored in .hg/shelved
and if feels that this metadata should be stored in a
simplekeyvaluefile format for potential extensibility purposes.
I want to avoid storing it in an unstructured text file where
order of lines determines their semantical meanings (as now
happens in .hg/shelvedstate. .hg/rebasestate and I suspect other
state files as well).

Not included in this series, I have ~30 commits, doubling test-shelve.t
in size and testing almost every tested shelve usecase for obs-shelve.
Here's the series for the curious now: http://pastebin.com/tGJKx0vM
I would like to send it to the mailing list and get accepted as well,
but:
1. it's big, so should I send like 6 patches a time or so?
2. instead of having a commit per test case, it more like
   a commit per some amount of copy-pasted code. I tried to keep
   it meaningful and named commits somewhat properly, but it is
   far from this list standards IMO. Any advice on how to get it
   in without turning it into a 100 commits and spending many
   days writing descriptions?
3. it makes test-shelve.t run for twice as long (and it is already
   a slow test). Newest test-shelve.r runs for ~1 minute.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -62,7 +62,7 @@ testedwith = 'ships-with-hg-core'
 
 backupdir = 'shelve-backup'
 shelvedir = 'shelved'
-shelvefileextensions = ['hg', 'patch']
+shelvefileextensions = ['hg', 'patch', 'oshelve']
 # universal extension is present in all types of shelves
 patchextension = 'patch'
 
@@ -70,6 +70,9 @@ patchextension = 'patch'
 # generic user for all shelve operations
 shelveuser = 'shelve@localhost'
 
+class obsshelvefile(scmutil.simplekeyvaluefile):
+KEYS = [('node', True)]
+
 class shelvedfile(object):
 """Helper for the file storing a single shelve
 
@@ -153,6 +156,12 @@ class shelvedfile(object):
 bundle2.writebundle(self.ui, cg, self.fname, btype, self.vfs,
 compression=compression)
 
+def writeobsshelveinfo(self, info):
+obsshelvefile(self.vfs, self.fname).write(info)
+
+def readobsshelveinfo(self):
+return obsshelvefile(self.vfs, self.fname).read()
+
 class shelvedstate(object):
 """Handle persistence during unshelving operations.
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 10 of 10 shelve-ext v2] shelve: disable inhibit for shelving period

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484838335 28800
#  Thu Jan 19 07:05:35 2017 -0800
# Node ID 6a7e7659886d35dfdc4d7d5efb3ef8479ae35367
# Parent  088c9191d662d5c0003310119c51540926a815f7
shelve: disable inhibit for shelving period

While shelving, we're creating a new commit and updating to it.
If inhibit is enabled, it will try to add this commit to an
inhibition set, which is not necessary (we're creating a marker
right after we perform this operation). Thus, disabling inhibit
speeds things up a bit.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -447,8 +447,12 @@ def _docreatecmd(ui, repo, pats, opts):
 opts['message'] = desc
 
 lock = tr = activebookmark = None
+_obsinhibit = _notset = object()
 try:
 lock = repo.lock()
+if util.safehasattr(repo, '_obsinhibit'):
+_obsinhibit = getattr(repo, '_obsinhibit')
+del repo._obsinhibit
 
 # depending on whether shelve is traditional or
 # obsolescense-based, we either abort or commit this
@@ -498,6 +502,8 @@ def _docreatecmd(ui, repo, pats, opts):
 _finishshelve(ui, repo, tr, node, activebookmark)
 finally:
 _restoreactivebookmark(repo, activebookmark)
+if _obsinhibit is not _notset:
+repo._obsinhibit = _obsinhibit
 lockmod.release(tr, lock)
 
 def _isbareshelve(pats, opts):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 07 of 10 shelve-ext v2] shelve: migrate config overrides to ui.configoverride

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484740179 28800
#  Wed Jan 18 03:49:39 2017 -0800
# Node ID abdf9565fdce15604ea4abf013cb7c98a11f70ca
# Parent  149fc6d767ce3502528b43b0e209eb411dd6e842
shelve: migrate config overrides to ui.configoverride

This patch also makes ui.quiet manipulations much more explicit
and readable, addressing previous Yuya's concerns.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -349,17 +349,16 @@ def getcommitfunc(extra, interactive, ed
 hasmq = util.safehasattr(repo, 'mq')
 if hasmq:
 saved, repo.mq.checkapplied = repo.mq.checkapplied, False
-backup = repo.ui.backupconfig('phases', 'new-commit')
 try:
-repo.ui.setconfig('phases', 'new-commit', phases.secret)
-editor_ = False
-if editor:
-editor_ = cmdutil.getcommiteditor(editform='shelve.shelve',
-  **opts)
-return repo.commit(message, shelveuser, opts.get('date'), match,
-   editor=editor_, extra=extra)
+overrides = {('phases', 'new-commit'): phases.secret}
+with repo.ui.configoverride(overrides):
+editor_ = False
+if editor:
+editor_ = cmdutil.getcommiteditor(editform='shelve.shelve',
+  **opts)
+return repo.commit(message, shelveuser, opts.get('date'),
+   match, editor=editor_, extra=extra)
 finally:
-repo.ui.restoreconfig(backup)
 if hasmq:
 repo.mq.checkapplied = saved
 
@@ -621,9 +620,7 @@ def unshelveabort(ui, repo, state, opts)
 def mergefiles(ui, repo, wctx, shelvectx):
 """updates to wctx and merges the changes from shelvectx into the
 dirstate."""
-oldquiet = ui.quiet
-try:
-ui.quiet = True
+with ui.configoverride({('ui', 'quiet'): True}):
 hg.update(repo, wctx.node())
 files = []
 files.extend(shelvectx.files())
@@ -638,8 +635,6 @@ def mergefiles(ui, repo, wctx, shelvectx
*pathtofiles(repo, files),
**{'no_backup': True})
 ui.popbuffer()
-finally:
-ui.quiet = oldquiet
 
 def restorebranch(ui, repo, branchtorestore):
 if branchtorestore and branchtorestore != repo.dirstate.branch():
@@ -710,17 +705,16 @@ def _commitworkingcopychanges(ui, repo, 
 tempopts = {}
 tempopts['message'] = "pending changes temporary commit"
 tempopts['date'] = opts.get('date')
-ui.quiet = True
-node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
+with ui.configoverride({('ui', 'quiet'): True}):
+node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
 tmpwctx = repo[node]
 return tmpwctx, addedbefore
 
-def _unshelverestorecommit(ui, repo, basename, oldquiet):
+def _unshelverestorecommit(ui, repo, basename):
 """Recreate commit in the repository during the unshelve"""
-ui.quiet = True
-shelvedfile(repo, basename, 'hg').applybundle()
-shelvectx = repo['tip']
-ui.quiet = oldquiet
+with ui.configoverride({('ui', 'quiet'): True}):
+shelvedfile(repo, basename, 'hg').applybundle()
+shelvectx = repo['tip']
 return repo, shelvectx
 
 def _rebaserestoredcommit(ui, repo, opts, tr, oldtiprev, basename, pctx,
@@ -886,13 +880,9 @@ def _dounshelve(ui, repo, *shelved, **op
 if not shelvedfile(repo, basename, patchextension).exists():
 raise error.Abort(_("shelved change '%s' not found") % basename)
 
-oldquiet = ui.quiet
 lock = tr = None
-forcemerge = ui.backupconfig('ui', 'forcemerge')
 try:
-ui.setconfig('ui', 'forcemerge', opts.get('tool', ''), 'unshelve')
 lock = repo.lock()
-
 tr = repo.transaction('unshelve', report=lambda x: None)
 oldtiprev = len(repo)
 
@@ -907,16 +897,18 @@ def _dounshelve(ui, repo, *shelved, **op
 tmpwctx, addedbefore = _commitworkingcopychanges(ui, repo, opts,
  tmpwctx)
 
-repo, shelvectx = _unshelverestorecommit(ui, repo, basename, oldquiet)
+repo, shelvectx = _unshelverestorecommit(ui, repo, basename)
 
 branchtorestore = ''
 if shelvectx.branch() != shelvectx.p1().branch():
 branchtorestore = shelvectx.branch()
 
-shelvectx = _re

[PATCH 05 of 10 shelve-ext v2] shelve: add obs-based shelve functionality

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484740179 28800
#  Wed Jan 18 03:49:39 2017 -0800
# Node ID 44425c4e86b2589184e23bed798999c15788b54b
# Parent  ceb709491816f84789b77a18bfcab15eaa9860fe
shelve: add obs-based shelve functionality

Obsolescense-based shelve works in a following way:
1. In order to shelve some changes, it creates a commit, records its
node into a .oshelve file and prunes created commit.
2. In order to finish a shelve operation, transaction is just
closed and not aborted.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -373,10 +373,15 @@ def _nothingtoshelvemessaging(ui, repo, 
 else:
 ui.status(_("nothing changed\n"))
 
-def _shelvecreatedcommit(repo, node, name):
-bases = list(mutableancestors(repo[node]))
-shelvedfile(repo, name, 'hg').writebundle(bases, node)
-cmdutil.export(repo, [node],
+def _shelvecreatedcommit(ui, repo, node, name, tr):
+if isobsshelve(repo, ui):
+shelvedfile(repo, name, 'oshelve').writeobsshelveinfo({
+'node': nodemod.hex(node)
+})
+else:
+bases = list(mutableancestors(repo[node]))
+shelvedfile(repo, name, 'hg').writebundle(bases, node)
+cmdutil.export(repo.unfiltered(), [node],
fp=shelvedfile(repo, name, patchextension).opener('wb'),
opts=mdiff.diffopts(git=True))
 
@@ -387,8 +392,13 @@ def _includeunknownfiles(repo, pats, opt
 extra['shelve_unknown'] = '\0'.join(s.unknown)
 repo[None].add(s.unknown)
 
-def _finishshelve(repo):
-_aborttransaction(repo)
+def _finishshelve(ui, repo, tr, node):
+if isobsshelve(repo, ui):
+obsolete.createmarkers(repo, [(repo.unfiltered()[node], ())])
+tr.close()
+tr.release()
+else:
+_aborttransaction(repo)
 
 def _docreatecmd(ui, repo, pats, opts):
 wctx = repo[None]
@@ -410,9 +420,12 @@ def _docreatecmd(ui, repo, pats, opts):
 try:
 lock = repo.lock()
 
-# use an uncommitted transaction to generate the bundle to avoid
-# pull races. ensure we don't print the abort message to stderr.
-tr = repo.transaction('commit', report=lambda x: None)
+# depending on whether shelve is traditional or
+# obsolescense-based, we either abort or commit this
+# transaction in the end. If we abort it, we don't
+# want to print anything to stderr
+report = None if isobsshelve(repo, ui) else (lambda x: None)
+tr = repo.transaction('commit', report=report)
 
 interactive = opts.get('interactive', False)
 includeunknown = (opts.get('unknown', False) and
@@ -438,16 +451,19 @@ def _docreatecmd(ui, repo, pats, opts):
 _nothingtoshelvemessaging(ui, repo, pats, opts)
 return 1
 
-_shelvecreatedcommit(repo, node, name)
+_shelvecreatedcommit(ui, repo, node, name, tr)
 
 if ui.formatted():
 desc = util.ellipsis(desc, ui.termwidth())
 ui.status(_('shelved as %s\n') % name)
-hg.update(repo, parent.node())
+# current wc parent may be already obsolete becuase
+# it might have been created previously and shelve just
+# reuses it
+hg.update(repo.unfiltered(), parent.node())
 if origbranch != repo['.'].branch() and not _isbareshelve(pats, opts):
 repo.dirstate.setbranch(origbranch)
 
-_finishshelve(repo)
+_finishshelve(ui, repo, tr, node)
 finally:
 lockmod.release(tr, lock)
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 09 of 10 shelve-ext v2] shelve: add logic to preserve active bookmarks

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484740179 28800
#  Wed Jan 18 03:49:39 2017 -0800
# Node ID 088c9191d662d5c0003310119c51540926a815f7
# Parent  94a237a046059ef246aacb2c3ad809c9f0bdbe70
shelve: add logic to preserve active bookmarks

This adds an explicit active-bookmark-handling logic
to *both* traditional and obs-based shelve. Although it
is possible to only add it to obs-based, I think it would
be ugly and I see no harm in explicitly handling bookmarks
in addition to reliance on trasnactions.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -29,6 +29,7 @@ import time
 
 from mercurial.i18n import _
 from mercurial import (
+bookmarks,
 bundle2,
 bundlerepo,
 changegroup,
@@ -188,6 +189,8 @@ class shelvedstate(object):
 _nokeep = 'nokeep'
 _obsbased = 'obsbased'
 _traditional = 'traditional'
+# colon is essential to differentiate from a real bookmark name
+_noactivebook = ':no-active-bookmark'
 
 def __init__(self, ui, repo):
 self.ui = ui
@@ -210,6 +213,7 @@ class shelvedstate(object):
 branchtorestore = fp.readline().strip()
 keep = fp.readline().strip() == cls._keep
 obsshelve = fp.readline().strip() == cls._obsbased
+activebook = fp.readline().strip()
 except (ValueError, TypeError) as err:
 raise error.CorruptedState(str(err))
 finally:
@@ -225,6 +229,9 @@ class shelvedstate(object):
 obj.branchtorestore = branchtorestore
 obj.keep = keep
 obj.obsshelve = obsshelve
+obj.activebookmark = ''
+if activebook != cls._noactivebook:
+obj.activebookmark = activebook
 except error.RepoLookupError as err:
 raise error.CorruptedState(str(err))
 
@@ -232,7 +239,7 @@ class shelvedstate(object):
 
 @classmethod
 def save(cls, repo, name, originalwctx, pendingctx, nodestoprune,
- branchtorestore, keep=False, obsshelve=False):
+ branchtorestore, keep=False, obsshelve=False, activebook=''):
 fp = repo.vfs(cls._filename, 'wb')
 fp.write('%i\n' % cls._version)
 fp.write('%s\n' % name)
@@ -245,6 +252,7 @@ class shelvedstate(object):
 fp.write('%s\n' % branchtorestore)
 fp.write('%s\n' % (cls._keep if keep else cls._nokeep))
 fp.write('%s\n' % (cls._obsbased if obsshelve else cls._traditional))
+fp.write('%s\n' % (activebook or cls._noactivebook))
 fp.close()
 
 @classmethod
@@ -283,6 +291,16 @@ def cleanupoldbackups(repo):
 if err.errno != errno.ENOENT:
 raise
 
+def _backupactivebookmark(repo):
+activebookmark = repo._activebookmark
+if activebookmark:
+bookmarks.deactivate(repo)
+return activebookmark
+
+def _restoreactivebookmark(repo, mark):
+if mark:
+bookmarks.activate(repo, mark)
+
 def _aborttransaction(repo):
 '''Abort current transaction for shelve/unshelve, but keep dirstate
 '''
@@ -402,7 +420,9 @@ def _includeunknownfiles(repo, pats, opt
 extra['shelve_unknown'] = '\0'.join(s.unknown)
 repo[None].add(s.unknown)
 
-def _finishshelve(ui, repo, tr, node):
+def _finishshelve(ui, repo, tr, node, activebookmark):
+if activebookmark:
+bookmarks.activate(repo, activebookmark)
 if isobsshelve(repo, ui):
 obsolete.createmarkers(repo, [(repo.unfiltered()[node], ())])
 tr.close()
@@ -426,7 +446,7 @@ def _docreatecmd(ui, repo, pats, opts):
 if not opts.get('message'):
 opts['message'] = desc
 
-lock = tr = None
+lock = tr = activebookmark = None
 try:
 lock = repo.lock()
 
@@ -442,6 +462,7 @@ def _docreatecmd(ui, repo, pats, opts):
   not opts.get('addremove', False))
 
 name = getshelvename(repo, parent, opts)
+activebookmark = _backupactivebookmark(repo)
 extra = {}
 if includeunknown:
 _includeunknownfiles(repo, pats, opts, extra)
@@ -456,7 +477,8 @@ def _docreatecmd(ui, repo, pats, opts):
 node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
 else:
 node = cmdutil.dorecord(ui, repo, commitfunc, None,
-False, cmdutil.recordfilter, *pats, **opts)
+False, cmdutil.recordfilter, *pats,
+**opts)
 if not node:
 _nothingtoshelvemessaging(ui, repo, pats, opts)
 return 1
@@ -473,8 +495,9 @@ def _docreatecmd(ui, repo, pats, opts):
 if origbranch != repo['.'].branch() and not _isbareshelve(pats, opts):
 repo.dirstat

[PATCH 08 of 10 shelve-ext v2] shelve: add obs-based unshelve functionality

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484835394 28800
#  Thu Jan 19 06:16:34 2017 -0800
# Node ID 94a237a046059ef246aacb2c3ad809c9f0bdbe70
# Parent  abdf9565fdce15604ea4abf013cb7c98a11f70ca
shelve: add obs-based unshelve functionality

Obsolescense-based unshelve works as follows:
1. Instead of stripping temporary nodes, markers are created to
obsolete them.
2. Restoring commit is just finding it in an unfiltered repo.
3. '--keep' is only passed to rebase on traditional unshelves
(and thus traditional rebases), becuase we want markers to be
created fro obsolete-based rebases.
4. 'hg unshelve' uses unfiltered repo to perform rebases
because we want rebase to be able to create markers between original
and new commits. 'rebaseskipobsolete' is disabled to make rebase not
skip the commit altogether.

I have a test for the case when shelve node has been stripped before
unshelve call, that test is together with ~30 commits I was talking
about in patch 1.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -25,6 +25,7 @@ from __future__ import absolute_import
 import collections
 import errno
 import itertools
+import time
 
 from mercurial.i18n import _
 from mercurial import (
@@ -252,8 +253,13 @@ class shelvedstate(object):
 
 def prunenodes(self):
 """Cleanup temporary nodes from the repo"""
-repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
- topic='shelve')
+if self.obsshelve:
+unfi = self.repo.unfiltered()
+relations = [(unfi[n], ()) for n in self.nodestoprune]
+obsolete.createmarkers(self.repo, relations)
+else:
+repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
+ topic='shelve')
 
 def cleanupoldbackups(repo):
 vfs = scmutil.vfs(repo.join(backupdir))
@@ -666,9 +672,14 @@ def unshelvecontinue(ui, repo, state, op
 util.rename(repo.join('unshelverebasestate'),
 repo.join('rebasestate'))
 try:
-rebase.rebase(ui, repo, **{
-'continue' : True
-})
+# if shelve is obs-based, we want rebase to be able
+# to create markers to already-obsoleted commits
+_repo = repo.unfiltered() if state.obsshelve else repo
+with ui.configoverride({('experimental', 'rebaseskipobsolete'):
+'off'}, 'unshelve'):
+rebase.rebase(ui, _repo, **{
+'continue' : True,
+})
 except Exception:
 util.rename(repo.join('rebasestate'),
 repo.join('unshelverebasestate'))
@@ -708,30 +719,58 @@ def _commitworkingcopychanges(ui, repo, 
 with ui.configoverride({('ui', 'quiet'): True}):
 node = cmdutil.commit(ui, repo, commitfunc, [], tempopts)
 tmpwctx = repo[node]
+ui.debug("temporary working copy commit: %s:%s\n" %
+ (tmpwctx.rev(), nodemod.short(node)))
 return tmpwctx, addedbefore
 
-def _unshelverestorecommit(ui, repo, basename):
+def _unshelverestorecommit(ui, repo, basename, obsshelve):
 """Recreate commit in the repository during the unshelve"""
 with ui.configoverride({('ui', 'quiet'): True}):
-shelvedfile(repo, basename, 'hg').applybundle()
-shelvectx = repo['tip']
+if obsshelve:
+md = shelvedfile(repo, basename, 'oshelve').readobsshelveinfo()
+shelvenode = nodemod.bin(md['node'])
+repo = repo.unfiltered()
+try:
+shelvectx = repo[shelvenode]
+except error.RepoLookupError:
+m = _("shelved node %s not found in repo")
+raise error.Abort(m % md['node'])
+else:
+shelvedfile(repo, basename, 'hg').applybundle()
+shelvectx = repo['tip']
 return repo, shelvectx
 
 def _rebaserestoredcommit(ui, repo, opts, tr, oldtiprev, basename, pctx,
-  tmpwctx, shelvectx, branchtorestore):
+  tmpwctx, shelvectx, branchtorestore, obsshelve):
 """Rebase restored commit from its original location to a destination"""
 # If the shelve is not immediately on top of the commit
 # we'll be merging with, rebase it to be on top.
 if tmpwctx.node() == shelvectx.parents()[0].node():
+# shelvectx is immediately on top of the tmpwctx
 return shelvectx
 
+# we need a new commit extra every time we perform a rebase to ensure
+# that &q

[PATCH 04 of 10 shelve-ext v2] shelve: add a function to check whether obs-based shelve is enabled

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484740179 28800
#  Wed Jan 18 03:49:39 2017 -0800
# Node ID ceb709491816f84789b77a18bfcab15eaa9860fe
# Parent  249273f6db8bf0fdfbbf36bcbd9e3553e5715212
shelve: add a function to check whether obs-based shelve is enabled

A central place to check whether code should use traditional or
obsolescense-based shelve behavior.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -40,6 +40,7 @@ from mercurial import (
 mdiff,
 merge,
 node as nodemod,
+obsolete,
 patch,
 phases,
 repair,
@@ -70,6 +71,18 @@ patchextension = 'patch'
 # generic user for all shelve operations
 shelveuser = 'shelve@localhost'
 
+def isobsshelve(repo, ui):
+"""Check whether obsolescense-based shelve is enabled"""
+obsshelve = ui.configbool('experimental', 'obsshelve')
+if not obsshelve:
+return False
+if not obsolete.isenabled(repo, obsolete.createmarkersopt):
+w = _('ignoring experimental.obsshelve because createmarkers option '
+  'is disabled')
+ui.warn(w)
+return False
+return True
+
 class obsshelvefile(scmutil.simplekeyvaluefile):
 KEYS = [('node', True)]
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 03 of 10 shelve-ext v2] shelve: move node-pruning functionality to be member of shelvedstate

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484740179 28800
#  Wed Jan 18 03:49:39 2017 -0800
# Node ID 249273f6db8bf0fdfbbf36bcbd9e3553e5715212
# Parent  155f97b77a4866075fa709daa3bef6dac9108e81
shelve: move node-pruning functionality to be member of shelvedstate

Node-pruning can be node stripping or marker creation, depending on
whether shelve is traditional or obs-based. Thus it makes sense to
move it to a separate function. Also, since we already have
shelvedstate object and this functionality operates on that object,
it makes sense to make it a method.

Having shelvedstate object contain repo and ui as members allows for
calling 'state.prunenodes()' instead of 'state.prunenodes(repo, ui)'
which is better IMO.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -173,8 +173,12 @@ class shelvedstate(object):
 _keep = 'keep'
 _nokeep = 'nokeep'
 
+def __init__(self, ui, repo):
+self.ui = ui
+self.repo = repo
+
 @classmethod
-def load(cls, repo):
+def load(cls, ui, repo):
 fp = repo.vfs(cls._filename)
 try:
 version = int(fp.readline().strip())
@@ -195,7 +199,7 @@ class shelvedstate(object):
 fp.close()
 
 try:
-obj = cls()
+obj = cls(ui, repo)
 obj.name = name
 obj.wctx = repo[wctx]
 obj.pendingctx = repo[pendingctx]
@@ -228,6 +232,11 @@ class shelvedstate(object):
 def clear(cls, repo):
 util.unlinkpath(repo.join(cls._filename), ignoremissing=True)
 
+def prunenodes(self):
+"""Cleanup temporary nodes from the repo"""
+repair.strip(self.ui, self.repo, self.nodestoprune, backup=False,
+ topic='shelve')
+
 def cleanupoldbackups(repo):
 vfs = scmutil.vfs(repo.join(backupdir))
 maxbackups = repo.ui.configint('shelve', 'maxbackups', 10)
@@ -570,8 +579,7 @@ def unshelveabort(ui, repo, state, opts)
 raise
 
 mergefiles(ui, repo, state.wctx, state.pendingctx)
-repair.strip(ui, repo, state.nodestoprune, backup=False,
- topic='shelve')
+state.prunenodes()
 finally:
 shelvedstate.clear(repo)
 ui.warn(_("unshelve of '%s' aborted\n") % state.name)
@@ -648,7 +656,7 @@ def unshelvecontinue(ui, repo, state, op
 mergefiles(ui, repo, state.wctx, shelvectx)
 restorebranch(ui, repo, state.branchtorestore)
 
-repair.strip(ui, repo, state.nodestoprune, backup=False, 
topic='shelve')
+state.prunenodes()
 shelvedstate.clear(repo)
 unshelvecleanup(ui, repo, state.name, opts)
 ui.status(_("unshelve of '%s' complete\n") % state.name)
@@ -804,7 +812,7 @@ def _dounshelve(ui, repo, *shelved, **op
 ui.warn(_('tool option will be ignored\n'))
 
 try:
-state = shelvedstate.load(repo)
+state = shelvedstate.load(ui, repo)
 if opts.get('keep') is None:
 opts['keep'] = state.keep
 except IOError as err:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 06 of 10 shelve-ext v2] shelve: add shelve type saving and loading

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484740179 28800
#  Wed Jan 18 03:49:39 2017 -0800
# Node ID 149fc6d767ce3502528b43b0e209eb411dd6e842
# Parent  44425c4e86b2589184e23bed798999c15788b54b
shelve: add shelve type saving and loading

We need shelve type to be stored in .hg/shelvedstate in order
to be able to run abort or continue action properly. If the shelve
is obsbased, those actions should create markes, if it is traditional,
the actions should strip commits.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -185,6 +185,8 @@ class shelvedstate(object):
 _filename = 'shelvedstate'
 _keep = 'keep'
 _nokeep = 'nokeep'
+_obsbased = 'obsbased'
+_traditional = 'traditional'
 
 def __init__(self, ui, repo):
 self.ui = ui
@@ -206,6 +208,7 @@ class shelvedstate(object):
 nodestoprune = [nodemod.bin(h) for h in fp.readline().split()]
 branchtorestore = fp.readline().strip()
 keep = fp.readline().strip() == cls._keep
+obsshelve = fp.readline().strip() == cls._obsbased
 except (ValueError, TypeError) as err:
 raise error.CorruptedState(str(err))
 finally:
@@ -220,6 +223,7 @@ class shelvedstate(object):
 obj.nodestoprune = nodestoprune
 obj.branchtorestore = branchtorestore
 obj.keep = keep
+obj.obsshelve = obsshelve
 except error.RepoLookupError as err:
 raise error.CorruptedState(str(err))
 
@@ -227,7 +231,7 @@ class shelvedstate(object):
 
 @classmethod
 def save(cls, repo, name, originalwctx, pendingctx, nodestoprune,
- branchtorestore, keep=False):
+ branchtorestore, keep=False, obsshelve=False):
 fp = repo.vfs(cls._filename, 'wb')
 fp.write('%i\n' % cls._version)
 fp.write('%s\n' % name)
@@ -239,6 +243,7 @@ class shelvedstate(object):
  ' '.join([nodemod.hex(n) for n in nodestoprune]))
 fp.write('%s\n' % branchtorestore)
 fp.write('%s\n' % (cls._keep if keep else cls._nokeep))
+fp.write('%s\n' % (cls._obsbased if obsshelve else cls._traditional))
 fp.close()
 
 @classmethod
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 02 of 10 shelve-ext v2] shelve: rename stripnodes to nodestoprune

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484740179 28800
#  Wed Jan 18 03:49:39 2017 -0800
# Node ID 155f97b77a4866075fa709daa3bef6dac9108e81
# Parent  d904df83e9ead56f65104e10d765c0157d647401
shelve: rename stripnodes to nodestoprune

Since we are introducing obs-based shelve, we are no longer
stripping temporary nodes, we are obsoleting them. Therefore
it looks like stipnodes would be a misleading name, while
prune has a connotaion of "strip but with obsolescense", so
nodestoprune seems like a good rename.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -186,7 +186,7 @@ class shelvedstate(object):
 wctx = nodemod.bin(fp.readline().strip())
 pendingctx = nodemod.bin(fp.readline().strip())
 parents = [nodemod.bin(h) for h in fp.readline().split()]
-stripnodes = [nodemod.bin(h) for h in fp.readline().split()]
+nodestoprune = [nodemod.bin(h) for h in fp.readline().split()]
 branchtorestore = fp.readline().strip()
 keep = fp.readline().strip() == cls._keep
 except (ValueError, TypeError) as err:
@@ -200,7 +200,7 @@ class shelvedstate(object):
 obj.wctx = repo[wctx]
 obj.pendingctx = repo[pendingctx]
 obj.parents = parents
-obj.stripnodes = stripnodes
+obj.nodestoprune = nodestoprune
 obj.branchtorestore = branchtorestore
 obj.keep = keep
 except error.RepoLookupError as err:
@@ -209,7 +209,7 @@ class shelvedstate(object):
 return obj
 
 @classmethod
-def save(cls, repo, name, originalwctx, pendingctx, stripnodes,
+def save(cls, repo, name, originalwctx, pendingctx, nodestoprune,
  branchtorestore, keep=False):
 fp = repo.vfs(cls._filename, 'wb')
 fp.write('%i\n' % cls._version)
@@ -219,7 +219,7 @@ class shelvedstate(object):
 fp.write('%s\n' %
  ' '.join([nodemod.hex(p) for p in repo.dirstate.parents()]))
 fp.write('%s\n' %
- ' '.join([nodemod.hex(n) for n in stripnodes]))
+ ' '.join([nodemod.hex(n) for n in nodestoprune]))
 fp.write('%s\n' % branchtorestore)
 fp.write('%s\n' % (cls._keep if keep else cls._nokeep))
 fp.close()
@@ -570,7 +570,7 @@ def unshelveabort(ui, repo, state, opts)
 raise
 
 mergefiles(ui, repo, state.wctx, state.pendingctx)
-repair.strip(ui, repo, state.stripnodes, backup=False,
+repair.strip(ui, repo, state.nodestoprune, backup=False,
  topic='shelve')
 finally:
 shelvedstate.clear(repo)
@@ -643,12 +643,12 @@ def unshelvecontinue(ui, repo, state, op
 shelvectx = state.pendingctx
 else:
 # only strip the shelvectx if the rebase produced it
-state.stripnodes.append(shelvectx.node())
+state.nodestoprune.append(shelvectx.node())
 
 mergefiles(ui, repo, state.wctx, shelvectx)
 restorebranch(ui, repo, state.branchtorestore)
 
-repair.strip(ui, repo, state.stripnodes, backup=False, topic='shelve')
+repair.strip(ui, repo, state.nodestoprune, backup=False, 
topic='shelve')
 shelvedstate.clear(repo)
 unshelvecleanup(ui, repo, state.name, opts)
 ui.status(_("unshelve of '%s' complete\n") % state.name)
@@ -700,9 +700,9 @@ def _rebaserestoredcommit(ui, repo, opts
 except error.InterventionRequired:
 tr.close()
 
-stripnodes = [repo.changelog.node(rev)
-  for rev in xrange(oldtiprev, len(repo))]
-shelvedstate.save(repo, basename, pctx, tmpwctx, stripnodes,
+nodestoprune = [repo.changelog.node(rev)
+for rev in xrange(oldtiprev, len(repo))]
+shelvedstate.save(repo, basename, pctx, tmpwctx, nodestoprune,
   branchtorestore, opts.get('keep'))
 
 util.rename(repo.join('rebasestate'),
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH shelve-ext] shelve: make unshelve not crash when there are missing files (issue4176)

[Kostia Balytskyi]

# HG changeset patch
# User Kostia Balytskyi 
# Date 1484848120 28800
#  Thu Jan 19 09:48:40 2017 -0800
# Node ID 2a1e998e369d2c5dc828ba805beceb15459746cd
# Parent  9f264adbe75bfae8551dc0e6e0fce8d43fc7b43a
shelve: make unshelve not crash when there are missing files (issue4176)

This patch makes it possible to unshelve while having missing files
in your repo as long as shelved changes don't touch those missing files.
It also makes error message better otherwise.

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -650,7 +650,7 @@ def _commitworkingcopychanges(ui, repo, 
 # contains unknown files that are part of the pending change
 s = repo.status()
 addedbefore = frozenset(s.added)
-if not (s.modified or s.added or s.removed or s.deleted):
+if not (s.modified or s.added or s.removed):
 return tmpwctx, addedbefore
 ui.status(_("temporarily committing pending changes "
 "(restore with 'hg unshelve --abort')\n"))
@@ -729,6 +729,17 @@ def _finishunshelve(repo, oldtiprev, tr)
 repo.unfiltered().changelog.strip(oldtiprev, tr)
 _aborttransaction(repo)
 
+def _checkunshelveuntrackedproblems(ui, repo, shelvectx):
+"""Check potential problems which may result from working
+copy having untracked changes."""
+wcdeleted = set(repo.status().deleted)
+shelvetouched = set(shelvectx.files())
+intersection = wcdeleted.intersection(shelvetouched)
+if intersection:
+m = _("shelved change touches missing files")
+hint = _("run hg status to see which files are missing")
+raise error.Abort(m, hint=hint)
+
 @command('unshelve',
  [('a', 'abort', None,
_('abort an incomplete unshelve operation')),
@@ -857,7 +868,7 @@ def _dounshelve(ui, repo, *shelved, **op
  tmpwctx)
 
 repo, shelvectx = _unshelverestorecommit(ui, repo, basename, oldquiet)
-
+_checkunshelveuntrackedproblems(ui, repo, shelvectx)
 branchtorestore = ''
 if shelvectx.branch() != shelvectx.p1().branch():
 branchtorestore = shelvectx.branch()
diff --git a/tests/test-shelve.t b/tests/test-shelve.t
--- a/tests/test-shelve.t
+++ b/tests/test-shelve.t
@@ -1710,3 +1710,30 @@ Unshelve respects --keep even if user in
   $ hg shelve --list
   default (*s ago)changes to: 1 (glob)
   $ cd ..
+
+Unshelving when there are deleted files does not crash (issue4176)
+  $ hg init unshelve-deleted-file && cd unshelve-deleted-file
+  $ echo a > a && echo b > b && hg ci -Am ab
+  adding a
+  adding b
+  $ echo aa > a && hg shelve
+  shelved as default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ rm b
+  $ hg st
+  ! b
+  $ hg unshelve
+  unshelving change 'default'
+  $ hg shelve
+  shelved as default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ rm a && echo b > b
+  $ hg st
+  ! a
+  $ hg unshelve
+  unshelving change 'default'
+  abort: shelved change touches missing files
+  (run hg status to see which files are missing)
+  [255]
+  $ hg st
+  ! a
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel