Re: Storage format for remotenames.

2017-11-08 Thread Gregory Szorc 


> On Nov 8, 2017, at 06:00, Yuya Nishihara  wrote:
> 
>> On Tue, 7 Nov 2017 09:58:04 -0800, Durham Goode wrote:
>> I wish we had some easily reusable serializer/deserializer instead of 
>> having to reinvent these every time.  What's our reasoning for not using 
>> json? I forget. If there are some weird characters, like control 
>> characters or something, that break json, I'd say we just use json and 
>> prevent users from creating bookmarks and paths with those names.
> 
> Just about json. Using json (in Mercurial) is bad because it's so easily
> to spill out unicode objects without noticing that, all tests pass (because
> the whole data is ascii), and we'll get nice UnicodeError in production.

This issue can be prevented with diligent coding (and possibly a custom wrapper 
to convert unicode to bytes). Python 3 would also uncover type coercion.

> 
> Another concern is that encoding conversion can be lossy even if it goes
> with no error. There are n:m mappings between unicode and legacy encoding.
> For example, we have three major Shift_JISes in Japan, and the Microsoft one
> allocates multiple code points for one character for "compatibility" reasons.

This is the bigger problem. JSON doesn’t do a good job at preserving byte 
sequences unless strings are valid UTF-8. The most common way to robustly round 
trip arbitrary byte sequences through JSON is to apply an encoding to string 
fields that won’t result in escape characters in JSON. Base64 is common.

Avoiding code points that need escaped in JSON seems reasonable for some use 
cases. For things like storing the author field in obs markers, it is not.

I’d just as soon we vendor and use a binary serialization format like Protobuf, 
Thrift, Capnproto, Msgpack, Avro, etc. Bonus points if Rust’s serde crate can 
parse it using zero copy.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1351: changegroup: use any node, not min(), in treemanifest's generatemanifests

2017-11-08 Thread martinvonz (Martin von Zweigbergk) 
martinvonz added a comment.


  As the author of that line of code, this looks good to me. I'll let someone 
else queue it since I was involved in the internal discussion and I don't want 
to feel like I'm queuing my own code (even though I wasn't involved in the 
actual coding).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1351

To: spectral, #hg-reviewers
Cc: martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1336: HG: hg rm -A option prints the message of every file in the repo

2017-11-08 Thread mharbison72 (Matt Harbison) 
mharbison72 added a comment.


  Thank you for the proposed change.  Does --quiet work for you?
  
  I don't think I necessarily agree with the premise here.  I think the only 
way every file in the repo is printed is if you delete every file first.  Many 
file operations like add, remove/forget, and addremove all follow the style of 
quiet if files are explicitly named, and verbose for inexact matches.  I've 
come to rely on that to catch mistakes when I'm not explicit about file paths.  
I'm not sure if it's worse to dump a ton of output to the screen, or not 
consistently follow the rules.
  
  I'm sure others will chime in, but in the meantime, there's an inline comment 
too.

INLINE COMMENTS

> cmdutil.py:2970
>  list = modified + deleted + clean + added
> -elif after:
> +elif after or ui.verbose:
>  list = deleted

Should ui.verbose be nested inside the elif instead?  The -v without -A case 
would have taken the 'else' case before, so this change seems to affect other 
things unintentionally.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1336

To: pavanpc, #hg-reviewers
Cc: mharbison72, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1351: changegroup: use any node, not min(), in treemanifest's generatemanifests

2017-11-08 Thread spectral (Kyle Lippincott) 
spectral created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is fixing quadratic behavior, which is probably not noticeable in the
  common case, but if a very large directory gets added here, it can get pretty
  bad. This was noticed because we had some pushes that spent >25s in 
changegroup
  generation calling min() here, according to profiling.
  
  The original reasoning for min() being used in 
https://phab.mercurial-scm.org/rHG829d369fc5a89f4c290013271c6e5dff2aea63de was 
that, at that
  point in the series, we were adding almost everything to tmfnodes during the
  first iteration through the loop , so we needed to avoid sending child
  directories before parents. Later changes made it so that the child 
directories
  were added only when we visited the parent directory (not all of them on the
  first iteration), so this is no longer necessary - there won't be any child
  directories in tmfnodes before the parents have been sent.
  
  This does mean that the manifests are now exchanged unordered, whereas
  previously we would essentially do [a, b, b/c, b/c/d, e], we now can send a, 
b,
  and e in any order; b/c must still follow b, and b/c/d must still follow b/c.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1351

AFFECTED FILES
  mercurial/changegroup.py

CHANGE DETAILS

diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -733,7 +733,9 @@
 
 size = 0
 while tmfnodes:
-dir = min(tmfnodes)
+# Pick some element from tmfnodes, this is not necessarily the 
'min'
+# element.
+dir = next(iter(tmfnodes))
 nodes = tmfnodes[dir]
 prunednodes = self.prune(dirlog(dir), nodes, commonrevs)
 if not dir or prunednodes:



To: spectral, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] debuglocks: allow setting a lock

2017-11-08 Thread Paul Morelle 
# HG changeset patch
# User Paul Morelle 
# Date 1510071568 -3600
#  Tue Nov 07 17:19:28 2017 +0100
# Node ID 5300b33397d0651eb2457502204969585d492cc5
# Parent  602c168c0207c443ac61f7a7c727b31cfb0b86ad
# EXP-Topic debugsetlocks
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
5300b33397d0
debuglocks: allow setting a lock

diff -r 602c168c0207 -r 5300b33397d0 mercurial/debugcommands.py
--- a/mercurial/debugcommands.pyTue Nov 07 13:18:49 2017 -0500
+++ b/mercurial/debugcommands.pyTue Nov 07 17:19:28 2017 +0100
@@ -1173,7 +1173,10 @@
 @command('debuglocks',
  [('L', 'force-lock', None, _('free the store lock (DANGEROUS)')),
   ('W', 'force-wlock', None,
-   _('free the working state lock (DANGEROUS)'))],
+   _('free the working state lock (DANGEROUS)')),
+  ('s', 'set-lock', None, _('set the store lock until stopped')),
+  ('S', 'set-wlock', None,
+   _('set the working state lock until stopped'))],
  _('[OPTION]...'))
 def debuglocks(ui, repo, **opts):
 """show or modify state of locks
@@ -1192,6 +1195,10 @@
 instance, on a shared filesystem). Removing locks may also be
 blocked by filesystem permissions.
 
+Setting a lock will prevent other commands from changing the data.
+The command will wait until an interruption (SIGINT, SIGTERM, ...) occurs.
+The set lock(s) is (are) removed when the command exits.
+
 Returns 0 if no locks are held.
 
 """
@@ -1203,6 +1210,26 @@
 if opts.get(r'force_lock') or opts.get(r'force_lock'):
 return 0
 
+locks = []
+if opts.get(r'set_wlock'):
+try:
+locks.append(repo.wlock(False))
+except error.LockHeld:
+raise error.Abort(_('wlock is already held'))
+if opts.get(r'set_lock'):
+try:
+locks.append(repo.lock(False))
+except error.LockHeld:
+raise error.Abort(_('lock is already held'))
+if len(locks):
+try:
+while True:
+time.sleep(60)
+except:
+for lock in locks:
+lock.release()
+raise
+
 now = time.time()
 held = 0
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] debugdeltachain: output information about sparse read if enabled

2017-11-08 Thread Paul Morelle 
# HG changeset patch
# User Paul Morelle 
# Date 1509002829 -7200
#  Thu Oct 26 09:27:09 2017 +0200
# Node ID 13a6c881be35e7651a12f8c3442abfade2b77c88
# Parent  602c168c0207c443ac61f7a7c727b31cfb0b86ad
# EXP-Topic debugdeltachain
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
13a6c881be35
debugdeltachain: output information about sparse read if enabled

diff -r 602c168c0207 -r 13a6c881be35 mercurial/debugcommands.py
--- a/mercurial/debugcommands.pyTue Nov 07 13:18:49 2017 -0500
+++ b/mercurial/debugcommands.pyThu Oct 26 09:27:09 2017 +0200
@@ -587,11 +587,22 @@
 the delta chain for this revision
 :``extraratio``: extradist divided by chainsize; another representation of
 how much unrelated data is needed to load this delta chain
+
+If the repository is configured to use the sparse read, additional keywords
+are available:
+
+:``readsize``: total size of data read from the disk for a revision
+   (sum of the sizes of all the blocks)
+:``largestblock``: size of the largest block of data read from the disk
+:``readdensity``:  density of useful bytes in the data read from the disk
+
+The sparse read can be enabled with experimental.sparse-read = True
 """
 opts = pycompat.byteskwargs(opts)
 r = cmdutil.openrevlog(repo, 'debugdeltachain', file_, opts)
 index = r.index
 generaldelta = r.version & revlog.FLAG_GENERALDELTA
+withsparseread = getattr(r, '_withsparseread', False)
 
 def revinfo(rev):
 e = index[rev]
@@ -625,17 +636,29 @@
 
 fm = ui.formatter('debugdeltachain', opts)
 
-fm.plain('rev  chain# chainlen prev   delta   '
- 'sizerawsize  chainsize ratio   lindist extradist '
- 'extraratio\n')
+header = ('rev  chain# chainlen prev   delta   '
+  'sizerawsize  chainsize ratio   lindist extradist '
+  'extraratio')
+fmfields = ('rev chainid chainlen prevrev deltatype compsize '
+'uncompsize chainsize chainratio lindist extradist '
+'extraratio')
+fmformats = '%7d %7d %8d %8d %7s %10d %10d %10d %9.5f %9d %9d %10.5f'
+if withsparseread:
+header += '   readsize largestblk rddensity'
+fmfields += ' readsize largestblock readdensity'
+fmformats += ' %10d %10d %9.5f'
+
+fm.plain(header + '\n')
 
 chainbases = {}
 for rev in r:
 comp, uncomp, deltatype, chain, chainsize = revinfo(rev)
 chainbase = chain[0]
 chainid = chainbases.setdefault(chainbase, len(chainbases) + 1)
-basestart = r.start(chainbase)
-revstart = r.start(rev)
+start = r.start
+length = r.length
+basestart = start(chainbase)
+revstart = start(rev)
 lineardist = revstart + comp - basestart
 extradist = lineardist - chainsize
 try:
@@ -646,19 +669,33 @@
 chainratio = float(chainsize) / float(uncomp)
 extraratio = float(extradist) / float(chainsize)
 
+fmargs = (rev, chainid, len(chain), prevrev, deltatype, comp,
+  uncomp, chainsize, chainratio, lineardist, extradist,
+  extraratio)
+fmkwargs = dict(rev=rev, chainid=chainid, chainlen=len(chain),
+prevrev=prevrev, deltatype=deltatype, compsize=comp,
+uncompsize=uncomp, chainsize=chainsize,
+chainratio=chainratio, lindist=lineardist,
+extradist=extradist, extraratio=extraratio)
+if withsparseread:
+readsize = 0
+largestblock = 0
+for revschunk in revlog._slicechunk(r, chain):
+blkend = start(revschunk[-1]) + length(revschunk[-1])
+blksize = blkend - start(revschunk[0])
+
+readsize += blksize
+if largestblock < blksize:
+largestblock = blksize
+
+readdensity = float(chainsize) / float(readsize)
+
+fmargs += (readsize, largestblock, readdensity)
+fmkwargs.update(readsize=readsize, largestblock=largestblock,
+readdensity=readdensity)
+
 fm.startitem()
-fm.write('rev chainid chainlen prevrev deltatype compsize '
- 'uncompsize chainsize chainratio lindist extradist '
- 'extraratio',
- '%7d %7d %8d %8d %7s %10d %10d %10d %9.5f %9d %9d %10.5f\n',
- rev, chainid, len(chain), prevrev, deltatype, comp,
- uncomp, chainsize, chainratio, lineardist, extradist,
- extraratio,
- rev=rev, chainid=chainid, chainlen=len(chain),
- prevrev=prevrev, deltatype=deltatype, compsize=comp,
- uncompsize=uncomp, chainsize=chain

D1348: histedit: add support to output nodechanges using formatter

2017-11-08 Thread pulkit (Pulkit Goyal) 
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The JSON output of nodechanges will help in automation and to improve editor
  integrations such as for Nuclide.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1348

AFFECTED FILES
  hgext/histedit.py
  tests/test-histedit-templates.t

CHANGE DETAILS

diff --git a/tests/test-histedit-templates.t b/tests/test-histedit-templates.t
new file mode 100644
--- /dev/null
+++ b/tests/test-histedit-templates.t
@@ -0,0 +1,54 @@
+Testing templating for histedit command
+
+Setup
+
+  $ cat >> $HGRCPATH < [extensions]
+  > histedit=
+  > [experimental]
+  > evolution=createmarkers
+  > EOF
+
+  $ hg init repo
+  $ cd repo
+  $ for ch in a b c d; do echo foo > $ch; hg commit -Aqm "Added "$ch; done
+
+  $ hg log -G -T "{rev}:{node|short} {desc}"
+  @  3:62615734edd5 Added d
+  |
+  o  2:28ad74487de9 Added c
+  |
+  o  1:29becc82797a Added b
+  |
+  o  0:18d04c59bb5d Added a
+  
+Getting the JSON output for nodechanges
+
+  $ hg histedit -Tjson --commands - 2>&1 < pick 28ad74487de9 Added c
+  > pick 62615734edd5 Added d
+  > pick 18d04c59bb5d Added a
+  > pick 29becc82797a Added b
+  > EOF
+  [
+   {
+"nodechanges": {"18d04c59bb5d2d4090ad9a5b59bd6274adb63add": 
["109f8ec895447f81b380ba8d4d8b66539ccdcb94"], 
"28ad74487de9599d00d81085be739c61fc340652": 
["bff9e07c1807942b161dab768aa793b48e9a7f9d"], 
"29becc82797a4bc11ec8880b58eaecd2ab3e7760": 
["f5dcf3b4db23f31f1aacf46c33d1393de303d26f"], 
"62615734edd52f06b6fb9c2beb429e4fe30d57b8": 
["201423b441c84d9e6858daed653e0d22485c1cfa"]}
+   }
+  ]
+
+  $ hg log -G -T "{rev}:{node|short} {desc}"
+  @  7:f5dcf3b4db23 Added b
+  |
+  o  6:109f8ec89544 Added a
+  |
+  o  5:201423b441c8 Added d
+  |
+  o  4:bff9e07c1807 Added c
+  
+  $ hg histedit -T "{nodechanges|json}" --commands - 2>&1 < pick bff9e07c1807 Added c
+  > pick 201423b441c8 Added d
+  > pick 109f8ec89544 Added a
+  > roll f5dcf3b4db23 Added b
+  > EOF
+  {"109f8ec895447f81b380ba8d4d8b66539ccdcb94": 
["8d01470bfeab64d3de13c49adb79d88790d38396"], 
"f3ec56a374bdbdf1953cacca505161442c6f3a3e": [], 
"f5dcf3b4db23f31f1aacf46c33d1393de303d26f": 
["8d01470bfeab64d3de13c49adb79d88790d38396"]} (no-eol)
diff --git a/hgext/histedit.py b/hgext/histedit.py
--- a/hgext/histedit.py
+++ b/hgext/histedit.py
@@ -916,7 +916,8 @@
  ('o', 'outgoing', False, _('changesets not found in destination')),
  ('f', 'force', False,
   _('force outgoing even for unrelated repositories')),
- ('r', 'rev', [], _('first revision to be edited'), _('REV'))],
+ ('r', 'rev', [], _('first revision to be edited'), _('REV'))] +
+  cmdutil.templateopts,
  _("[OPTIONS] ([ANCESTOR] | --outgoing [URL])"))
 def histedit(ui, repo, *freeargs, **opts):
 """interactively edit changeset history
@@ -1093,6 +1094,8 @@
 _('histedit requires exactly one ancestor revision'))
 
 def _histedit(ui, repo, state, *freeargs, **opts):
+fm = ui.formatter('histedit', opts)
+fm.startitem()
 goal = _getgoal(opts)
 revs = opts.get('rev', [])
 rules = opts.get('commands', '')
@@ -1115,7 +1118,8 @@
 _newhistedit(ui, repo, state, revs, freeargs, opts)
 
 _continuehistedit(ui, repo, state)
-_finishhistedit(ui, repo, state)
+_finishhistedit(ui, repo, state, fm)
+fm.end()
 
 def _continuehistedit(ui, repo, state):
 """This function runs after either:
@@ -1162,7 +1166,7 @@
 state.write()
 ui.progress(_("editing"), None)
 
-def _finishhistedit(ui, repo, state):
+def _finishhistedit(ui, repo, state, fm):
 """This action runs when histedit is finishing its session"""
 repo.ui.pushbuffer()
 hg.update(repo, state.parentctxnode, quietempty=True)
@@ -1196,6 +1200,12 @@
 mapping = {k: v for k, v in mapping.items()
if k in nodemap and all(n in nodemap for n in v)}
 scmutil.cleanupnodes(repo, mapping, 'histedit')
+hf = fm.hexfunc
+fl = fm.formatlist
+fd = fm.formatdict
+nodechanges = fd({hf(oldn): fl([hf(n) for n in newn], 'succ')
+  for oldn, newn in mapping.iteritems()})
+fm.data(nodechanges=nodechanges)
 
 state.clear()
 if os.path.exists(repo.sjoin('undo')):



To: pulkit, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1270: help: adding a topic on flags

2017-11-08 Thread martinvonz (Martin von Zweigbergk) 
martinvonz added inline comments.

INLINE COMMENTS

> flags.txt:8
> +Additionally, some flags, such as --repository, are global and can be used 
> with
> +any command - those are seen in :hg:`help -v`, and can be specified.
> +

The final "and can be specified" sounds a little truncated. Was there supposed 
to be something after? Or maybe it's no longer needed since you already said 
"can be used with any command"?

> flags.txt:36-39
> +--flagname=foo
> +--flagname foo
> +-f=foo
> +-f foo

`-ffoo` is also valid

> flags.txt:67
> +[alias]
> +commit = commit --interactive
> +

Do we recommend overriding the command like this or should the left side be 
called something else (maybe "icommit")? I really don't know what we recommend 
here, so don't take my question to imply that you shouldn't do it the way you 
have done it.

> flags.txt:86
> +
> +hg commit -m "Used value"
> +

should be `committemp` here

> flags.txt:94
> +as described above.
> +Except for list flags, defaults can be overridden on the command line 
> simplify
> +by specifying the flag in that location.

s/simplify/simply/

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1270

To: rdamazio, #hg-reviewers
Cc: av6, dlax, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1347: dirstate: make map implementation overridable

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Other implementations of dirstate will want to replace the class used for the
  dirstate map.  Allow this to happen by making the class an attribute of the
  dirstate.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1347

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -80,6 +80,7 @@
 self._plchangecallbacks = {}
 self._origpl = None
 self._updatedfiles = set()
+self._mapcls = dirstatemap
 
 @contextlib.contextmanager
 def parentchange(self):
@@ -128,7 +129,7 @@
 @propertycache
 def _map(self):
 '''Returns the dirstate map.'''
-self._map = dirstatemap(self._ui, self._opener, self._root)
+self._map = self._mapcls(self._ui, self._opener, self._root)
 return self._map
 
 @property



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1344: dirstate: remove _droppath method

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This method now doesn't do very much interesting and can be removed.  The
  updated files set can be updated where _droppath was originally called.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1344

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -386,9 +386,6 @@
 def copies(self):
 return self._map.copymap
 
-def _droppath(self, f):
-self._updatedfiles.add(f)
-
 def _addpath(self, f, state, mode, size, mtime):
 oldstate = self[f]
 if state == 'a' or oldstate == 'r':
@@ -465,7 +462,6 @@
 def remove(self, f):
 '''Mark a file removed.'''
 self._dirty = True
-self._droppath(f)
 oldstate = self[f]
 size = 0
 if self._pl[1] != nullid:
@@ -477,6 +473,7 @@
 elif entry[0] == 'n' and entry[2] == -2: # other parent
 size = -2
 self._map.otherparentset.add(f)
+self._updatedfiles.add(f)
 self._map.removefile(f, oldstate, size)
 if size == 0:
 self._map.copymap.pop(f, None)
@@ -492,7 +489,7 @@
 oldstate = self[f]
 if self._map.dropfile(f, oldstate):
 self._dirty = True
-self._droppath(f)
+self._updatedfiles.add(f)
 self._map.copymap.pop(f, None)
 
 def _discoverpath(self, path, normed, ignoremissing, exists, storemap):



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1345: dirstate: add explicit methods for querying directories

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The set-like object returned by dirstate.dirs may be difficult for other
  implementations of the dirstate to provide, and is unnecessary as it is
  only ever used for __contains__.  Instead, provide an explicit method for
  testing for a directory.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1345

AFFECTED FILES
  contrib/perf.py
  hgext/largefiles/reposetup.py
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -157,8 +157,8 @@
 def _pl(self):
 return self._map.parents()
 
-def dirs(self):
-return self._map.dirs
+def hasdir(self, d):
+return self._map.hastrackeddir(d)
 
 @rootcache('.hgignore')
 def _ignore(self):
@@ -390,11 +390,11 @@
 oldstate = self[f]
 if state == 'a' or oldstate == 'r':
 scmutil.checkfilename(f)
-if f in self._map.dirs:
+if self._map.hastrackeddir(f):
 raise error.Abort(_('directory %r already in dirstate') % f)
 # shadows
 for d in util.finddirs(f):
-if d in self._map.dirs:
+if self._map.hastrackeddir(d):
 break
 entry = self._map.get(d)
 if entry is not None and entry[0] != 'r':
@@ -770,7 +770,6 @@
 results = dict.fromkeys(subrepos)
 results['.hg'] = None
 
-alldirs = None
 for ff in files:
 # constructing the foldmap is expensive, so don't do it for the
 # common case where files is ['.']
@@ -801,9 +800,7 @@
 if nf in dmap: # does it exactly match a missing file?
 results[nf] = None
 else: # does it match a missing directory?
-if alldirs is None:
-alldirs = util.dirs(dmap._map)
-if nf in alldirs:
+if self._map.hasdir(nf):
 if matchedir:
 matchedir(nf)
 notfoundadd(nf)
@@ -1199,7 +1196,8 @@
 self._map.clear()
 self.copymap.clear()
 self.setparents(nullid, nullid)
-util.clearcachedproperty(self, "dirs")
+util.clearcachedproperty(self, "_dirs")
+util.clearcachedproperty(self, "_alldirs")
 util.clearcachedproperty(self, "filefoldmap")
 util.clearcachedproperty(self, "dirfoldmap")
 util.clearcachedproperty(self, "nonnormalset")
@@ -1232,8 +1230,10 @@
 
 def addfile(self, f, oldstate, state, mode, size, mtime):
 """Add a tracked file to the dirstate."""
-if oldstate in "?r" and "dirs" in self.__dict__:
-self.dirs.addpath(f)
+if oldstate in "?r" and "_dirs" in self.__dict__:
+self._dirs.addpath(f)
+if oldstate == "?" and "_alldirs" in self.__dict__:
+self._alldirs.addpath(f)
 self._map[f] = dirstatetuple(state, mode, size, mtime)
 if state != 'n' or mtime == -1:
 self.nonnormalset.add(f)
@@ -1248,8 +1248,10 @@
 the file's previous state.  In the future, we should refactor this
 to be more explicit about what that state is.
 """
-if oldstate not in "?r" and "dirs" in self.__dict__:
-self.dirs.delpath(f)
+if oldstate not in "?r" and "_dirs" in self.__dict__:
+self._dirs.delpath(f)
+if oldstate == "?" and "_alldirs" in self.__dict__:
+self._alldirs.addpath(f)
 if "filefoldmap" in self.__dict__:
 normed = util.normcase(f)
 self.filefoldmap.pop(normed, None)
@@ -1263,8 +1265,10 @@
 """
 exists = f in self._map
 if exists:
-if oldstate != "r" and "dirs" in self.__dict__:
-self.dirs.delpath(f)
+if oldstate != "r" and "_dirs" in self.__dict__:
+self._dirs.delpath(f)
+if "_alldirs" in self.__dict__:
+self._alldirs.delpath(f)
 del self._map[f]
 if "filefoldmap" in self.__dict__:
 normed = util.normcase(f)
@@ -1314,13 +1318,28 @@
 f['.'] = '.' # prevents useless util.fspath() invocation
 return f
 
+def hastrackeddir(self, d):
+"""
+Returns True if the dirstate contains a tracked (not removed) file
+in this directory.
+"""
+return d in self._dirs
+
+def hasdir(self, d):
+"""
+Returns True if the dirstate contains a file (tracked or removed)
+in this directory.
+"""
+return d in self._alldirs
+
 @propertycache
-def dirs(self):
-"""Returns a set-like object containing all the directories in the
-current

D1346: fsmonitor: only access inner dirstate map if it is available

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As part of the dirstate refactor, fsmonitor was updated to directly access the
  inner map of the dirstatemap object.
  
  Dirstatemap reimplementations may not use a map like this, so only access it 
if
  it is there.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1346

AFFECTED FILES
  hgext/fsmonitor/__init__.py

CHANGE DETAILS

diff --git a/hgext/fsmonitor/__init__.py b/hgext/fsmonitor/__init__.py
--- a/hgext/fsmonitor/__init__.py
+++ b/hgext/fsmonitor/__init__.py
@@ -273,7 +273,11 @@
 
 matchfn = match.matchfn
 matchalways = match.always()
-dmap = self._map._map
+dmap = self._map
+if util.safehasattr(dmap, '_map'):
+# directly access the inner dirstate map if the standard dirstate
+# implementation is in use.
+dmap = dmap._map
 nonnormalset = self._map.nonnormalset
 
 copymap = self._map.copymap



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1342: dirstate: move management of the dirstate dirs into the dirstatemap

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The dirstate dirs object is owned by the map, so move management of that 
object
  there.  A future implementation of the dirstate will manage the dirs object
  differently.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1342

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -387,9 +387,6 @@
 return self._map.copymap
 
 def _droppath(self, f):
-if self[f] not in "?r" and "dirs" in self._map.__dict__:
-self._map.dirs.delpath(f)
-
 if "filefoldmap" in self._map.__dict__:
 normed = util.normcase(f)
 if normed in self._map.filefoldmap:
@@ -411,11 +408,9 @@
 if entry is not None and entry[0] != 'r':
 raise error.Abort(
 _('file %r in dirstate clashes with %r') % (d, f))
-if oldstate in "?r" and "dirs" in self._map.__dict__:
-self._map.dirs.addpath(f)
 self._dirty = True
 self._updatedfiles.add(f)
-self._map.addfile(f, state, mode, size, mtime)
+self._map.addfile(f, oldstate, state, mode, size, mtime)
 
 def normal(self, f):
 '''Mark a file normal and clean.'''
@@ -476,6 +471,7 @@
 '''Mark a file removed.'''
 self._dirty = True
 self._droppath(f)
+oldstate = self[f]
 size = 0
 if self._pl[1] != nullid:
 entry = self._map.get(f)
@@ -486,7 +482,7 @@
 elif entry[0] == 'n' and entry[2] == -2: # other parent
 size = -2
 self._map.otherparentset.add(f)
-self._map.removefile(f, size)
+self._map.removefile(f, oldstate, size)
 if size == 0:
 self._map.copymap.pop(f, None)
 
@@ -498,7 +494,8 @@
 
 def drop(self, f):
 '''Drop a file from the dirstate'''
-if self._map.dropfile(f):
+oldstate = self[f]
+if self._map.dropfile(f, oldstate):
 self._dirty = True
 self._droppath(f)
 self._map.copymap.pop(f, None)
@@ -1241,32 +1238,38 @@
 """Loads the underlying data, if it's not already loaded"""
 self._map
 
-def addfile(self, f, state, mode, size, mtime):
+def addfile(self, f, oldstate, state, mode, size, mtime):
 """Add a tracked file to the dirstate."""
+if oldstate in "?r" and "dirs" in self.__dict__:
+self.dirs.addpath(f)
 self._map[f] = dirstatetuple(state, mode, size, mtime)
 if state != 'n' or mtime == -1:
 self.nonnormalset.add(f)
 if size == -2:
 self.otherparentset.add(f)
 
-def removefile(self, f, size):
+def removefile(self, f, oldstate, size):
 """
 Mark a file as removed in the dirstate.
 
 The `size` parameter is used to store sentinel values that indicate
 the file's previous state.  In the future, we should refactor this
 to be more explicit about what that state is.
 """
+if oldstate not in "?r" and "dirs" in self.__dict__:
+self.dirs.delpath(f)
 self._map[f] = dirstatetuple('r', 0, size, 0)
 self.nonnormalset.add(f)
 
-def dropfile(self, f):
+def dropfile(self, f, oldstate):
 """
 Remove a file from the dirstate.  Returns True if the file was
 previously recorded.
 """
 exists = f in self._map
 if exists:
+if oldstate != "r" and "dirs" in self.__dict__:
+self.dirs.delpath(f)
 del self._map[f]
 self.nonnormalset.discard(f)
 return exists



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1339: dirstate: don't remove normallookup files from nonnormalset

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The dirstate typically tries to keep the nonnormalset and otherparentset
  up-to-date when making changes to the dirstate.  In the case of files marked
  normallookup, however, it erroneously removes the file from the nonnormalset,
  after _addpath had just added it.
  
  This doesn't seem to matter at the moment, as nothing relies on the
  nonnormalset being correct at this point, however a future re-implementations
  of the dirstate map will require this to be kept up-to-date correctly.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1339

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -458,8 +458,6 @@
 return
 self._addpath(f, 'n', 0, -1, -1)
 self._map.copymap.pop(f, None)
-if f in self._map.nonnormalset:
-self._map.nonnormalset.remove(f)
 
 def otherparent(self, f):
 '''Mark as coming from the other parent, always dirty.'''



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1343: dirstate: move dropping of folded filenames into the dirstate map

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  When dropping files from the dirstate, the corresponding entry in the
  filefoldmap is also dropped.  Move this into the dirstate map object.  A 
future
  implementation of the dirstate will maintain the filefoldmap differently.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1343

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -387,11 +387,6 @@
 return self._map.copymap
 
 def _droppath(self, f):
-if "filefoldmap" in self._map.__dict__:
-normed = util.normcase(f)
-if normed in self._map.filefoldmap:
-del self._map.filefoldmap[normed]
-
 self._updatedfiles.add(f)
 
 def _addpath(self, f, state, mode, size, mtime):
@@ -1258,6 +1253,9 @@
 """
 if oldstate not in "?r" and "dirs" in self.__dict__:
 self.dirs.delpath(f)
+if "filefoldmap" in self.__dict__:
+normed = util.normcase(f)
+self.filefoldmap.pop(normed, None)
 self._map[f] = dirstatetuple('r', 0, size, 0)
 self.nonnormalset.add(f)
 
@@ -1271,6 +1269,9 @@
 if oldstate != "r" and "dirs" in self.__dict__:
 self.dirs.delpath(f)
 del self._map[f]
+if "filefoldmap" in self.__dict__:
+normed = util.normcase(f)
+self.filefoldmap.pop(normed, None)
 self.nonnormalset.discard(f)
 return exists
 



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1341: dirstate: move management of nonnormal sets into dirstate map

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The dirstate map owns the nonnormal sets, and so should be the class to update
  them.  A future implementation of the dirstate will manage these maps
  differently.
  
  The action of clearing ambiguous times is now entirely controlled by the
  dirstate map, so it moves there too.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1341

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -415,10 +415,6 @@
 self._map.dirs.addpath(f)
 self._dirty = True
 self._updatedfiles.add(f)
-if state != 'n' or mtime == -1:
-self._map.nonnormalset.add(f)
-if size == -2:
-self._map.otherparentset.add(f)
 self._map.addfile(f, state, mode, size, mtime)
 
 def normal(self, f):
@@ -490,7 +486,6 @@
 elif entry[0] == 'n' and entry[2] == -2: # other parent
 size = -2
 self._map.otherparentset.add(f)
-self._map.nonnormalset.add(f)
 self._map.removefile(f, size)
 if size == 0:
 self._map.copymap.pop(f, None)
@@ -506,8 +501,6 @@
 if self._map.dropfile(f):
 self._dirty = True
 self._droppath(f)
-if f in self._map.nonnormalset:
-self._map.nonnormalset.remove(f)
 self._map.copymap.pop(f, None)
 
 def _discoverpath(self, path, normed, ignoremissing, exists, storemap):
@@ -631,12 +624,7 @@
 
 # emulate dropping timestamp in 'parsers.pack_dirstate'
 now = _getfsnow(self._opener)
-dmap = self._map
-for f in self._updatedfiles:
-e = dmap.get(f)
-if e is not None and e[0] == 'n' and e[3] == now:
-dmap.addfile(f, e[0], e[1], e[2], -1)
-self._map.nonnormalset.add(f)
+self._map.clearambiguoustimes(self._updatedfiles, now)
 
 # emulate that all 'dirstate.normal' results are written out
 self._lastnormaltime = 0
@@ -1256,6 +1244,10 @@
 def addfile(self, f, state, mode, size, mtime):
 """Add a tracked file to the dirstate."""
 self._map[f] = dirstatetuple(state, mode, size, mtime)
+if state != 'n' or mtime == -1:
+self.nonnormalset.add(f)
+if size == -2:
+self.otherparentset.add(f)
 
 def removefile(self, f, size):
 """
@@ -1266,6 +1258,7 @@
 to be more explicit about what that state is.
 """
 self._map[f] = dirstatetuple('r', 0, size, 0)
+self.nonnormalset.add(f)
 
 def dropfile(self, f):
 """
@@ -1275,8 +1268,16 @@
 exists = f in self._map
 if exists:
 del self._map[f]
+self.nonnormalset.discard(f)
 return exists
 
+def clearambiguoustimes(self, files, now):
+for f in files:
+e = self.get(f)
+if e is not None and e[0] == 'n' and e[3] == now:
+self._map[f] = dirstatetuple(e[0], e[1], e[2], -1)
+self.nonnormalset.add(f)
+
 def nonnormalentries(self):
 '''Compute the nonnormal dirstate entries from the dmap'''
 try:



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1337: util: add util.clearcachedproperty

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This utility function allows clearing of the cached value set up
  @propertycache, if there is one.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1337

AFFECTED FILES
  mercurial/util.py

CHANGE DETAILS

diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -931,6 +931,11 @@
 # __dict__ assignment required to bypass __setattr__ (eg: repoview)
 obj.__dict__[self.name] = value
 
+def clearcachedproperty(obj, prop):
+'''clear a cached property value, if one has been set'''
+if prop in obj.__dict__:
+del obj.__dict__[prop]
+
 def pipefilter(s, cmd):
 '''filter string S through command CMD, returning its output'''
 p = subprocess.Popen(cmd, shell=True, close_fds=closefds,



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1340: dirstate: add explicit methods for modifying dirstate

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Instead of assigning dirstatetuple objects to entries in the dirstate, move
  responsibility for creating tuples into the dirstatemap.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1340

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -127,8 +127,7 @@
 
 @propertycache
 def _map(self):
-'''Return the dirstate contents as a map from filename to
-(state, mode, size, time).'''
+'''Returns the dirstate map.'''
 self._map = dirstatemap(self._ui, self._opener, self._root)
 return self._map
 
@@ -416,11 +415,11 @@
 self._map.dirs.addpath(f)
 self._dirty = True
 self._updatedfiles.add(f)
-self._map[f] = dirstatetuple(state, mode, size, mtime)
 if state != 'n' or mtime == -1:
 self._map.nonnormalset.add(f)
 if size == -2:
 self._map.otherparentset.add(f)
+self._map.addfile(f, state, mode, size, mtime)
 
 def normal(self, f):
 '''Mark a file normal and clean.'''
@@ -491,8 +490,8 @@
 elif entry[0] == 'n' and entry[2] == -2: # other parent
 size = -2
 self._map.otherparentset.add(f)
-self._map[f] = dirstatetuple('r', 0, size, 0)
 self._map.nonnormalset.add(f)
+self._map.removefile(f, size)
 if size == 0:
 self._map.copymap.pop(f, None)
 
@@ -504,10 +503,9 @@
 
 def drop(self, f):
 '''Drop a file from the dirstate'''
-if f in self._map:
+if self._map.dropfile(f):
 self._dirty = True
 self._droppath(f)
-del self._map[f]
 if f in self._map.nonnormalset:
 self._map.nonnormalset.remove(f)
 self._map.copymap.pop(f, None)
@@ -637,7 +635,7 @@
 for f in self._updatedfiles:
 e = dmap.get(f)
 if e is not None and e[0] == 'n' and e[3] == now:
-dmap[f] = dirstatetuple(e[0], e[1], e[2], -1)
+dmap.addfile(f, e[0], e[1], e[2], -1)
 self._map.nonnormalset.add(f)
 
 # emulate that all 'dirstate.normal' results are written out
@@ -1245,22 +1243,40 @@
 def __contains__(self, key):
 return key in self._map
 
-def __setitem__(self, key, value):
-self._map[key] = value
-
 def __getitem__(self, key):
 return self._map[key]
 
-def __delitem__(self, key):
-del self._map[key]
-
 def keys(self):
 return self._map.keys()
 
 def preload(self):
 """Loads the underlying data, if it's not already loaded"""
 self._map
 
+def addfile(self, f, state, mode, size, mtime):
+"""Add a tracked file to the dirstate."""
+self._map[f] = dirstatetuple(state, mode, size, mtime)
+
+def removefile(self, f, size):
+"""
+Mark a file as removed in the dirstate.
+
+The `size` parameter is used to store sentinel values that indicate
+the file's previous state.  In the future, we should refactor this
+to be more explicit about what that state is.
+"""
+self._map[f] = dirstatetuple('r', 0, size, 0)
+
+def dropfile(self, f):
+"""
+Remove a file from the dirstate.  Returns True if the file was
+previously recorded.
+"""
+exists = f in self._map
+if exists:
+del self._map[f]
+return exists
+
 def nonnormalentries(self):
 '''Compute the nonnormal dirstate entries from the dmap'''
 try:
@@ -1390,8 +1406,6 @@
 # Avoid excess attribute lookups by fast pathing certain checks
 self.__contains__ = self._map.__contains__
 self.__getitem__ = self._map.__getitem__
-self.__setitem__ = self._map.__setitem__
-self.__delitem__ = self._map.__delitem__
 self.get = self._map.get
 
 def write(self, st, now):



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1338: dirstate: clear map cached properties when clearing the map

2017-11-08 Thread mbthomas (Mark Thomas) 
mbthomas created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  dirstatemap.clear should remove all record of the files in the map.  This
  includes removing caches of values derived from these.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1338

AFFECTED FILES
  mercurial/dirstate.py

CHANGE DETAILS

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -1226,6 +1226,11 @@
 self._map.clear()
 self.copymap.clear()
 self.setparents(nullid, nullid)
+util.clearcachedproperty(self, "dirs")
+util.clearcachedproperty(self, "filefoldmap")
+util.clearcachedproperty(self, "dirfoldmap")
+util.clearcachedproperty(self, "nonnormalset")
+util.clearcachedproperty(self, "otherparentset")
 
 def iteritems(self):
 return self._map.iteritems()



To: mbthomas, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1293: rebase: use fm.formatlist() and fm.formatdict() to support user template

2017-11-08 Thread pulkit (Pulkit Goyal) 
pulkit added a comment.


  In https://phab.mercurial-scm.org/D1293#21761, @yuja wrote:
  
  > Can you add a test that should be fixed by this patch?
  >  Perhaps that will show we'll probably want to set the name of dict item 
pair
  >  other than the default 'key'/'value'.
  
  
  Sorry but I am unable to understand the test part. Can you point me to an 
example?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1293

To: pulkit, #hg-reviewers, yuja
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1270: help: adding a topic on flags

2017-11-08 Thread rdamazio (Rodrigo Damazio Bovendorp) 
rdamazio updated this revision to Diff 3334.
rdamazio marked 4 inline comments as done.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D1270?vs=3168&id=3334

REVISION DETAIL
  https://phab.mercurial-scm.org/D1270

AFFECTED FILES
  contrib/wix/help.wxs
  mercurial/help.py
  mercurial/help/flags.txt
  tests/test-globalopts.t
  tests/test-help.t
  tests/test-hgweb-json.t

CHANGE DETAILS

diff --git a/tests/test-hgweb-json.t b/tests/test-hgweb-json.t
--- a/tests/test-hgweb-json.t
+++ b/tests/test-hgweb-json.t
@@ -1581,6 +1581,10 @@
 "topic": "filesets"
   },
   {
+"summary": "Command-line flags",
+"topic": "flags"
+  },
+  {
 "summary": "Glossary",
 "topic": "glossary"
   },
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -110,6 +110,7 @@
environment   Environment Variables
extensionsUsing Additional Features
filesets  Specifying File Sets
+   flags Command-line flags
glossary  Glossary
hgignore  Syntax for Mercurial Ignore Files
hgweb Configuring hgweb
@@ -188,6 +189,7 @@
environment   Environment Variables
extensionsUsing Additional Features
filesets  Specifying File Sets
+   flags Command-line flags
glossary  Glossary
hgignore  Syntax for Mercurial Ignore Files
hgweb Configuring hgweb
@@ -865,6 +867,7 @@
environment   Environment Variables
extensionsUsing Additional Features
filesets  Specifying File Sets
+   flags Command-line flags
glossary  Glossary
hgignore  Syntax for Mercurial Ignore Files
hgweb Configuring hgweb
@@ -2013,6 +2016,13 @@
   Specifying File Sets
   
   
+  
+  flags
+  
+  
+  Command-line flags
+  
+  
   
   glossary
   
diff --git a/tests/test-globalopts.t b/tests/test-globalopts.t
--- a/tests/test-globalopts.t
+++ b/tests/test-globalopts.t
@@ -355,6 +355,7 @@
environment   Environment Variables
extensionsUsing Additional Features
filesets  Specifying File Sets
+   flags Command-line flags
glossary  Glossary
hgignore  Syntax for Mercurial Ignore Files
hgweb Configuring hgweb
@@ -439,6 +440,7 @@
environment   Environment Variables
extensionsUsing Additional Features
filesets  Specifying File Sets
+   flags Command-line flags
glossary  Glossary
hgignore  Syntax for Mercurial Ignore Files
hgweb Configuring hgweb
diff --git a/mercurial/help/flags.txt b/mercurial/help/flags.txt
new file mode 100644
--- /dev/null
+++ b/mercurial/help/flags.txt
@@ -0,0 +1,105 @@
+Most Mercurial commands accept various flags.
+
+Flag names
+==
+
+Flags for each command are listed in :hg:`help` for that command.
+Additionally, some flags, such as --repository, are global and can be used with
+any command - those are seen in :hg:`help -v`, and can be specified.
+
+Every flag has at least a long name, such as --repository. Some flags may also
+have a short one-letter name, such as the equivalent -R. Using the short or 
long
+name is equivalent and has the same effect.
+
+Flags that have a short name can also be bundled together - for instance, to
+specify both --edit (short -e) and --interactive (short -i), one could use::
+
+hg commit -ei
+
+If any of the bundled flags takes a value (i.e. is not a boolean), it must be
+last, followed by the value::
+
+hg commit -im 'Message'
+
+Flag types
+==
+
+Mercurial command-line flags can be strings, numbers, booleans, or lists of
+strings.
+
+Specifying flag values
+==
+
+The following syntaxes are allowed, assuming a flag 'flagname' with short name
+'f'::
+
+--flagname=foo
+--flagname foo
+-f=foo
+-f foo
+
+This syntax applies to all non-boolean flags (strings, numbers or lists).
+
+Specifying boolean flags
+
+
+Boolean flags do not take a value parameter. To specify a boolean, use the flag
+name to set it to true, or the same name prefixed with 'no-' to set it to
+false::
+
+hg commit --interactive
+hg commit --no-interactive
+
+Specifying list flags
+=
+
+List flags take multiple values. To specify them, pass the flag multiple 
times::
+
+hg files --include mercurial --include tests
+
+Setting flag defaults
+=
+
+In order to set a default value for a flag in an hgrc file, it is recommended 
to
+use aliases::
+
+[alias]
+commit = commit --interactive
+
+For more information on hgrc files, see :hg:`help config`.
+
+Overriding flags on the command line
+
+
+If the same non-list flag is specified multiple times on the command line, the
+latest specification is used::
+
+hg commit -m "Ignored value" -m "Used value"
+
+This includes the use of aliases - e.g., if one

D1270: help: adding a topic on flags

2017-11-08 Thread rdamazio (Rodrigo Damazio Bovendorp) 
rdamazio added a comment.


  In https://phab.mercurial-scm.org/D1270#21058, @martinvonz wrote:
  
  > We're in code freeze for approximately two more days, so this will have to 
wait until ~Thursday. Still, thanks for the patch. I'm happy to include 
something like this. Things that struck me as missing:
  >
  > 1. Long vs short form (that e.g. "--verbose" and "-v" are equivalent if the 
refer to the same flag, which they do)
  > 2. Non-list, non-boolean flags that take arguments, such as "--rev", and 
that "--rev=tip", "--rev tip", "-r=tip", "-r tip", and "-rtip" are equivalent
  > 3. Short flags can be combined: "-pvr tip" is valid
  > 4. Later flags override earlier flags
  >
  >   We don't necessarily have to include all that in the first patch.
  
  
  Tried to address all of these.

INLINE COMMENTS

> av6 wrote in flags.txt:7
> This "lists of strings" caught my attention, but it took me some time to 
> figure out what's the matter. I don't feel strongly, but here are some points:
> 
> 1. it's not clear what a "list of strings" is and how to specify it: is it a 
> comma-separated list?
> 2. it's not a type that //one// command-line flag can take (i.e. no 
> `--flag=foo,bar` or anything), you must use multiple flags on command line to 
> build this list, so it's unlike the 3 previous types
> 3. I don't think we have anywhere (in the code) a list of integers or 
> booleans right now, but nothing prevents this in future, or in an extension
> 
> I think this data type is a result of looking at the internals that allow 
> some python variables (associated with handling command line flags in the 
> code) to be lists, but because our CLI is not python, this phrase is more 
> distracting than helpful. I think taking the note that's in `hg help` and 
> explaining it is better: "[+] can be repeated" (essentially merging 
> "​Specifying list flags" and "Overriding flag defaults").

That's the whole point of the "Specifying list flags" section below, and in 
fact what motivated me to send this patch.
I don't think the code supports parsing a list as numbers or booleans, right 
now.

> av6 wrote in flags.txt:35
> This is a weird-looking macro. Does it work when rendered into e.g. HTML (can 
> be seen hgweb)? I've only seen this format:
> 
>   :hg:`help config`

Assume that I don't know what I'm doing, as far as the format of this file is 
concerned :) I just tried hgweb and fixed a few things.

> av6 wrote in flags.txt:49
> There are also global flags, they are hidden by default too. I guess you can 
> put them into these 3 categories, but I don't think of --quiet or --verbose 
> as advanced (much less experimental or deprecated). Just a nitpick.

--verbose is used to show those flags in hg help, it's not that that flag 
itself is advanced.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1270

To: rdamazio, #hg-reviewers
Cc: av6, dlax, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Storage format for remotenames.

2017-11-08 Thread Yuya Nishihara 
On Tue, 7 Nov 2017 09:58:04 -0800, Durham Goode wrote:
> I wish we had some easily reusable serializer/deserializer instead of 
> having to reinvent these every time.  What's our reasoning for not using 
> json? I forget. If there are some weird characters, like control 
> characters or something, that break json, I'd say we just use json and 
> prevent users from creating bookmarks and paths with those names.

Just about json. Using json (in Mercurial) is bad because it's so easily
to spill out unicode objects without noticing that, all tests pass (because
the whole data is ascii), and we'll get nice UnicodeError in production.

Another concern is that encoding conversion can be lossy even if it goes
with no error. There are n:m mappings between unicode and legacy encoding.
For example, we have three major Shift_JISes in Japan, and the Microsoft one
allocates multiple code points for one character for "compatibility" reasons.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 6 STABLE] amend: do not drop missing files (issue5732)

2017-11-08 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1509806899 -32400
#  Sat Nov 04 23:48:19 2017 +0900
# Branch stable
# Node ID 53b507cba99d5fe71a5a018cba1c0ab67d64ce79
# Parent  bbdfaa3d4dc9ca3293065cfdaa611e95e6aa8cc6
amend: do not drop missing files (issue5732)

samefile() can be wrong since wctx.manifest() does not include missing files,
whereas missing files should be preserved on commit.

diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -3166,9 +3166,11 @@ def amend(ui, repo, old, extra, pats, op
 # introduced file X and the file was renamed in the working
 # copy, then those two files are the same and
 # we can discard X from our list of files. Likewise if X
-# was deleted, it's no longer relevant
+# was removed, it's no longer relevant. If X is missing (aka
+# deleted), old X must be preserved.
 files.update(filestoamend)
-files = [f for f in files if not samefile(f, wctx, base)]
+files = [f for f in files if (not samefile(f, wctx, base)
+  or f in wctx.deleted())]
 
 def filectxfn(repo, ctx_, path):
 try:
diff --git a/tests/test-amend.t b/tests/test-amend.t
--- a/tests/test-amend.t
+++ b/tests/test-amend.t
@@ -268,8 +268,6 @@ amend r1 to include wc changes
 
 clean/modified/removed/added states of the amended revision
 
-BROKEN: missing files are removed
-
   $ hg status --all --change . 'glob:content1_*_content1-tracked'
   C content1_content1_content1-tracked
   C content1_content2_content1-tracked
@@ -297,7 +295,7 @@ BROKEN: missing files are removed
   $ hg status --all --change . 'glob:missing_content2_*'
   A missing_content2_content2-tracked
   A missing_content2_content3-tracked
-  A missing_content2_missing-tracked (false !)
+  A missing_content2_missing-tracked
   $ hg status --all --change . 'glob:missing_missing_*'
   A missing_missing_content3-tracked
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 6 STABLE] subrepo: extract preprocess of repo.commit() to free function

2017-11-08 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1509790899 -32400
#  Sat Nov 04 19:21:39 2017 +0900
# Branch stable
# Node ID 34c95aad895f0a96e3becbdfec61a073fbe759f3
# Parent  4edd1c6048fac782a8606d8b8e22c24a6208de7a
subrepo: extract preprocess of repo.commit() to free function

No code change other than extracting a function. Maybe we should stop mutating
the status argument, but that's out of the scope of stable changes.

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1845,58 +1845,8 @@ class localrepository(object):
 status.modified.extend(status.clean) # mq may commit clean 
files
 
 # check subrepos
-subs = []
-commitsubs = set()
-newstate = wctx.substate.copy()
-# only manage subrepos and .hgsubstate if .hgsub is present
-if '.hgsub' in wctx:
-# we'll decide whether to track this ourselves, thanks
-for c in status.modified, status.added, status.removed:
-if '.hgsubstate' in c:
-c.remove('.hgsubstate')
-
-# compare current state to last committed state
-# build new substate based on last committed state
-oldstate = wctx.p1().substate
-for s in sorted(newstate.keys()):
-if not match(s):
-# ignore working copy, use old state if present
-if s in oldstate:
-newstate[s] = oldstate[s]
-continue
-if not force:
-raise error.Abort(
-_("commit with new subrepo %s excluded") % s)
-dirtyreason = wctx.sub(s).dirtyreason(True)
-if dirtyreason:
-if not self.ui.configbool('ui', 'commitsubrepos'):
-raise error.Abort(dirtyreason,
-hint=_("use --subrepos for recursive commit"))
-subs.append(s)
-commitsubs.add(s)
-else:
-bs = wctx.sub(s).basestate()
-newstate[s] = (newstate[s][0], bs, newstate[s][2])
-if oldstate.get(s, (None, None, None))[1] != bs:
-subs.append(s)
-
-# check for removed subrepos
-for p in wctx.parents():
-r = [s for s in p.substate if s not in newstate]
-subs += [s for s in r if match(s)]
-if subs:
-if (not match('.hgsub') and
-'.hgsub' in (wctx.modified() + wctx.added())):
-raise error.Abort(
-_("can't commit subrepos without .hgsub"))
-status.modified.insert(0, '.hgsubstate')
-
-elif '.hgsub' in status.removed:
-# clean up .hgsubstate when .hgsub is removed
-if ('.hgsubstate' in wctx and
-'.hgsubstate' not in (status.modified + status.added +
-  status.removed)):
-status.removed.insert(0, '.hgsubstate')
+subs, commitsubs, newstate = subrepo.precommit(
+self.ui, wctx, status, match, force=force)
 
 # make sure all explicit patterns are matched
 if not force:
diff --git a/mercurial/subrepo.py b/mercurial/subrepo.py
--- a/mercurial/subrepo.py
+++ b/mercurial/subrepo.py
@@ -293,6 +293,71 @@ def submerge(repo, wctx, mctx, actx, ove
 writestate(repo, sm)
 return sm
 
+def precommit(ui, wctx, status, match, force=False):
+"""Calculate .hgsubstate changes that should be applied before committing
+
+Returns (subs, commitsubs, newstate) where
+- subs: changed subrepos (including dirty ones)
+- commitsubs: dirty subrepos which the caller needs to commit recursively
+- newstate: new state dict which the caller must write to .hgsubstate
+
+This also updates the given status argument.
+"""
+subs = []
+commitsubs = set()
+newstate = wctx.substate.copy()
+
+# only manage subrepos and .hgsubstate if .hgsub is present
+if '.hgsub' in wctx:
+# we'll decide whether to track this ourselves, thanks
+for c in status.modified, status.added, status.removed:
+if '.hgsubstate' in c:
+c.remove('.hgsubstate')
+
+# compare current state to last committed state
+# build new substate based on last committed state
+oldstate = wctx.p1().substate
+for s in sorted(newstate.keys()):
+if not match(s):
+# ignore working copy, use old state if present
+if s in oldstate:
+newstate

[PATCH 6 of 6 STABLE] amend: update .hgsubstate before committing a memctx (issue5677)

2017-11-08 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1509793660 -32400
#  Sat Nov 04 20:07:40 2017 +0900
# Branch stable
# Node ID 17f13d94fbfb4f59bdae5e7a0deb04c3314a992e
# Parent  34c95aad895f0a96e3becbdfec61a073fbe759f3
amend: update .hgsubstate before committing a memctx (issue5677)

This is a minimal copy of localrepo.commit(). As the current amend() function
heavily depends on the wctx API, it wasn't easy to port it to use a separate
status tuple. So for now, wctx._status is updated in-place.

diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -3153,6 +3153,18 @@ def amend(ui, repo, old, extra, pats, op
 raise error.Abort(
 _("failed to mark all new/missing files as added/removed"))
 
+# Check subrepos. This depends on in-place wctx._status update in
+# subrepo.precommit(). To minimize the risk of this hack, we do
+# nothing if .hgsub does not exist.
+if '.hgsub' in wctx or '.hgsub' in old:
+from . import subrepo  # avoid cycle: cmdutil -> subrepo -> cmdutil
+subs, commitsubs, newsubstate = subrepo.precommit(
+ui, wctx, wctx._status, matcher)
+# amend should abort if commitsubrepos is enabled
+assert not commitsubs
+if subs:
+subrepo.writestate(repo, newsubstate)
+
 filestoamend = set(f for f in wctx.files() if matcher(f))
 
 changes = (len(filestoamend) > 0)
diff --git a/tests/test-amend-subrepo.t b/tests/test-amend-subrepo.t
--- a/tests/test-amend-subrepo.t
+++ b/tests/test-amend-subrepo.t
@@ -31,28 +31,18 @@ Link first subrepo
 
 amend without .hgsub
 
-BROKEN: should say "can't commit subrepos without .hgsub"
   $ hg amend s
-  nothing changed
-  [1]
+  abort: can't commit subrepos without .hgsub
+  [255]
 
 amend with subrepo
 
-BROKEN: should update .hgsubstate
   $ hg amend
   saved backup bundle to * (glob) (obsstore-off !)
   $ hg status --change .
   A .hgsub
+  A .hgsubstate
   A a
-
-FIX UP .hgsubstate
-
-  $ hg ci -mfix
-  $ hg rollback -q
-  $ hg add .hgsubstate
-  $ hg amend
-  saved backup bundle to * (glob) (obsstore-off !)
-
   $ cat .hgsubstate
    s
 
@@ -69,46 +59,27 @@ amend with dirty subrepo
   $ echo a >> s/a
   $ hg add -R s
   adding s/a
-BROKEN: should say "uncommitted changes in subrepository"
   $ hg amend
-  nothing changed
-  [1]
+  abort: uncommitted changes in subrepository "s"
+  (use --subrepos for recursive commit)
+  [255]
 
 amend with modified subrepo
 
   $ hg ci -R s -m0
-BROKEN: should update .hgsubstate
-  $ hg amend
-  nothing changed
-  [1]
-  $ hg status --change .
-  M a
-
-FIX UP .hgsubstate
-
-  $ hg ci -mfix
-  $ hg rollback -q
   $ hg amend
   saved backup bundle to * (glob) (obsstore-off !)
-
+  $ hg status --change .
+  M .hgsubstate
+  M a
   $ cat .hgsubstate
   f7b1eb17ad24730a1651fccd46c43826d1bbc2ac s
 
 revert subrepo change
 
   $ hg up -R s -q null
-BROKEN: should update .hgsubstate
-  $ hg amend
-  nothing changed
-  [1]
-
-FIX UP .hgsubstate
-
-  $ hg ci -mfix
-  $ hg rollback -q
   $ hg amend
   saved backup bundle to * (glob) (obsstore-off !)
-
   $ hg status --change .
   M a
 
@@ -131,21 +102,13 @@ amend with another subrepo
   $ hg ci -R t -Am0
   adding b
   $ echo 't = t' >> .hgsub
-BROKEN: should update .hgsubstate
   $ hg amend
   saved backup bundle to * (glob) (obsstore-off !)
   $ hg status --change .
   M .hgsub
+  M .hgsubstate
   M a
   A b
-
-FIX UP .hgsubstate
-
-  $ hg ci -mfix
-  $ hg rollback -q
-  $ hg amend
-  saved backup bundle to * (glob) (obsstore-off !)
-
   $ cat .hgsubstate
    s
   bfb1a4fb358498a9533dabf4f2043d94162f1fcd t
@@ -161,23 +124,12 @@ add new commit to be amended
 amend with one subrepo dropped
 
   $ echo 't = t' > .hgsub
-BROKEN: should update .hgsubstate
   $ hg amend
   saved backup bundle to * (glob) (obsstore-off !)
   $ hg status --change .
   M .hgsub
+  M .hgsubstate
   M a
-
-FIX UP .hgsubstate
-
-  $ echo 's = s' > .hgsub
-  $ hg amend -q
-  $ echo 't = t' > .hgsub
-  $ hg ci -mfix
-  $ hg rollback -q
-  $ hg amend
-  saved backup bundle to * (glob) (obsstore-off !)
-
   $ cat .hgsubstate
   bfb1a4fb358498a9533dabf4f2043d94162f1fcd t
 
@@ -192,22 +144,11 @@ add new commit to be amended
 amend with .hgsub removed
 
   $ hg rm .hgsub
-BROKEN: should update .hgsubstate
   $ hg amend
   saved backup bundle to * (glob) (obsstore-off !)
   $ hg status --change .
   M a
   R .hgsub
-
-FIX UP .hgsubstate
-
-  $ hg forget .hgsubstate
-  $ hg amend
-  saved backup bundle to * (glob) (obsstore-off !)
-
-  $ hg status --change .
-  M a
-  R .hgsub
   R .hgsubstate
 
   $ cd ..
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 6 STABLE] amend: do not take untracked files as modified or clean (issue5732)

2017-11-08 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1509806759 -32400
#  Sat Nov 04 23:45:59 2017 +0900
# Branch stable
# Node ID bbdfaa3d4dc9ca3293065cfdaa611e95e6aa8cc6
# Parent  a300140d16436606c8fd73f82dd7c8943de38240
amend: do not take untracked files as modified or clean (issue5732)

fctx.exists() shouldn't be used here as it checks if a file physically exists,
which may disagree with the dirstate.

diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -3180,12 +3180,11 @@ def amend(ui, repo, old, extra, pats, op
 if path not in filestoamend:
 return old.filectx(path)
 
+# Return None for removed files.
+if path in wctx.removed():
+return None
+
 fctx = wctx[path]
-
-# Return None for removed files.
-if not fctx.exists():
-return None
-
 flags = fctx.flags()
 mctx = context.memfilectx(repo,
   fctx.path(), fctx.data(),
diff --git a/tests/test-amend.t b/tests/test-amend.t
--- a/tests/test-amend.t
+++ b/tests/test-amend.t
@@ -268,7 +268,6 @@ amend r1 to include wc changes
 
 clean/modified/removed/added states of the amended revision
 
-BROKEN: untracked files aren't removed
 BROKEN: missing files are removed
 
   $ hg status --all --change . 'glob:content1_*_content1-tracked'
@@ -285,21 +284,16 @@ BROKEN: missing files are removed
   R content1_missing_missing-tracked
   C content1_content1_missing-tracked
   $ hg status --all --change . 'glob:content1_*_*-untracked'
-  M content1_content1_content3-untracked (true !)
-  M content1_content2_content2-untracked (true !)
-  M content1_content2_content3-untracked (true !)
-  R content1_content1_content1-untracked (false !)
-  R content1_content1_content3-untracked (false !)
+  R content1_content1_content1-untracked
+  R content1_content1_content3-untracked
   R content1_content1_missing-untracked
-  R content1_content2_content1-untracked (false !)
-  R content1_content2_content2-untracked (false !)
-  R content1_content2_content3-untracked (false !)
+  R content1_content2_content1-untracked
+  R content1_content2_content2-untracked
+  R content1_content2_content3-untracked
   R content1_content2_missing-untracked
   R content1_missing_content1-untracked
   R content1_missing_content3-untracked
   R content1_missing_missing-untracked
-  C content1_content1_content1-untracked (true !)
-  C content1_content2_content1-untracked (true !)
   $ hg status --all --change . 'glob:missing_content2_*'
   A missing_content2_content2-tracked
   A missing_content2_content3-tracked
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 6 STABLE] tests: add more complete test for status changes on amend

2017-11-08 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1509806426 -32400
#  Sat Nov 04 23:40:26 2017 +0900
# Branch stable
# Node ID a300140d16436606c8fd73f82dd7c8943de38240
# Parent  4cc910cd81d04a9b29c48e0890e06047f21dd8fb
tests: add more complete test for status changes on amend

This demonstrates that missing/untracked files are handled incorrectly. The
correct outputs are suppressed by (false !), and wrong outputs are added with
(true !) instead.

The setup code is copied from test-status-rev.t.

diff --git a/tests/test-amend.t b/tests/test-amend.t
--- a/tests/test-amend.t
+++ b/tests/test-amend.t
@@ -235,3 +235,105 @@ Amend a merge changeset
   |
   o  A
   
+
+More complete test for status changes (issue5732)
+-
+
+Generates history of files having 3 states, r0_r1_wc:
+
+ r0: ground (content/missing)
+ r1: old state to be amended (content/missing, where missing means removed)
+ wc: changes to be included in r1 (content/missing-tracked/untracked)
+
+  $ hg init $TESTTMP/wcstates
+  $ cd $TESTTMP/wcstates
+
+  $ $PYTHON $TESTDIR/generate-working-copy-states.py state 2 1
+  $ hg addremove -q --similarity 0
+  $ hg commit -m0
+
+  $ $PYTHON $TESTDIR/generate-working-copy-states.py state 2 2
+  $ hg addremove -q --similarity 0
+  $ hg commit -m1
+
+  $ $PYTHON $TESTDIR/generate-working-copy-states.py state 2 wc
+  $ hg addremove -q --similarity 0
+  $ hg forget *_*_*-untracked
+  $ rm *_*_missing-*
+
+amend r1 to include wc changes
+
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+
+clean/modified/removed/added states of the amended revision
+
+BROKEN: untracked files aren't removed
+BROKEN: missing files are removed
+
+  $ hg status --all --change . 'glob:content1_*_content1-tracked'
+  C content1_content1_content1-tracked
+  C content1_content2_content1-tracked
+  C content1_missing_content1-tracked
+  $ hg status --all --change . 'glob:content1_*_content[23]-tracked'
+  M content1_content1_content3-tracked
+  M content1_content2_content2-tracked
+  M content1_content2_content3-tracked
+  M content1_missing_content3-tracked
+  $ hg status --all --change . 'glob:content1_*_missing-tracked'
+  M content1_content2_missing-tracked
+  R content1_missing_missing-tracked
+  C content1_content1_missing-tracked
+  $ hg status --all --change . 'glob:content1_*_*-untracked'
+  M content1_content1_content3-untracked (true !)
+  M content1_content2_content2-untracked (true !)
+  M content1_content2_content3-untracked (true !)
+  R content1_content1_content1-untracked (false !)
+  R content1_content1_content3-untracked (false !)
+  R content1_content1_missing-untracked
+  R content1_content2_content1-untracked (false !)
+  R content1_content2_content2-untracked (false !)
+  R content1_content2_content3-untracked (false !)
+  R content1_content2_missing-untracked
+  R content1_missing_content1-untracked
+  R content1_missing_content3-untracked
+  R content1_missing_missing-untracked
+  C content1_content1_content1-untracked (true !)
+  C content1_content2_content1-untracked (true !)
+  $ hg status --all --change . 'glob:missing_content2_*'
+  A missing_content2_content2-tracked
+  A missing_content2_content3-tracked
+  A missing_content2_missing-tracked (false !)
+  $ hg status --all --change . 'glob:missing_missing_*'
+  A missing_missing_content3-tracked
+
+working directory should be all clean (with some missing/untracked files)
+
+  $ hg status --all 'glob:*_content?-tracked'
+  C content1_content1_content1-tracked
+  C content1_content1_content3-tracked
+  C content1_content2_content1-tracked
+  C content1_content2_content2-tracked
+  C content1_content2_content3-tracked
+  C content1_missing_content1-tracked
+  C content1_missing_content3-tracked
+  C missing_content2_content2-tracked
+  C missing_content2_content3-tracked
+  C missing_missing_content3-tracked
+  $ hg status --all 'glob:*_missing-tracked'
+  ! content1_content1_missing-tracked
+  ! content1_content2_missing-tracked
+  ! content1_missing_missing-tracked
+  ! missing_content2_missing-tracked
+  ! missing_missing_missing-tracked
+  $ hg status --all 'glob:*-untracked'
+  ? content1_content1_content1-untracked
+  ? content1_content1_content3-untracked
+  ? content1_content2_content1-untracked
+  ? content1_content2_content2-untracked
+  ? content1_content2_content3-untracked
+  ? content1_missing_content1-untracked
+  ? content1_missing_content3-untracked
+  ? missing_content2_content2-untracked
+  ? missing_content2_content3-untracked
+  ? missing_missing_content3-untracked
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 6 STABLE] tests: demonstrate that .hgsubstate isn't updated on amend

2017-11-08 Thread Yuya Nishihara 
# HG changeset patch
# User Yuya Nishihara 
# Date 1509789297 -32400
#  Sat Nov 04 18:54:57 2017 +0900
# Branch stable
# Node ID 4edd1c6048fac782a8606d8b8e22c24a6208de7a
# Parent  53b507cba99d5fe71a5a018cba1c0ab67d64ce79
tests: demonstrate that .hgsubstate isn't updated on amend

Loosely based on test-amend.t and test-subrepo.t

diff --git a/tests/test-amend-subrepo.t b/tests/test-amend-subrepo.t
new file mode 100644
--- /dev/null
+++ b/tests/test-amend-subrepo.t
@@ -0,0 +1,213 @@
+#testcases obsstore-off obsstore-on
+
+  $ cat << EOF >> $HGRCPATH
+  > [extensions]
+  > amend =
+  > EOF
+
+#if obsstore-on
+  $ cat << EOF >> $HGRCPATH
+  > [experimental]
+  > evolution.createmarkers = True
+  > EOF
+#endif
+
+Prepare parent repo
+---
+
+  $ hg init r
+  $ cd r
+
+  $ echo a > a
+  $ hg ci -Am0
+  adding a
+
+Link first subrepo
+--
+
+  $ echo 's = s' >> .hgsub
+  $ hg add .hgsub
+  $ hg init s
+
+amend without .hgsub
+
+BROKEN: should say "can't commit subrepos without .hgsub"
+  $ hg amend s
+  nothing changed
+  [1]
+
+amend with subrepo
+
+BROKEN: should update .hgsubstate
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+  $ hg status --change .
+  A .hgsub
+  A a
+
+FIX UP .hgsubstate
+
+  $ hg ci -mfix
+  $ hg rollback -q
+  $ hg add .hgsubstate
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+
+  $ cat .hgsubstate
+   s
+
+Update subrepo
+--
+
+add new commit to be amended
+
+  $ echo a >> a
+  $ hg ci -m1
+
+amend with dirty subrepo
+
+  $ echo a >> s/a
+  $ hg add -R s
+  adding s/a
+BROKEN: should say "uncommitted changes in subrepository"
+  $ hg amend
+  nothing changed
+  [1]
+
+amend with modified subrepo
+
+  $ hg ci -R s -m0
+BROKEN: should update .hgsubstate
+  $ hg amend
+  nothing changed
+  [1]
+  $ hg status --change .
+  M a
+
+FIX UP .hgsubstate
+
+  $ hg ci -mfix
+  $ hg rollback -q
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+
+  $ cat .hgsubstate
+  f7b1eb17ad24730a1651fccd46c43826d1bbc2ac s
+
+revert subrepo change
+
+  $ hg up -R s -q null
+BROKEN: should update .hgsubstate
+  $ hg amend
+  nothing changed
+  [1]
+
+FIX UP .hgsubstate
+
+  $ hg ci -mfix
+  $ hg rollback -q
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+
+  $ hg status --change .
+  M a
+
+Link another subrepo
+
+
+add new commit to be amended
+
+  $ echo b >> b
+  $ hg ci -qAm2
+
+also checks if non-subrepo change is included
+
+  $ echo a >> a
+
+amend with another subrepo
+
+  $ hg init t
+  $ echo b >> t/b
+  $ hg ci -R t -Am0
+  adding b
+  $ echo 't = t' >> .hgsub
+BROKEN: should update .hgsubstate
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+  $ hg status --change .
+  M .hgsub
+  M a
+  A b
+
+FIX UP .hgsubstate
+
+  $ hg ci -mfix
+  $ hg rollback -q
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+
+  $ cat .hgsubstate
+   s
+  bfb1a4fb358498a9533dabf4f2043d94162f1fcd t
+
+Unlink one subrepo
+--
+
+add new commit to be amended
+
+  $ echo a >> a
+  $ hg ci -m3
+
+amend with one subrepo dropped
+
+  $ echo 't = t' > .hgsub
+BROKEN: should update .hgsubstate
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+  $ hg status --change .
+  M .hgsub
+  M a
+
+FIX UP .hgsubstate
+
+  $ echo 's = s' > .hgsub
+  $ hg amend -q
+  $ echo 't = t' > .hgsub
+  $ hg ci -mfix
+  $ hg rollback -q
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+
+  $ cat .hgsubstate
+  bfb1a4fb358498a9533dabf4f2043d94162f1fcd t
+
+Unlink subrepos completely
+--
+
+add new commit to be amended
+
+  $ echo a >> a
+  $ hg ci -m3
+
+amend with .hgsub removed
+
+  $ hg rm .hgsub
+BROKEN: should update .hgsubstate
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+  $ hg status --change .
+  M a
+  R .hgsub
+
+FIX UP .hgsubstate
+
+  $ hg forget .hgsubstate
+  $ hg amend
+  saved backup bundle to * (glob) (obsstore-off !)
+
+  $ hg status --change .
+  M a
+  R .hgsub
+  R .hgsubstate
+
+  $ cd ..
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1336: Summary: hg rm -A option prints the message of every file in the repo. This is not very user friendly for a big repository with thousands of files. So enabling this feature only when run in --v

2017-11-08 Thread pavanpc (Pavan Kumar PC) 
pavanpc created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

TEST PLAN
  check if the message is printed in verbose mode.
  
cd  hg-crew/tests
rt test-remove.t

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1336

AFFECTED FILES
  mercurial/cmdutil.py
  tests/test-remove.t

CHANGE DETAILS

diff --git a/tests/test-remove.t b/tests/test-remove.t
--- a/tests/test-remove.t
+++ b/tests/test-remove.t
@@ -168,11 +168,11 @@
   \r (no-eol) (esc)
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
-20 state added, options -A
+20 state added, options -Av
 
   $ echo b > bar
   $ hg add bar
-  $ remove -A bar
+  $ remove -Av bar
   \r (no-eol) (esc)
   deleting [===>] 1/1\r (no-eol) (esc)
   \r (no-eol) (esc)
@@ -189,9 +189,9 @@
   \r (no-eol) (esc)
   0 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
-21 state clean, options -A
+21 state clean, options -Av
 
-  $ remove -A foo
+  $ remove -Av foo
   \r (no-eol) (esc)
   deleting [===>] 1/1\r (no-eol) (esc)
   \r (no-eol) (esc)
@@ -205,10 +205,10 @@
   ./foo
   0 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
-22 state modified, options -A
+22 state modified, options -Av
 
   $ echo b >> foo
-  $ remove -A foo
+  $ remove -Av foo
   \r (no-eol) (esc)
   deleting [===>] 1/1\r (no-eol) (esc)
   \r (no-eol) (esc)
@@ -357,10 +357,10 @@
   \r (no-eol) (esc)
   2 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
-dir, options -A
+dir, options -Av
 
   $ rm test/bar
-  $ remove -A test
+  $ remove -Av test
   \r (no-eol) (esc)
   deleting [===>] 1/1\r (no-eol) (esc)
   \r (no-eol) (esc)
diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -2967,18 +2967,19 @@
 
 if force:
 list = modified + deleted + clean + added
-elif after:
+elif after or ui.verbose:
 list = deleted
-remaining = modified + added + clean
-total = len(remaining)
-count = 0
-for f in remaining:
-count += 1
-ui.progress(_('skipping'), count, total=total, unit=_('files'))
-warnings.append(_('not removing %s: file still exists\n')
-% m.rel(f))
-ret = 1
-ui.progress(_('skipping'), None)
+if ui.verbose:
+remaining = modified + added + clean
+total = len(remaining)
+count = 0
+for f in remaining:
+count += 1
+ui.progress(_('skipping'), count, total=total, unit=_('files'))
+warnings.append(_('not removing %s: file still exists\n')
+% m.rel(f))
+ret = 1
+ui.progress(_('skipping'), None)
 else:
 list = deleted + clean
 total = len(modified) + len(added)



To: pavanpc, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel