D4312: New bookflow extension for bookmark-based branching

2018-10-17 Thread smf (Sean Farley)
smf added a comment.


  Thanks for submitting your patch, it’s clearly a work of passion and I 
appreciate that. I know feature branches is a much sought-after behavior. 
However, I must express my objection to this direction in bookmarks. (apologies 
in advance)
  
  I think the core issue at hand is the difference between data models. In 
particular, there are two huge differences:
  
  - mapping (one-to-many vs one-to-one) and
  - garbage collection
  
  A bit of background: I've written many versions of this ref-like behavior 
over the years, including one during my time at Bitbucket. My experience with 
each of these extensions never felt right and I ended up throwing them all away.
  
  As programmers, we like to abstract and create functional mappings around 
data. But we can't hack our way around the difference between a one-to-many 
relation vs a one-to-one relation. At a fundamental mathematical level, the two 
relations are not the same. For the last ten years, millions of Mercurial repos 
have been created that have (named) branches that are stored in the metadata. 
This is pretty ingrained in each developer's mind: one commit <-> one branch 
name. I think that changing this at this point would cause unnecessary 
frustration for our users.
  
  For hosting sites like Bitbucket, this is even worse. Having two branching 
models is just impossible. A simple example is writing the branch api: should 
it return named branches? bookmarks? both? What happens when there is a call to 
create a branch? Create a branch? bookmark? both??? There are other rough edges 
as well: documentation, user tutorials, UI/UX, etc. The lessons I learned at 
Bitbucket helped me understand that core Mercurial should not recommend 
bookmarks as feature branches.
  
  It doesn't stop there, though. Possibly the biggest misunderstanding is what 
and how anonymous heads work and why they don't disappear like they do in git. 
It is straight-up impossible to remove anonymous heads in Mercurial and trying 
to paper over them just makes things worse. I've tried assigning auto-generated 
bookmarks, auto-marking obsolescence markers, auto-hiding commits locally, etc; 
anything I could think of. Nothing ever really worked in my experience.
  
  Perhaps the show stopper, though, is evolve. Much like Mercurial, evolve 
views the world through  changesets. Changesets and their successors / 
predecessors are related by obsolescence markers and form a DAG, just like 
changesets of a repo are in a DAG. These relations have a starting point and an 
ending point and the most natural fit to that is, of course, storing the branch 
name in the actual changeset. When needing to evolve changsets, the only 
practical (perhaps even only possible) way to determine which branch it lived 
on is with named branches. Bookmarks / refs lose that information. Therefore, 
trying to shoehorn that model into evolve or even core is reinventing what 
branches already achieve.
  
  Everything I tried in the bookmarks world fought against the core design 
principles of Mercurial. For those reasons, I think it is best to stick to one 
branching model and keep bookmarks (as much as possible) in third-party 
extensions (or if my dreams come true: in hg-git). As I've witnessed over the 
years, while at both Mercurial and Bitbucket, users will assume an implicit 
blessing for behavior when a feature is included in hgext. See for example, the 
nightmare of largefiles (and that need causing Facebook's rewrite in lfs), eol, 
keyword, etc, etc. We can put as many "features of last resort" warnings that 
we want but the ultimate protection we have to save ourselves the headache is 
to keep things out of hgext.
  
  All of this is not to say I don't want feature branches in Mercurial, I do. 
It's something Erik and I have thought a lot about and we both want to improve 
on (named) branches that allow feature workflows and work better with phases 
and evolve. I have even worked over the last few months on this exact approach 
and, on my side, I will use this as a fire to get my own work out the door. I 
would welcome and appreciate any kind of help in planning, coding, or even just 
spitballing.

REVISION DETAIL
  https://phab.mercurial-scm.org/D4312

To: idlsoft, #hg-reviewers, pulkit, marcink
Cc: smf, markand, marcink, durin42, jwatt, pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[PATCH] py3: fix test-propertycache.py on Windows

2018-10-17 Thread Matt Harbison
# HG changeset patch
# User Matt Harbison 
# Date 1538099653 14400
#  Thu Sep 27 21:54:13 2018 -0400
# Node ID 7b9cf4dddf95b5964d3a05f65b9609b751f4334a
# Parent  b08be5625c0094a2fdd6cc00bec2a4dcdcc9ea75
py3: fix test-propertycache.py on Windows

diff --git a/tests/test-propertycache.py b/tests/test-propertycache.py
--- a/tests/test-propertycache.py
+++ b/tests/test-propertycache.py
@@ -16,6 +16,10 @@ from mercurial import (
 util,
 )
 
+from mercurial.utils import (
+procutil,
+)
+
 # create some special property cache that trace they call
 
 calllog = []
@@ -46,7 +50,9 @@ localrepo.localrepository.testcachedunfi
 # Create an empty repo and instantiate it. It is important to run
 # these tests on the real object to detect regression.
 repopath = pycompat.fsencode(os.path.join(os.environ['TESTTMP'], 'repo'))
-assert subprocess.call(['hg', 'init', repopath]) == 0
+assert subprocess.call(pycompat.rapply(procutil.tonativestr,
+   [b'hg', b'init', repopath])) == 0
+
 ui = uimod.ui.load()
 repo = hg.repository(ui, path=repopath).unfiltered()
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[PATCH 3 of 3] py3: fix test-import-context.t

2018-10-17 Thread Matt Harbison
# HG changeset patch
# User Matt Harbison 
# Date 1539827689 14400
#  Wed Oct 17 21:54:49 2018 -0400
# Node ID b08be5625c0094a2fdd6cc00bec2a4dcdcc9ea75
# Parent  0f8e789ea9450bd401cadd53cdf1cb81ec688cb2
py3: fix test-import-context.t

diff --git a/tests/test-import-context.t b/tests/test-import-context.t
--- a/tests/test-import-context.t
+++ b/tests/test-import-context.t
@@ -19,7 +19,10 @@ Test applying context diffs
   > EOF
   $ cat > cat.py < import sys
-  > sys.stdout.write(repr(open(sys.argv[1], 'rb').read()) + '\n')
+  > from mercurial import pycompat
+  > from mercurial.utils import stringutil
+  > pycompat.stdout.write(b'%s\n'
+  >   % stringutil.pprint(open(sys.argv[1], 'rb').read()))
   > EOF
 
 Initialize the test repository
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[PATCH 2 of 3] py3: restore perfstartup() prior to b456b2e0ad9f on Windows

2018-10-17 Thread Matt Harbison
# HG changeset patch
# User Matt Harbison 
# Date 1539655330 14400
#  Mon Oct 15 22:02:10 2018 -0400
# Node ID 0f8e789ea9450bd401cadd53cdf1cb81ec688cb2
# Parent  d09e43b6d57e02680191b71823afa69a2e80581b
py3: restore perfstartup() prior to b456b2e0ad9f on Windows

Otherwise the test errors out with:

--- c:/Users/Matt/projects/hg_py3/tests/test-contrib-perf.t
+++ c:/Users/Matt/projects/hg_py3/tests/test-contrib-perf.t.err
@@ -184,6 +184,8 @@
   $ hg perfrevrange
   $ hg perfrevset 'all()'
   $ hg perfstartup
+  'b'c:' is not recognized as an internal or external command,
+  operable program or batch file.
   $ hg perfstatus
   $ hg perftags
   $ hg perftemplating

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -970,13 +970,13 @@ def perfindex(ui, repo, **opts):
 def perfstartup(ui, repo, **opts):
 opts = _byteskwargs(opts)
 timer, fm = gettimer(ui, opts)
-cmd = fsencode(sys.argv[0])
 def d():
 if os.name != r'nt':
-os.system(b"HGRCPATH= %s version -q > /dev/null" % cmd)
+os.system(b"HGRCPATH= %s version -q > /dev/null" %
+  fsencode(sys.argv[0]))
 else:
 os.environ[r'HGRCPATH'] = r' '
-os.system(r"%s version -q > NUL" % cmd)
+os.system(r"%s version -q > NUL" % sys.argv[0])
 timer(d)
 fm.end()
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[PATCH 1 of 3] help: document the server capabilities added by the LFS extension

2018-10-17 Thread Matt Harbison
# HG changeset patch
# User Matt Harbison 
# Date 1539824743 14400
#  Wed Oct 17 21:05:43 2018 -0400
# Node ID 197a387c4ed9a510a5d92e68e7be7752b327cb99
# Parent  d916ed3ca951a3bfd37bf29f54dc70f82787512a
help: document the server capabilities added by the LFS extension

I didn't bother marking these experimental because it references the extension
that is already marked experimental.

diff --git a/mercurial/help/internals/wireprotocol.txt 
b/mercurial/help/internals/wireprotocol.txt
--- a/mercurial/help/internals/wireprotocol.txt
+++ b/mercurial/help/internals/wireprotocol.txt
@@ -702,6 +702,25 @@ Whether the server supports the ``known`
 
 This capability/command was introduced in Mercurial 1.9 (released July 2011).
 
+lfs
+---
+
+Indicates that the LFS extension is enabled on the server.  It makes no claims
+about the repository actually having LFS blobs committed to it.
+
+This capability was introduced by the LFS extension in Mercurial 4.5 (released
+Feb 2018).
+
+lfs-serve
+-
+
+Indicates that the LFS extension is enabled on the server, and LFS blobs are
+committed to the remote repository.  (Specifically, it indicates that the 'lfs'
+requirement is present in the remote repository.)
+
+This capability was introduced by the LFS extension in Mercurial 4.8 (released
+Nov 2018).
+
 lookup
 --
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


mercurial@40323: 10 new changesets

2018-10-17 Thread Mercurial Commits
10 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/73c2b9c9cd3c
changeset:   40314:73c2b9c9cd3c
user:Yuya Nishihara 
date:Tue Oct 16 07:00:41 2018 +0200
summary: py3: convert string literals to bytes in contrib/hgclient.py

https://www.mercurial-scm.org/repo/hg/rev/431a831342d2
changeset:   40315:431a831342d2
user:Yuya Nishihara 
date:Tue Oct 16 07:08:12 2018 +0200
summary: py3: work around unicode stdio streams in contrib/hgclient.py

https://www.mercurial-scm.org/repo/hg/rev/09540a5f0a15
changeset:   40316:09540a5f0a15
user:Yuya Nishihara 
date:Tue Oct 16 07:04:07 2018 +0200
summary: py3: reinvent print() function for contrib/hgclient.py

https://www.mercurial-scm.org/repo/hg/rev/6958eb9bdcd6
changeset:   40317:6958eb9bdcd6
user:Yuya Nishihara 
date:Tue Oct 16 08:06:17 2018 +0200
summary: py3: rewrite StringIO fallback for Python 3

https://www.mercurial-scm.org/repo/hg/rev/55fd0fefbec4
changeset:   40318:55fd0fefbec4
user:Yuya Nishihara 
date:Tue Oct 16 07:18:30 2018 +0200
summary: py3: flush std streams before/after running user code in 
heredoctest.py

https://www.mercurial-scm.org/repo/hg/rev/eb131464c805
changeset:   40319:eb131464c805
user:Yuya Nishihara 
date:Tue Oct 16 07:19:38 2018 +0200
summary: py3: byte-stringify literals in test-keyword.t

https://www.mercurial-scm.org/repo/hg/rev/9b2e1b00ee94
changeset:   40320:9b2e1b00ee94
user:Gregory Szorc 
date:Tue Oct 16 17:48:28 2018 +0200
summary: tests: use byte literals in test-storage.py

https://www.mercurial-scm.org/repo/hg/rev/6994a8be3663
changeset:   40321:6994a8be3663
user:Gregory Szorc 
date:Tue Oct 16 17:45:39 2018 +0200
summary: storageutil: convert fileid to bytes to avoid cast to %s

https://www.mercurial-scm.org/repo/hg/rev/ddeb510d6815
changeset:   40322:ddeb510d6815
user:Gregory Szorc 
date:Tue Oct 16 15:24:06 2018 +0200
summary: testing: remove expectation of error on bad node insert

https://www.mercurial-scm.org/repo/hg/rev/2c0aa02ecd5a
changeset:   40323:2c0aa02ecd5a
bookmark:@
tag: tip
user:Gregory Szorc 
date:Tue Oct 16 16:24:46 2018 +0200
summary: testing: switch to inserting deltas

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5130: lfs: autoload the extension when cloning from repo with lfs enabled

2018-10-17 Thread mharbison72 (Matt Harbison)
mharbison72 added a comment.


  One thing I forgot to mention in the commit comment is that `hg clone --rev 
REV`, where REV is some commit prior to committing any lfs blobs, will cause 
the requirement to be added to the clone.  That didn't happen previously with 
the transaction hook.  But that's probably enough of a corner case to not 
really care.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5130

To: mharbison72, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5113: revisions: when using prefixhexnode, ensure we prefix "0"

2018-10-17 Thread spectral (Kyle Lippincott)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGd916ed3ca951: revisions: when using prefixhexnode, ensure 
we prefix 0 (authored by spectral, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D5113?vs=12160=12230#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5113?vs=12160=12230

REVISION DETAIL
  https://phab.mercurial-scm.org/D5113

AFFECTED FILES
  mercurial/scmutil.py
  tests/test-revisions.t
  tests/test-template-functions.t

CHANGE DETAILS

diff --git a/tests/test-template-functions.t b/tests/test-template-functions.t
--- a/tests/test-template-functions.t
+++ b/tests/test-template-functions.t
@@ -804,6 +804,8 @@
   e777603221
   bcc7ff960b
   f7769ec2ab
+  $ hg log --template '{shortest(node, 1)}\n' -r null
+  00
   $ hg log --template '{node|shortest}\n' -l1
   e777
 
@@ -915,6 +917,55 @@
 
   $ cd ..
 
+Test prefixhexnode when the first character of the hash is 0.
+  $ hg init hashcollision2
+  $ cd hashcollision2
+  $ cat <> .hg/hgrc
+  > [experimental]
+  > evolution.createmarkers=True
+  > EOF
+  $ echo 0 > a
+  $ hg ci -qAm 0
+  $ echo 21 > a
+  $ hg ci -qm 21
+  $ hg up -q null
+  $ hg log -r0: -T '{rev}:{node}\n'
+  0:b4e73ffab476aa0ee32ed81ca51e07169844bc6a
+  1:0cf177ba2b1dc3862a00fb81715fec90950201be
+
+ we need the 'x' prefix to ensure we aren't colliding with rev0. We identify
+ the collision with nullid if we aren't using disambiguatewithin, so we need 
to set
+ that as well.
+  $ hg --config experimental.revisions.disambiguatewithin='descendants(0)' \
+  >--config experimental.revisions.prefixhexnode=yes \
+  >log -r 1 -T '{rev}:{shortest(node, 0)}\n'
+  1:x0
+
+  $ hg debugobsolete 0cf177ba2b1dc3862a00fb81715fec90950201be
+  obsoleted 1 changesets
+  $ hg up -q 0
+  $ echo 61 > a
+  $ hg ci -m 61
+  $ hg log -r0: -T '{rev}:{node}\n'
+  0:b4e73ffab476aa0ee32ed81ca51e07169844bc6a
+  2:01384dde84b3a511ae0835f35ac40bd806c99bb8
+
+ we still have the 'x' prefix because '0' is still the shortest prefix, since
+ rev1's '0c' is hidden.
+  $ hg --config experimental.revisions.disambiguatewithin=0:-1-0 \
+  >--config experimental.revisions.prefixhexnode=yes \
+  >log -r 0:-1-0 -T '{rev}:{shortest(node, 0)}\n'
+  2:x0
+
+ we don't have the 'x' prefix on 2 because '01' is not a synonym for rev1.
+  $ hg --config experimental.revisions.disambiguatewithin=0:-1-0 \
+  >--config experimental.revisions.prefixhexnode=yes \
+  >log -r 0:-1-0 -T '{rev}:{shortest(node, 0)}\n' --hidden
+  1:0c
+  2:01
+
+  $ cd ..
+
 Test pad function
 
   $ cd r
diff --git a/tests/test-revisions.t b/tests/test-revisions.t
--- a/tests/test-revisions.t
+++ b/tests/test-revisions.t
@@ -25,7 +25,7 @@
   > revisions.disambiguatewithin=not 4
   > EOF
   $ hg l
-  5:0
+  5:00
   4:7ba5d
   3:7b
   2:72
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -477,8 +477,9 @@
 i = int(prefix)
 # if we are a pure int, then starting with zero will not be
 # confused as a rev; or, obviously, if the int is larger
-# than the value of the tip rev
-if prefix[0:1] == b'0' or i >= len(repo):
+# than the value of the tip rev. We still need to disambiguate if
+# prefix == '0', since that *is* a valid revnum.
+if (prefix != b'0' and prefix[0:1] == b'0') or i >= len(repo):
 return False
 return True
 except ValueError:



To: spectral, #hg-reviewers, martinvonz
Cc: martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5113: revisions: when using prefixhexnode, ensure we prefix "0"

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz added inline comments.

INLINE COMMENTS

> test-template-functions.t:928-929
> +
> + we need the 'x' prefix to ensure we aren't colliding with rev0. We *do*
> + identify the collision if we aren't using disambiguatewithin, so we need to 
> set
> + that as well.

"We *do* identify the collision if..." is a little misleading because it's a 
different collision that's identified, so I've changed this to "We identify the 
collision with nullid if..." in flight.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5113

To: spectral, #hg-reviewers, martinvonz
Cc: martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5113: revisions: when using prefixhexnode, ensure we prefix "0"

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz accepted this revision.
martinvonz added a comment.
This revision is now accepted and ready to land.


  I'll also note that this bug was there all along, but it got way more visible 
with with the disambiguatewithin stuff. It used to fail only when calculating 
the shortest prefix for nullid in a repo without any other revisions starting 
with hex digit 0. The following patch shows that (it failed back on 4.6 too). 
I'm folding it in in flight. I'm also fixing test-revisions.t (which already 
showed the bug, but I had not realized it was a bug :P).
  
  diff --git a/tests/test-template-functions.t b/tests/test-template-functions.t
  
  - a/tests/test-template-functions.t
  
  +++ b/tests/test-template-functions.t
  @@ -804,6 +804,8 @@ Test shortest(node) function:
  
e777603221
bcc7ff960b
f7769ec2ab
  
  +  $ hg log --template '{shortest(node, 1)}\n' -r null
  +  00
  
$ hg log --template '{node|shortest}\n' -l1
e777

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5113

To: spectral, #hg-reviewers, martinvonz
Cc: martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5137: streamclone: new server config and some API changes for narrow stream clones

2018-10-17 Thread pulkit (Pulkit Goyal)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGaf62936c2508: streamclone: new server config and some API 
changes for narrow stream clones (authored by pulkit, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D5137?vs=12208=12227#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5137?vs=12208=12227

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

AFFECTED FILES
  mercurial/bundle2.py
  mercurial/configitems.py
  mercurial/streamclone.py
  tests/test-narrow-clone-stream.t

CHANGE DETAILS

diff --git a/tests/test-narrow-clone-stream.t b/tests/test-narrow-clone-stream.t
new file mode 100644
--- /dev/null
+++ b/tests/test-narrow-clone-stream.t
@@ -0,0 +1,39 @@
+Tests narrow stream clones
+
+  $ . "$TESTDIR/narrow-library.sh"
+
+Server setup
+
+  $ hg init master
+  $ cd master
+  $ mkdir dir
+  $ mkdir dir/src
+  $ cd dir/src
+  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit 
-m "Commit src $x"; done
+
+  $ cd ..
+  $ mkdir tests
+  $ cd tests
+  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit 
-m "Commit src $x"; done
+  $ cd ../../..
+
+Trying to stream clone when the server does not support it
+
+  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include 
"dir/src/f10" --stream
+  streaming all changes
+  remote: abort: server does not support narrow stream clones
+  abort: pull failed on remote
+  [255]
+
+Enable stream clone on the server
+
+  $ echo "[server]" >> master/.hg/hgrc
+  $ echo "stream-narrow-clones=True" >> master/.hg/hgrc
+
+Cloning a specific file when stream clone is supported
+
+  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include 
"dir/src/f10" --stream
+  streaming all changes
+  remote: abort: server does not support narrow stream clones
+  abort: pull failed on remote
+  [255]
diff --git a/mercurial/streamclone.py b/mercurial/streamclone.py
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -531,7 +531,7 @@
 finally:
 fp.close()
 
-def generatev2(repo):
+def generatev2(repo, includes, excludes):
 """Emit content for version 2 of a streaming clone.
 
 the data stream consists the following entries:
@@ -544,6 +544,10 @@
 Returns a 3-tuple of (file count, file size, data iterator).
 """
 
+# temporarily raise error until we add storage level logic
+if includes or excludes:
+raise error.Abort(_("server does not support narrow stream clones"))
+
 with repo.lock():
 
 entries = []
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -610,6 +610,9 @@
 coreconfigitem('experimental', 'server.manifestdata.recommended-batch-size',
 default=10,
 )
+coreconfigitem('experimental.server', 'stream-narrow-clones',
+default=False,
+)
 coreconfigitem('experimental', 'single-head-per-branch',
 default=False,
 )
diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -1687,7 +1687,18 @@
 # to avoid compression to consumers of the bundle.
 bundler.prefercompressed = False
 
-filecount, bytecount, it = streamclone.generatev2(repo)
+# get the includes and excludes
+includepats = kwargs.get(r'includepats')
+excludepats = kwargs.get(r'excludepats')
+
+narrowstream = repo.ui.configbool('experimental.server',
+  'stream-narrow-clones')
+
+if (includepats or excludepats) and not narrowstream:
+raise error.Abort(_('server does not support narrow stream clones'))
+
+filecount, bytecount, it = streamclone.generatev2(repo, includepats,
+  excludepats)
 requirements = _formatrequirementsspec(repo.requirements)
 part = bundler.newpart('stream2', data=it)
 part.addparam('bytecount', '%d' % bytecount, mandatory=True)



To: pulkit, durin42, #hg-reviewers, martinvonz
Cc: martinvonz, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4850: store: pass matcher to store.datafiles()

2018-10-17 Thread pulkit (Pulkit Goyal)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2d45b549392f: store: pass matcher to store.datafiles() 
(authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4850?vs=12210=12229

REVISION DETAIL
  https://phab.mercurial-scm.org/D4850

AFFECTED FILES
  mercurial/store.py

CHANGE DETAILS

diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -359,17 +359,21 @@
 l.sort()
 return l
 
-def datafiles(self):
+def datafiles(self, matcher=None):
 return self._walk('data', True) + self._walk('meta', True)
 
 def topfiles(self):
 # yield manifest before changelog
 return reversed(self._walk('', False))
 
-def walk(self):
-'''yields (unencoded, encoded, size)'''
+def walk(self, matcher=None):
+'''yields (unencoded, encoded, size)
+
+if a matcher is passed, storage files of only those tracked paths
+are passed with matches the matcher
+'''
 # yield data files first
-for x in self.datafiles():
+for x in self.datafiles(matcher):
 yield x
 for x in self.topfiles():
 yield x
@@ -407,7 +411,7 @@
 self.vfs = vfsmod.filtervfs(vfs, encodefilename)
 self.opener = self.vfs
 
-def datafiles(self):
+def datafiles(self, matcher=None):
 for a, b, size in super(encodedstore, self).datafiles():
 try:
 a = decodefilename(a)
@@ -536,7 +540,7 @@
 def getsize(self, path):
 return self.rawvfs.stat(path).st_size
 
-def datafiles(self):
+def datafiles(self, matcher=None):
 for f in sorted(self.fncache):
 ef = self.encode(f)
 try:



To: pulkit, #hg-reviewers, martinvonz
Cc: indygreg, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4851: streamclone: pass narrowing related info in _walkstreamfiles()

2018-10-17 Thread pulkit (Pulkit Goyal)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGf0e8f27768eb: streamclone: pass narrowing related info in 
_walkstreamfiles() (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4851?vs=12209=12228

REVISION DETAIL
  https://phab.mercurial-scm.org/D4851

AFFECTED FILES
  mercurial/streamclone.py

CHANGE DETAILS

diff --git a/mercurial/streamclone.py b/mercurial/streamclone.py
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -16,6 +16,7 @@
 branchmap,
 cacheutil,
 error,
+narrowspec,
 phases,
 pycompat,
 repository,
@@ -194,8 +195,8 @@
 return True
 
 # This is it's own function so extensions can override it.
-def _walkstreamfiles(repo):
-return repo.store.walk()
+def _walkstreamfiles(repo, matcher=None):
+return repo.store.walk(matcher)
 
 def generatev1(repo):
 """Emit content for version 1 of a streaming clone.
@@ -553,8 +554,12 @@
 entries = []
 totalfilesize = 0
 
+matcher = None
+if includes or excludes:
+matcher = narrowspec.match(repo.root, includes, excludes)
+
 repo.ui.debug('scanning\n')
-for name, ename, size in _walkstreamfiles(repo):
+for name, ename, size in _walkstreamfiles(repo, matcher):
 if size:
 entries.append((_srcstore, name, _fileappend, size))
 totalfilesize += size



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5137: streamclone: new server config and some API changes for narrow stream clones

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz added inline comments.

INLINE COMMENTS

> bundle2.py:1691-1692
> +# get the inlcudes and excludes
> +includepats = kwargs[r'includepats']
> +excludepats = kwargs[r'excludepats']
> +

I had to change these to `kwargs.get()` to get `test-http.t` and many others to 
pass. (I also fixed the typo in the comment above)

> config.txt:1851-1854
> +``stream-narrow-clones``
> +Whether the server supports streaming narrow clones or not.
> +(default: False)
> +

I removed this per indygreg's comment

> streamclone.py:549
> +if includes or excludes:
> +raise error.Abort(_("support for narrow stream clones is missing"))
> +

I changed this to "server does not support narrow stream clones" to match the 
error from bundle2.

Btw, why is it raised in two places? Is the one in bundle2 that actually 
happens? Maybe this should be a ProgrammingError then?

> test-narrow-clone-stream.t:37
> +  streaming all changes
> +  remote: abort: support for narrow stream clones is missing
> +  abort: pull failed on remote

I changed this to "server does not support narrow stream clones" to match the 
actual output (test was failing)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

To: pulkit, durin42, #hg-reviewers, martinvonz
Cc: martinvonz, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5137: streamclone: new server config and some API changes for narrow stream clones

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz added inline comments.

INLINE COMMENTS

> indygreg wrote in configitems.py:1014
> Perhaps we should mark this as experimental via an inline comment and forego 
> documenting the config option for now?

Makes sense, but I don't know how to do that, so I'll just rename it to 
`experimental.server.stream-narrow-clones` in flight (because we're close to 
the freeze)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

To: pulkit, durin42, #hg-reviewers
Cc: martinvonz, indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4810: repository: introduce an ellipses repo requirement

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz added a comment.


  I think this patch (and later patches that start using it, of course) would 
be a good idea. I suppose it won't make it into 4.8, but we don't have a 
freeze, so please send an updated version whenever you have time

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4810

To: pulkit, #hg-reviewers
Cc: indygreg, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4931: narrow: only send the narrowspecs back if ACL in play

2018-10-17 Thread pulkit (Pulkit Goyal)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGcb516a854bc7: narrow: only send the narrowspecs back if ACL 
in play (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4931?vs=12192=12226

REVISION DETAIL
  https://phab.mercurial-scm.org/D4931

AFFECTED FILES
  mercurial/exchange.py
  tests/test-narrow-rebase.t
  tests/test-narrow-widen-no-ellipsis.t

CHANGE DETAILS

diff --git a/tests/test-narrow-widen-no-ellipsis.t 
b/tests/test-narrow-widen-no-ellipsis.t
--- a/tests/test-narrow-widen-no-ellipsis.t
+++ b/tests/test-narrow-widen-no-ellipsis.t
@@ -137,7 +137,6 @@
   adding file changes
   added 5 changesets with 4 changes to 2 files
   new changesets *:* (glob)
-  3 local changesets published
   (run 'hg update' to get a working copy)
   $ hg update -r 'desc("add wider")'
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
diff --git a/tests/test-narrow-rebase.t b/tests/test-narrow-rebase.t
--- a/tests/test-narrow-rebase.t
+++ b/tests/test-narrow-rebase.t
@@ -86,7 +86,6 @@
 
   $ hg update -q 'desc("conflicting outside/f1")'
   $ hg phase -f -d .
-  no phases changed
   $ hg rebase -d 'desc("modify outside/f1")'
   rebasing 4:707c035aadb6 "conflicting outside/f1"
   abort: conflict in file 'outside/f1' is outside narrow clone
diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1907,6 +1907,7 @@
 new_args = {}
 new_args.update(kwargs)
 new_args[r'narrow'] = True
+new_args[r'narrow_acl'] = True
 new_args[r'includepats'] = req_includes
 if req_excludes:
 new_args[r'excludepats'] = req_excludes
@@ -2171,7 +2172,8 @@
 if 'treemanifest' in repo.requirements:
 part.addparam('treemanifest', '1')
 
-if kwargs.get(r'narrow', False) and (include or exclude):
+if (kwargs.get(r'narrow', False) and kwargs.get('narrow_acl', False)
+and (include or exclude)):
 narrowspecpart = bundler.newpart('narrow:spec')
 if include:
 narrowspecpart.addparam(



To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


Last call for 4.8 patches

2018-10-17 Thread Augie Fackler
Howdy folks,

We're trying something new this cycle: not freezing the default branch after 
cutting the 4.8 RC. What that means is that sometime _tomorrow_ I'll merge 
default into stable and do a 4.8 release, but we'll keep taking patches on 
default after that (so those will eventually be in 4.9). Bugfixes for the RC 
should go to stable as usual.

Thanks!
Augie
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5142: zsh_completion: add -l/--list flag for hg bookmarks completion

2018-10-17 Thread av6 (Anton Shestakov)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGd365e2b7aa2a: zsh_completion: add -l/--list flag for hg 
bookmarks completion (authored by av6, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5142?vs=12215=12225

REVISION DETAIL
  https://phab.mercurial-scm.org/D5142

AFFECTED FILES
  contrib/zsh_completion

CHANGE DETAILS

diff --git a/contrib/zsh_completion b/contrib/zsh_completion
--- a/contrib/zsh_completion
+++ b/contrib/zsh_completion
@@ -532,10 +532,11 @@
 _hg_cmd_bookmarks() {
   _arguments -s -S : $_hg_global_opts \
   '(--force -f)'{-f,--force}'[force]' \
-  '(--inactive -i)'{-i,--inactive}'[mark a bookmark inactive]' \
-  '(--rev -r --delete -d --rename 
-m)'{-r+,--rev=}'[revision]:revision:_hg_labels' \
-  '(--rev -r --delete -d --rename -m)'{-d,--delete}'[delete a given bookmark]' 
\
-  '(--rev -r --delete -d --rename -m)'{-m+,--rename=}'[rename a given 
bookmark]:bookmark:_hg_bookmarks' \
+  '(--inactive -i --delete -d --list -l)'{-i,--inactive}'[mark a bookmark 
inactive]' \
+  '(--rev -r --delete -d --rename -m --list 
-l)'{-r+,--rev=}'[revision]:revision:_hg_labels' \
+  '(--rev -r --delete -d --rename -m --list -l --inactive 
-i)'{-d,--delete}'[delete a given bookmark]' \
+  '(--rev -r --delete -d --rename -m --list -l)'{-m+,--rename=}'[rename a 
given bookmark]:bookmark:_hg_bookmarks' \
+  '(--inactive -i --delete -d --list -l)'{-l,--list}'[list existing 
bookmarks]' \
   ':bookmark:_hg_bookmarks'
 }
 



To: av6, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5145: tests: fix "running x tests using y ... " output in a few more places

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGd5ff54152636: tests: fix running x tests using y ... 
 output in a few more places (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5145?vs=12219=1

REVISION DETAIL
  https://phab.mercurial-scm.org/D5145

AFFECTED FILES
  tests/test-run-tests.t

CHANGE DETAILS

diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -173,6 +173,7 @@
 
 #if no-windows pygments
   $ rt test-failure.t --color always
+  running 1 tests using 1 parallel processes 
   
   \x1b[38;5;124m--- $TESTTMP/test-failure.t\x1b[39m (esc)
   \x1b[38;5;34m+++ $TESTTMP/test-failure.t.err\x1b[39m (esc)
@@ -192,6 +193,7 @@
   [1]
 
   $ rt test-failure.t 2> tmp.log
+  running 1 tests using 1 parallel processes 
   [1]
   $ cat tmp.log
   



To: martinvonz, #hg-reviewers, lothiraldan
Cc: lothiraldan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5141: zsh_completion: fix a couple of flags still not being perfect

2018-10-17 Thread av6 (Anton Shestakov)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG01bdf7294274: zsh_completion: fix a couple of flags still 
not being perfect (authored by av6, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5141?vs=12214=12224

REVISION DETAIL
  https://phab.mercurial-scm.org/D5141

AFFECTED FILES
  contrib/zsh_completion

CHANGE DETAILS

diff --git a/contrib/zsh_completion b/contrib/zsh_completion
--- a/contrib/zsh_completion
+++ b/contrib/zsh_completion
@@ -553,11 +553,11 @@
 _hg_cmd_bundle() {
   _arguments -s -S : $_hg_global_opts $_hg_remote_opts \
   '(--force -f)'{-f,--force}'[run even when the destination is unrelated]' \
-  '(2)*--base[a base changeset assumed to be available at the 
destination]:revision:_hg_labels' \
+  '(2)*--base=[a base changeset assumed to be available at the 
destination]:revision:_hg_labels' \
   '*'{-b+,--branch=}'[a specific branch you would like to 
bundle]:branch:_hg_branches' \
   '*'{-r+,--rev=}'[a changeset intended to be added to the 
destination]:revision:_hg_labels' \
   '(--all -a)'{-a,--all}'[bundle all changesets in the repository]' \
-  '--type[bundle compression type to use (default: bzip2)]:bundle type' \
+  '(--type -t)'{-t+,--type=}'[bundle compression type to use (default: 
bzip2)]:bundle type' \
   ':output file:_files' \
   ':destination repository:_files -/'
 }



To: av6, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5140: zsh_completion: use $_hg_remote_opts after it is defined

2018-10-17 Thread av6 (Anton Shestakov)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG4c9418e3a7d3: zsh_completion: use $_hg_remote_opts after it 
is defined (authored by av6, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5140?vs=12213=12223

REVISION DETAIL
  https://phab.mercurial-scm.org/D5140

AFFECTED FILES
  contrib/zsh_completion

CHANGE DETAILS

diff --git a/contrib/zsh_completion b/contrib/zsh_completion
--- a/contrib/zsh_completion
+++ b/contrib/zsh_completion
@@ -404,12 +404,6 @@
   '*'{-I+,--include=}'[include names matching the given 
patterns]:pattern:_files -W $(_hg_cmd root) -/'
   '*'{-X+,--exclude=}'[exclude names matching the given 
patterns]:pattern:_files -W $(_hg_cmd root) -/')
 
-_hg_clone_opts=(
-  $_hg_remote_opts
-  '(--noupdate -U)'{-U,--noupdate}'[do not update the new working directory]'
-  '--pull[use pull protocol to copy metadata]'
-  '--uncompressed[use uncompressed transfer (fast over LAN)]')
-
 _hg_date_user_opts=(
   '(--currentdate -D)'{-D,--currentdate}'[record the current date as commit 
date]'
   '(--currentuser -U)'{-U,--currentuser}'[record the current user as 
committer]'
@@ -464,6 +458,13 @@
   '--insecure[do not verify server certificate (ignoring web.cacerts config)]'
 )
 
+_hg_clone_opts=(
+  $_hg_remote_opts
+  '(--noupdate -U)'{-U,--noupdate}'[do not update the new working directory]'
+  '--pull[use pull protocol to copy metadata]'
+  '--stream[clone with minimal data processing]'
+)
+
 _hg_subrepos_opts=(
   '(--subrepos -S)'{-S,--subrepos}'[recurse into subrepositories]')
 



To: av6, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5146: histedit: import chistedit curses UI from hg-experimental

2018-10-17 Thread durin42 (Augie Fackler)
durin42 updated this revision to Diff 12221.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5146?vs=12220=12221

REVISION DETAIL
  https://phab.mercurial-scm.org/D5146

AFFECTED FILES
  hgext/histedit.py
  mercurial/ui.py

CHANGE DETAILS

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1203,7 +1203,11 @@
 "chunkselector": [
 "text",
 "curses",
-]
+],
+"histedit": [
+"text",
+"curses",
+],
 }
 
 # Feature-specific interface
diff --git a/hgext/histedit.py b/hgext/histedit.py
--- a/hgext/histedit.py
+++ b/hgext/histedit.py
@@ -183,7 +183,11 @@
 
 from __future__ import absolute_import
 
+import fcntl
+import functools
 import os
+import struct
+import termios
 
 from mercurial.i18n import _
 from mercurial import (
@@ -198,6 +202,7 @@
 extensions,
 hg,
 lock,
+logcmdutil,
 merge as mergemod,
 mergeutil,
 node,
@@ -235,6 +240,9 @@
 configitem('histedit', 'singletransaction',
 default=False,
 )
+configitem('ui', 'interface.histedit',
+default=None,
+)
 
 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' 
for
 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
@@ -916,6 +924,562 @@
 raise error.Abort(msg, hint=hint)
 return repo[roots[0]].node()
 
+# Curses Support
+try:
+import curses
+except ImportError:
+curses = None
+
+KEY_LIST = ['pick', 'edit', 'fold', 'drop', 'mess', 'roll']
+ACTION_LABELS = {
+'fold': '^fold',
+'roll': '^roll',
+}
+
+COLOR_HELP, COLOR_SELECTED, COLOR_OK, COLOR_WARN  = 1, 2, 3, 4
+
+E_QUIT, E_HISTEDIT = 1, 2
+E_PAGEDOWN, E_PAGEUP, E_LINEUP, E_LINEDOWN, E_RESIZE = 3, 4, 5, 6, 7
+MODE_INIT, MODE_PATCH, MODE_RULES, MODE_HELP = 0, 1, 2, 3
+
+KEYTABLE = {
+'global': {
+'h': 'next-action',
+'KEY_RIGHT': 'next-action',
+'l': 'prev-action',
+'KEY_LEFT':  'prev-action',
+'q': 'quit',
+'c': 'histedit',
+'C': 'histedit',
+'v': 'showpatch',
+'?': 'help',
+},
+MODE_RULES: {
+'d': 'action-drop',
+'e': 'action-edit',
+'f': 'action-fold',
+'m': 'action-mess',
+'p': 'action-pick',
+'r': 'action-roll',
+' ': 'select',
+'j': 'down',
+'k': 'up',
+'KEY_DOWN':  'down',
+'KEY_UP':'up',
+'J': 'move-down',
+'K': 'move-up',
+'KEY_NPAGE': 'move-down',
+'KEY_PPAGE': 'move-up',
+'0': 'goto',  # Used for 0..9
+},
+MODE_PATCH: {
+' ': 'page-down',
+'KEY_NPAGE': 'page-down',
+'KEY_PPAGE': 'page-up',
+'j': 'line-down',
+'k': 'line-up',
+'KEY_DOWN':  'line-down',
+'KEY_UP':'line-up',
+'J': 'down',
+'K': 'up',
+},
+MODE_HELP: {
+},
+}
+
+def screen_size():
+return struct.unpack('hh', fcntl.ioctl(1, termios.TIOCGWINSZ, ''))
+
+class histeditrule(object):
+def __init__(self, ctx, pos, action='pick'):
+self.ctx = ctx
+self.action = action
+self.origpos = pos
+self.pos = pos
+self.conflicts = []
+
+def __str__(self):
+# Some actions ('fold' and 'roll') combine a patch with a previous one.
+# Add a marker showing which patch they apply to, and also omit the
+# description for 'roll' (since it will get discarded). Example 
display:
+#
+#  #10 pick   316392:06a16c25c053   add option to skip tests
+#  #11 ^roll  316393:71313c964cc5
+#  #12 pick   316394:ab31f3973b0d   include mfbt for mozilla-config.h
+#  #13 ^fold  316395:14ce5803f4c3   fix warnings
+#
+# The carets point to the changeset being folded into ("roll this
+# changeset into the changeset above").
+action = ACTION_LABELS.get(self.action, self.action)
+h = self.ctx.hex()[0:12]
+r = self.ctx.rev()
+desc = self.ctx.description().splitlines()[0].strip()
+if self.action == 'roll':
+desc = ''
+return "#{0:<2} {1:<6} {2}:{3}   {4}".format(
+self.origpos, action, r, h, desc)
+
+def checkconflicts(self, other):
+if other.pos > self.pos and other.origpos <= self.origpos:
+if set(other.ctx.files()) & set(self.ctx.files()) != set():
+self.conflicts.append(other)
+return self.conflicts
+
+if other in self.conflicts:
+self.conflicts.remove(other)
+return self.conflicts
+
+#  EVENTS ===
+def movecursor(state, oldpos, newpos):
+

D5146: histedit: import chistedit curses UI from hg-experimental

2018-10-17 Thread durin42 (Augie Fackler)
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  I don't tend to like curses interfaces, but this gets enough use at
  work that it seems like it's worth bringing into core. This is a
  minimal import from hg-experimental revision 
https://phab.mercurial-scm.org/rFBHGX4c7f33bf5f00b91c2caf28f095c320873584dd34, 
in that
  I've done the smallest amount of code movement and editing in order to
  import the functionality.
  
  .. feature::
  
`hg histedit` will now present a curses UI if curses is available
and `ui.interface` or `ui.interface.histedit` is set to `curses`.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5146

AFFECTED FILES
  hgext/histedit.py
  mercurial/ui.py

CHANGE DETAILS

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1203,7 +1203,11 @@
 "chunkselector": [
 "text",
 "curses",
-]
+],
+"histedit": [
+"text",
+"curses",
+],
 }
 
 # Feature-specific interface
diff --git a/hgext/histedit.py b/hgext/histedit.py
--- a/hgext/histedit.py
+++ b/hgext/histedit.py
@@ -183,6 +183,7 @@
 
 from __future__ import absolute_import
 
+import functools
 import os
 
 from mercurial.i18n import _
@@ -916,6 +917,565 @@
 raise error.Abort(msg, hint=hint)
 return repo[roots[0]].node()
 
+# Curses Support
+try:
+import curses
+except ImportError:
+curses = None
+
+KEY_LIST = ['pick', 'edit', 'fold', 'drop', 'mess', 'roll']
+ACTION_LABELS = {
+'fold': '^fold',
+'roll': '^roll',
+}
+
+COLOR_HELP, COLOR_SELECTED, COLOR_OK, COLOR_WARN  = 1, 2, 3, 4
+
+E_QUIT, E_HISTEDIT = 1, 2
+E_PAGEDOWN, E_PAGEUP, E_LINEUP, E_LINEDOWN, E_RESIZE = 3, 4, 5, 6, 7
+MODE_INIT, MODE_PATCH, MODE_RULES, MODE_HELP = 0, 1, 2, 3
+
+KEYTABLE = {
+'global': {
+'h': 'next-action',
+'KEY_RIGHT': 'next-action',
+'l': 'prev-action',
+'KEY_LEFT':  'prev-action',
+'q': 'quit',
+'c': 'histedit',
+'C': 'histedit',
+'v': 'showpatch',
+'?': 'help',
+},
+MODE_RULES: {
+'d': 'action-drop',
+'e': 'action-edit',
+'f': 'action-fold',
+'m': 'action-mess',
+'p': 'action-pick',
+'r': 'action-roll',
+' ': 'select',
+'j': 'down',
+'k': 'up',
+'KEY_DOWN':  'down',
+'KEY_UP':'up',
+'J': 'move-down',
+'K': 'move-up',
+'KEY_NPAGE': 'move-down',
+'KEY_PPAGE': 'move-up',
+'0': 'goto',  # Used for 0..9
+},
+MODE_PATCH: {
+' ': 'page-down',
+'KEY_NPAGE': 'page-down',
+'KEY_PPAGE': 'page-up',
+'j': 'line-down',
+'k': 'line-up',
+'KEY_DOWN':  'line-down',
+'KEY_UP':'line-up',
+'J': 'down',
+'K': 'up',
+},
+MODE_HELP: {
+},
+}
+
+def screen_size():
+import termios
+from fcntl import ioctl
+from struct import unpack
+return unpack('hh', ioctl(1, termios.TIOCGWINSZ, ''))
+
+class histeditrule(object):
+def __init__(self, ctx, pos, action='pick'):
+self.ctx = ctx
+self.action = action
+self.origpos = pos
+self.pos = pos
+self.conflicts = []
+
+def __str__(self):
+# Some actions ('fold' and 'roll') combine a patch with a previous one.
+# Add a marker showing which patch they apply to, and also omit the
+# description for 'roll' (since it will get discarded). Example 
display:
+#
+#  #10 pick   316392:06a16c25c053   add option to skip tests
+#  #11 ^roll  316393:71313c964cc5
+#  #12 pick   316394:ab31f3973b0d   include mfbt for mozilla-config.h
+#  #13 ^fold  316395:14ce5803f4c3   fix warnings
+#
+# The carets point to the changeset being folded into ("roll this
+# changeset into the changeset above").
+action = ACTION_LABELS.get(self.action, self.action)
+h = self.ctx.hex()[0:12]
+r = self.ctx.rev()
+desc = self.ctx.description().splitlines()[0].strip()
+if self.action == 'roll':
+desc = ''
+return "#{0:<2} {1:<6} {2}:{3}   {4}".format(
+self.origpos, action, r, h, desc)
+
+def checkconflicts(self, other):
+if other.pos > self.pos and other.origpos <= self.origpos:
+if set(other.ctx.files()) & set(self.ctx.files()) != set():
+self.conflicts.append(other)
+return self.conflicts
+
+if other in self.conflicts:
+self.conflicts.remove(other)
+

D5145: tests: fix "running x tests using y ... " output in a few more places

2018-10-17 Thread lothiraldan (Boris Feld)
lothiraldan accepted this revision.
lothiraldan added a comment.


  LGTM

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5145

To: martinvonz, #hg-reviewers, lothiraldan
Cc: lothiraldan, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


I2P anonymous protocol want to use Mercurial

2018-10-17 Thread za3ter
Hi There,

im from I2P community and our developers were using monotone in order to 
develop I2P. 

Sadly the project died since 4 years ago , and we looking for alternatives. 

The coniditons to migrate to new development tool are these:

- supports resume
- key signed checkins
- revocation of keys/trust function implementation
- easy migration of mtn history into new system

Does mercurial support these options? if not, is it on the road map? if not 
then why not?

Thank You!

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5145: tests: fix "running x tests using y ... " output in a few more places

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  These seem to have been missed by 
https://phab.mercurial-scm.org/rHG1039404c5e1d12f7ee3b09eedde803f52f8b369a 
(run-tests: print
  number of tests and parallel process count, 2018-10-13).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5145

AFFECTED FILES
  tests/test-run-tests.t

CHANGE DETAILS

diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -173,6 +173,7 @@
 
 #if no-windows pygments
   $ rt test-failure.t --color always
+  running 1 tests using 1 parallel processes 
   
   \x1b[38;5;124m--- $TESTTMP/test-failure.t\x1b[39m (esc)
   \x1b[38;5;34m+++ $TESTTMP/test-failure.t.err\x1b[39m (esc)
@@ -192,6 +193,7 @@
   [1]
 
   $ rt test-failure.t 2> tmp.log
+  running 1 tests using 1 parallel processes 
   [1]
   $ cat tmp.log
   



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5144: subrepo: access status members by name instead of by position

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Taking my first Mercurial project closer to completion.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5144

AFFECTED FILES
  mercurial/subrepo.py

CHANGE DETAILS

diff --git a/mercurial/subrepo.py b/mercurial/subrepo.py
--- a/mercurial/subrepo.py
+++ b/mercurial/subrepo.py
@@ -1583,17 +1583,15 @@
 if self._gitmissing():
 return []
 
-(modified, added, removed,
- deleted, unknown, ignored, clean) = self.status(None, unknown=True,
- clean=True)
+s = self.status(None, unknown=True, clean=True)
 
 tracked = set()
 # dirstates 'amn' warn, 'r' is added again
-for l in (modified, added, deleted, clean):
+for l in (s.modified, s.added, s.deleted, s.clean):
 tracked.update(l)
 
 # Unknown files not of interest will be rejected by the matcher
-files = unknown
+files = s.unknown
 files.extend(match.files())
 
 rejected = []



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4895: narrow: when widening, don't include manifests the client already has

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz added a comment.


  In https://phab.mercurial-scm.org/D4895#76913, @martinvonz wrote:
  
  > In https://phab.mercurial-scm.org/D4895#76912, @pulkit wrote:
  >
  > > I have tested this and it works well with our internal repo too. Thanks!
  > >
  > > We already have some tests and none fails, do you want to add more tests 
or I can queue this?
  >
  >
  > I should at least update the commit message. I'll try to do that ~now.
  
  
  Done. I also added an extra test.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4895

To: martinvonz, durin42, #hg-reviewers
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4895: narrow: when widening, don't include manifests the client already has

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz updated this revision to Diff 12217.
martinvonz edited the summary of this revision.
martinvonz retitled this revision from "narrow: don't include manifests the 
client already has" to "narrow: when widening, don't include manifests the 
client already has".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4895?vs=12184=12217

REVISION DETAIL
  https://phab.mercurial-scm.org/D4895

AFFECTED FILES
  hgext/narrow/narrowbundle2.py
  hgext/narrow/narrowwirepeer.py
  mercurial/bundle2.py
  mercurial/changegroup.py
  mercurial/exchange.py

CHANGE DETAILS

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -2145,14 +2145,12 @@
 if kwargs.get(r'narrow', False):
 include = sorted(filter(bool, kwargs.get(r'includepats', [])))
 exclude = sorted(filter(bool, kwargs.get(r'excludepats', [])))
-filematcher = narrowspec.match(repo.root, include=include,
-   exclude=exclude)
+matcher = narrowspec.match(repo.root, include=include, exclude=exclude)
 else:
-filematcher = None
+matcher = None
 
 cgstream = changegroup.makestream(repo, outgoing, version, source,
-  bundlecaps=bundlecaps,
-  filematcher=filematcher)
+  bundlecaps=bundlecaps, matcher=matcher)
 
 part = bundler.newpart('changegroup', data=cgstream)
 if cgversions:
diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -727,14 +727,17 @@
 progress.complete()
 
 class cgpacker(object):
-def __init__(self, repo, filematcher, version,
+def __init__(self, repo, oldmatcher, matcher, version,
  builddeltaheader, manifestsend,
  forcedeltaparentprev=False,
  bundlecaps=None, ellipses=False,
  shallow=False, ellipsisroots=None, fullnodes=None):
 """Given a source repo, construct a bundler.
 
-filematcher is a matcher that matches on files to include in the
+oldmatcher is a matcher that matches on files the client already has.
+These will not be included in the changegroup.
+
+matcher is a matcher that matches on files to include in the
 changegroup. Used to facilitate sparse changegroups.
 
 forcedeltaparentprev indicates whether delta parents must be against
@@ -761,8 +764,10 @@
 ellipsis because for very large histories we expect this to be
 significantly smaller.
 """
-assert filematcher
-self._filematcher = filematcher
+assert oldmatcher
+assert matcher
+self._oldmatcher = oldmatcher
+self._matcher = matcher
 
 self.version = version
 self._forcedeltaparentprev = forcedeltaparentprev
@@ -1027,7 +1032,7 @@
 tree, nodes = tmfnodes.popitem()
 store = mfl.getstorage(tree)
 
-if not self._filematcher.visitdir(store.tree[:-1] or '.'):
+if not self._matcher.visitdir(store.tree[:-1] or '.'):
 # No nodes to send because this directory is out of
 # the client's view of the repository (probably
 # because of narrow clones).
@@ -1051,7 +1056,16 @@
 fullclnodes=self._fullclnodes,
 precomputedellipsis=self._precomputedellipsis)
 
-yield tree, deltas
+if not self._oldmatcher.visitdir(store.tree[:-1] or '.'):
+yield tree, deltas
+else:
+# 'deltas' is a generator and we need to consume it even if
+# we are not going to send it because a side-effect is that
+# it updates tmdnodes (via lookupfn)
+for d in deltas:
+pass
+if not tree:
+yield tree, []
 
 def _prunemanifests(self, store, nodes, commonrevs):
 # This is split out as a separate method to allow filtering
@@ -1066,7 +1080,8 @@
 # The 'source' parameter is useful for extensions
 def generatefiles(self, changedfiles, commonrevs, source,
   mfdicts, fastpathlinkrev, fnodes, clrevs):
-changedfiles = list(filter(self._filematcher, changedfiles))
+changedfiles = [f for f in changedfiles
+if self._matcher(f) and not self._oldmatcher(f)]
 
 if not fastpathlinkrev:
 def normallinknodes(unused, fname):
@@ -1151,12 +1166,13 @@
 
 progress.complete()
 
-def _makecg1packer(repo, filematcher, bundlecaps, ellipses=False,
-   shallow=False, ellipsisroots=None, fullnodes=None):
+def _makecg1packer(repo, oldmatcher, matcher, bundlecaps,
+   ellipses=False, shallow=False, ellipsisroots=None,
+   

D5143: tests: add test for widening from an empty clone

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Narrow clones that track no paths currently don't even include the
  root manifest (which is the only manifest when using flat
  manifests). That means that when we widen from such a clone, we need
  to make sure that we send the root manifest (and other manifests if
  using tree manifests). That currently works because we always resend
  all manifest that match the new narrowspec. However, we're about to
  stop resending manifests that the client already has and there's a
  risk of this breaking then, so let's add a test.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5143

AFFECTED FILES
  tests/test-narrow-widen-no-ellipsis.t

CHANGE DETAILS

diff --git a/tests/test-narrow-widen-no-ellipsis.t 
b/tests/test-narrow-widen-no-ellipsis.t
--- a/tests/test-narrow-widen-no-ellipsis.t
+++ b/tests/test-narrow-widen-no-ellipsis.t
@@ -30,17 +30,30 @@
 
 narrow clone the inside file
 
-  $ hg clone --narrow ssh://user@dummy/master narrow --include inside
+  $ hg clone --narrow ssh://user@dummy/master narrow
   requesting all changes
   adding changesets
   adding manifests
   adding file changes
-  added 3 changesets with 1 changes to 1 files
+  added 3 changesets with 0 changes to 0 files
   new changesets *:* (glob)
   updating to branch default
-  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ cd narrow
   $ hg tracked
+  $ hg files
+  [1]
+
+widen from an empty clone
+
+  $ hg tracked --addinclude inside
+  comparing with ssh://user@dummy/master
+  searching for changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 0 changesets with 1 changes to 1 files
+  $ hg tracked
   I path:inside
   $ ls
   inside
@@ -137,7 +150,6 @@
   adding file changes
   added 5 changesets with 4 changes to 2 files
   new changesets *:* (glob)
-  3 local changesets published
   (run 'hg update' to get a working copy)
   $ hg update -r 'desc("add wider")'
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved



To: martinvonz, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


Rust extensions: the next step

2018-10-17 Thread Georges Racinet
Hi all,

first, many thanks for the Stockholm sprint, it was my first interaction
with the Mercurial community, and it's been very welcoming to me.

I've been pursuing some experiments I started then to convert the Rust
bindings I've done in the patch series about ancestry iteration (now
landed) to a proper Python extension, using the cpython crate and Python
capsules. In short, it works.

Early benchmarking shows that it's a few percent slower than the direct
bindings through C code, which I think is acceptable compared to the
other benefits (clearer integration, easier to generalise, no C code at
all).

The end result is a unique shared library importable as
'mercurial.rustext', which is itself made of several submodules, ie, one
can do:

   from mercurial.rustext.ancestor import AncestorsIterator

It will take me some more time, though, to get that experiment into a
reviewable state (have to switch soon to other, unrelated, works) and
we're too close to the freeze anyway, but if someone wants to see it, I
can share it right away.

Also, I could summarize some of these thoughts on the Oxidation wiki
page. Greg, are you okay with that ?

Regards,

-- 
Georges Racinet
Anybox SAS, http://anybox.fr
Téléphone: +33 6 51 32 07 27
GPG: B59E 22AB B842 CAED 77F7 7A7F C34F A519 33AB 0A35, sur serveurs publics


___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4895: narrow: don't include manifests the client already has

2018-10-17 Thread martinvonz (Martin von Zweigbergk)
martinvonz added a comment.


  In https://phab.mercurial-scm.org/D4895#76912, @pulkit wrote:
  
  > I have tested this and it works well with our internal repo too. Thanks!
  >
  > We already have some tests and none fails, do you want to add more tests or 
I can queue this?
  
  
  I should at least update the commit message. I'll try to do that ~now.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4895

To: martinvonz, durin42, #hg-reviewers
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4895: narrow: don't include manifests the client already has

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit added a comment.


  I have tested this and it works well with our internal repo too. Thanks!
  
  We already have some tests and none fails, do you want to add more tests or I 
can queue this?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4895

To: martinvonz, durin42, #hg-reviewers
Cc: pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[PATCH] phase: add an archived phases

2018-10-17 Thread Boris Feld
# HG changeset patch
# User Boris Feld 
# Date 1539780421 -7200
#  Wed Oct 17 14:47:01 2018 +0200
# Node ID 12dd6c5ad6ba17fa4f98f4009bc0b53a7f7f5f8b
# Parent  5644f7c8982e805e53f56fcbfe0322e9de58a934
# EXP-Topic archived-phase
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
12dd6c5ad6ba
phase: add an archived phases

This phase allows for hidden changesets in the "user space". It differs from
the "internal" phase which is intended for internal by-product only. There
have been discussions at the 4.8 sprint to use such phase to speedup cleanup
after history rewriting operation.

Shipping it in the same release as the 'internal-phase' groups the associated
`requires` entry. The important bit is to have support for this phase in the
earliest version of mercurial possible. Adding the UI to manipulate this new
phase later seems fine.

The current plan for archived usage and user interface are as follow. On a
repository with internal-phase on and evolution off:

* history rewriting command set rewritten changeset in the archived phase.
  (This mean updating the cleanupnodes method).
* keep `hg unbundle .hg/strip-backup/X.hg` as a way to restore changeset for
  now
  (backup bundle need to contains phase data)
* [maybe] add a `hg strip --soft` advance flag
  (a light way to expose the feature without getting in the way of a better
  UI)

Mercurial 4.8 freeze is too close to get the above in by then. However, having
a consistency `requirement` meaning would be nice.

diff --git a/mercurial/phases.py b/mercurial/phases.py
--- a/mercurial/phases.py
+++ b/mercurial/phases.py
@@ -129,11 +129,13 @@ HIDEABLE_FLAG = 32 # Phases that are hid
 # record phase index
 public, draft, secret = range(3)
 internal = INTERNAL_FLAG | HIDEABLE_FLAG
+archived = HIDEABLE_FLAG
 allphases = range(internal + 1)
 trackedphases = allphases[1:]
 # record phase names
 phasenames = [None] * len(allphases)
 phasenames[:3] = ['public', 'draft', 'secret']
+phasenames[archived] = 'archived'
 phasenames[internal] = 'internal'
 # record phase property
 mutablephases = tuple(allphases[1:])
@@ -446,8 +448,9 @@ class phasecache(object):
 def _retractboundary(self, repo, tr, targetphase, nodes):
 # Be careful to preserve shallow-copied values: do not update
 # phaseroots values, replace them.
-if targetphase == internal and not supportinternal(repo):
-msg = 'this repository does not support the internal phase'
+if targetphase in (archived, internal) and not supportinternal(repo):
+name = phasenames[targetphase]
+msg = 'this repository does not support the %s phase' % name
 raise error.ProgrammingError(msg)
 
 repo = repo.unfiltered()
diff --git a/tests/test-phases.t b/tests/test-phases.t
--- a/tests/test-phases.t
+++ b/tests/test-phases.t
@@ -850,6 +850,10 @@ Check we deny its usage on older reposit
   ** ProgrammingError: this repository does not support the internal phase
   raise error.ProgrammingError(msg)
   mercurial.error.ProgrammingError: this repository does not support the 
internal phase
+  $ hg --config "phases.new-commit=archived" commit -m "my test archived 
commit" 2>&1 | grep ProgrammingError
+  ** ProgrammingError: this repository does not support the archived phase
+  raise error.ProgrammingError(msg)
+  mercurial.error.ProgrammingError: this repository does not support the 
archived phase
 
   $ cd ..
 
@@ -878,7 +882,8 @@ Commit an internal changesets
   test-debug-phase: new rev 1:  x -> 96
   test-hook-close-phase: c01c42dffc7f81223397e99652a0703f83e1c5ea:   -> 
internal
 
-Usual visibility rules apply when working directory parents
+The changeset is a working parent descendant.
+Per the usual visibility rules, it is made visible.
 
   $ hg log -G -l 3
   @  changeset:   1:c01c42dffc7f
@@ -904,3 +909,45 @@ Commit is hidden as expected
  date:Thu Jan 01 00:00:00 1970 +
  summary: A
   
+
+Test for archived phase
+---
+
+Commit an archived changesets
+
+  $ echo B > B
+  $ hg add B
+  $ hg status
+  A B
+  $ hg --config "phases.new-commit=archived" commit -m "my test archived 
commit"
+  test-debug-phase: new rev 2:  x -> 32
+  test-hook-close-phase: 8df5997c3361518f733d1ae67cd3adb9b0eaf125:   -> 
archived
+
+The changeset is a working parent descendant.
+Per the usual visibility rules, it is made visible.
+
+  $ hg log -G -l 3
+  @  changeset:   2:8df5997c3361
+  |  tag: tip
+  |  parent:  0:4a2df7238c3b
+  |  user:test
+  |  date:Thu Jan 01 00:00:00 1970 +
+  |  summary: my test archived commit
+  |
+  o  changeset:   0:4a2df7238c3b
+ user:test
+ date:Thu Jan 01 00:00:00 1970 +
+ summary: A
+  
+
+Commit is hidden as expected
+
+  $ hg up 0
+  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
+  $ hg log -G
+  

D5142: zsh_completion: add -l/--list flag for hg bookmarks completion

2018-10-17 Thread av6 (Anton Shestakov)
av6 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Flags in parentheses are mutually exclusive. Logic is taken from commands.py:
  
selactions = [k for k in ['delete', 'rename', 'list'] if opts.get(k)]
if len(selactions) > 1:
raise error.Abort(_('--%s and --%s are incompatible')
  % tuple(selactions[:2]))

...

if rev and action in {'delete', 'rename', 'list'}:
raise error.Abort(_("--rev is incompatible with --%s") % action)
if inactive and action in {'delete', 'list'}:
raise error.Abort(_("--inactive is incompatible with --%s") % action)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5142

AFFECTED FILES
  contrib/zsh_completion

CHANGE DETAILS

diff --git a/contrib/zsh_completion b/contrib/zsh_completion
--- a/contrib/zsh_completion
+++ b/contrib/zsh_completion
@@ -532,10 +532,11 @@
 _hg_cmd_bookmarks() {
   _arguments -s -S : $_hg_global_opts \
   '(--force -f)'{-f,--force}'[force]' \
-  '(--inactive -i)'{-i,--inactive}'[mark a bookmark inactive]' \
-  '(--rev -r --delete -d --rename 
-m)'{-r+,--rev=}'[revision]:revision:_hg_labels' \
-  '(--rev -r --delete -d --rename -m)'{-d,--delete}'[delete a given bookmark]' 
\
-  '(--rev -r --delete -d --rename -m)'{-m+,--rename=}'[rename a given 
bookmark]:bookmark:_hg_bookmarks' \
+  '(--inactive -i --delete -d --list -l)'{-i,--inactive}'[mark a bookmark 
inactive]' \
+  '(--rev -r --delete -d --rename -m --list 
-l)'{-r+,--rev=}'[revision]:revision:_hg_labels' \
+  '(--rev -r --delete -d --rename -m --list -l --inactive 
-i)'{-d,--delete}'[delete a given bookmark]' \
+  '(--rev -r --delete -d --rename -m --list -l)'{-m+,--rename=}'[rename a 
given bookmark]:bookmark:_hg_bookmarks' \
+  '(--inactive -i --delete -d --list -l)'{-l,--list}'[list existing 
bookmarks]' \
   ':bookmark:_hg_bookmarks'
 }
 



To: av6, #hg-reviewers
Cc: mercurial-devel, spectral
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5141: zsh_completion: fix a couple of flags still not being perfect

2018-10-17 Thread av6 (Anton Shestakov)
av6 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5141

AFFECTED FILES
  contrib/zsh_completion

CHANGE DETAILS

diff --git a/contrib/zsh_completion b/contrib/zsh_completion
--- a/contrib/zsh_completion
+++ b/contrib/zsh_completion
@@ -553,11 +553,11 @@
 _hg_cmd_bundle() {
   _arguments -s -S : $_hg_global_opts $_hg_remote_opts \
   '(--force -f)'{-f,--force}'[run even when the destination is unrelated]' \
-  '(2)*--base[a base changeset assumed to be available at the 
destination]:revision:_hg_labels' \
+  '(2)*--base=[a base changeset assumed to be available at the 
destination]:revision:_hg_labels' \
   '*'{-b+,--branch=}'[a specific branch you would like to 
bundle]:branch:_hg_branches' \
   '*'{-r+,--rev=}'[a changeset intended to be added to the 
destination]:revision:_hg_labels' \
   '(--all -a)'{-a,--all}'[bundle all changesets in the repository]' \
-  '--type[bundle compression type to use (default: bzip2)]:bundle type' \
+  '(--type -t)'{-t+,--type=}'[bundle compression type to use (default: 
bzip2)]:bundle type' \
   ':output file:_files' \
   ':destination repository:_files -/'
 }



To: av6, #hg-reviewers
Cc: mercurial-devel, spectral
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5140: zsh_completion: use $_hg_remote_opts after it is defined

2018-10-17 Thread av6 (Anton Shestakov)
av6 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Before this patch, zsh wouldn't complete --ssh, --remotecmd or --insecure for
  hg clone.
  
  While at it, replace --uncompressed by --stream.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5140

AFFECTED FILES
  contrib/zsh_completion

CHANGE DETAILS

diff --git a/contrib/zsh_completion b/contrib/zsh_completion
--- a/contrib/zsh_completion
+++ b/contrib/zsh_completion
@@ -404,12 +404,6 @@
   '*'{-I+,--include=}'[include names matching the given 
patterns]:pattern:_files -W $(_hg_cmd root) -/'
   '*'{-X+,--exclude=}'[exclude names matching the given 
patterns]:pattern:_files -W $(_hg_cmd root) -/')
 
-_hg_clone_opts=(
-  $_hg_remote_opts
-  '(--noupdate -U)'{-U,--noupdate}'[do not update the new working directory]'
-  '--pull[use pull protocol to copy metadata]'
-  '--uncompressed[use uncompressed transfer (fast over LAN)]')
-
 _hg_date_user_opts=(
   '(--currentdate -D)'{-D,--currentdate}'[record the current date as commit 
date]'
   '(--currentuser -U)'{-U,--currentuser}'[record the current user as 
committer]'
@@ -464,6 +458,13 @@
   '--insecure[do not verify server certificate (ignoring web.cacerts config)]'
 )
 
+_hg_clone_opts=(
+  $_hg_remote_opts
+  '(--noupdate -U)'{-U,--noupdate}'[do not update the new working directory]'
+  '--pull[use pull protocol to copy metadata]'
+  '--stream[clone with minimal data processing]'
+)
+
 _hg_subrepos_opts=(
   '(--subrepos -S)'{-S,--subrepos}'[recurse into subrepositories]')
 



To: av6, #hg-reviewers
Cc: mercurial-devel, spectral
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5137: streamclone: new server config and some API changes for narrow stream clones

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit added a comment.


  In https://phab.mercurial-scm.org/D5137#76876, @indygreg wrote:
  
  > There are no children reviews of this one. Could you please set the 
parent-child relationships so the entire stack renders in the web UI?
  >
  > (I'm assuming there are commits that follow this one - an unimplemented 
narrow stream clone feature doesn't seem very useful!)
  
  
  Should be good now.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

To: pulkit, durin42, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5137: streamclone: new server config and some API changes for narrow stream clones

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg added a comment.


  There are no children reviews of this one. Could you please set the 
parent-child relationships so the entire stack renders in the web UI?
  
  (I'm assuming there are commits that follow this one - an unimplemented 
narrow stream clone feature doesn't seem very useful!)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

To: pulkit, durin42, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5137: streamclone: new server config and some API changes for narrow stream clones

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit added a comment.


  In https://phab.mercurial-scm.org/D5137#76870, @indygreg wrote:
  
  > I'll hold off formally accepting until the whole series is up. But this 
looks pretty good.
  >
  > We need to land this in 4.8 or less the client-side feature detection logic 
will require a new server capability (or a bump of the version of the 
capability string).
  
  
  The whole series is up now. Although this is important for us, I realize 
there is just 1 day left for the freeze and I won't push hard for getting this 
pushed in this cycle and will graft these csets if looks good. I know you are 
on a vacation, so thanks for taking time and reviewing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

To: pulkit, durin42, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5137: streamclone: new server config and some API changes for narrow stream clones

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg added a comment.


  I'll hold off formally accepting until the whole series is up. But this looks 
pretty good.
  
  We need to land this in 4.8 or less the client-side feature detection logic 
will require a new server capability (or a bump of the version of the 
capability string).

INLINE COMMENTS

> configitems.py:1014
>  )
> +coreconfigitem('server', 'stream-narrow-clones',
> +default=False,

Perhaps we should mark this as experimental via an inline comment and forego 
documenting the config option for now?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

To: pulkit, durin42, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5094: merge-tools: when calling external merge tool, describe the resolve inputs

2018-10-17 Thread spectral (Kyle Lippincott)
spectral planned changes to this revision.
spectral added a comment.


  In https://phab.mercurial-scm.org/D5094#76457, @yuja wrote:
  
  > > +def _describemerge(ui, repo, env, toolpath, args):
  > >  +template = ui.config('ui', 'pre-merge-tool-output-template')
  > >  +if not template:
  > >  +return
  > >  +
  > >  +# Remove HG_ prefix from entries in `env` and lowercase them
  > >  +def sanitizeenv(k):
  > >  +if k.startswith('HG_'):
  > >  +return k[3:].lower()
  > >  +return k
  >
  > Nit: `HG_FILE` should be called as `path` per
  >  https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary
  
  
  Done.
  
  > We might want to structure these variables as `base.node|short` instead of
  >  `base_node` for example, but that would require more work.
  
  I tried to do this, producing this json output:
  
{
 "base": {"islink": false, "label": "base", "name": "base", "node": 
"TZ>fP͏~\u0019��%`��z3��)"},
 "local": {"islink": false, "label": "working copy", "name": "local", 
"node": "ot���G-������xq���\u0001��o1c��"},
 "other": {"islink": false, "label": "merge rev", "name": "other", "node": 
" ���`���\u0013s_\u0011��0��\u0002�K��"},
 "path": "mercurial/dispatch.py",
 "toolargs": 
"/usr/local/google/home/spectral/src/hg/hg_work/mercurial/dispatch.py 
'/usr/local/google/tmp/tmp/dispatch~base.erx9yG.py' 
'/usr/local/google/tmp/tmp/dispatch~other.gbxRqg.py'",
 "toolpath": "/bin/false"
}
  
  Unfortunately, this doesn't work super well, when using the following flags:
  
--config ui.pre-merge-tool-output-template='{myfunc(local)}' --config 
'templatealias.myfunc(d)="{d.label}"'
  
  I get this error:
  
hg: parse error: {'node': 
'ot\xc4G-\xd0\xc3\x92\xdcxq\x97\x01\x90\xdbo1c\xb1\x95', 'name': 'local', 
'islink': False, 'label': 'working copy'} is not a dictionary
(keyword 'local' does not support member operation)
  
  I can fix this with a pretty simple patch to templateutil's makewrapped, 
making it return hybriddict or hybridlist if it receives a dict or a list.  I 
couldn't find any other similar ways of causing this error, though, so it's 
possible I'm doing something wrong?
  
  Things I tried:
  
≻ hg log -r . -T'{myfunc(dict(k1="hi", k2="bye"))}' --config 
'templatealias.myfunc(d)="{d.k1}"'
hi

≻ hg log -r . -T'{myfunc(namespaces)}' --config 
'templatealias.myfunc(d)="{d.tags}"'
tip
  
  I looked through all functions that accept 'formatteropts' and couldn't find 
any hg command that produces a dict that's *not* in a list.  I also couldn't 
identify any way of interacting with these dicts-inside-of-lists as a single 
unit (i.e. `hg version -T'{extensions%"{extension|json}"}'` doesn't work: 
'extension' isn't a way of referring to the dictionary that's in the 
'extensions' list.  I have to do something like `hg version 
-T'{extensions%dict(name, ver, bundled)|json)}'` to re-create the dictionary.)
  
  >> +data = {sanitizeenv(k): v for k, v in env.items()}
  >>  +data['toolpath'] = toolpath
  >>  +data['toolargs'] = args
  >>  +
  >>  +# TODO: make all of this something that can be specified on a 
per-tool basis
  >>  +template = templater.unquotestring(template)
  >>  +
  >>  +fm = ui.formatter("extmerge", pycompat.byteskwargs({'template': 
template}))
  > 
  >   
  > 
  > Unnecessary. It's bytes dict.
  
  Done.
  
  > 
  > 
  >> +fm.data(repo=repo, **data)
  > 
  >   ^^
  > 
  > Here pycompat.strkwargs() would be needed.
  
  Done.
  
  > And use `fm.context(repo=repo)` or `fm.context(fctx=fcd)` since `repo` is
  >  unprintable.
  
  Done. Did both (fm.context(repo=repo, fctx=fcl))
  
  > 
  > 
  >> +coreconfigitem('ui', 'pre-merge-tool-output-template',
  >>  +default=None,
  >>  +)
  > 
  > Can you update the help/config.txt as well?
  
  Will do before sending final version (marking as 'changes planned' for now - 
I'm going to be unavailable for several days).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5094

To: spectral, #hg-reviewers
Cc: yuja, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5139: store: introduce _matchtrackedpath() and use it to filter store files

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This patch introduces a function to filter store files on the basis of the 
path
  which they are tracking.
  
  The function assumes that the entries can be of two types, 'meta/*' and 
'data/*'
  which means it will just work on revlog based storage and not with another
  storage ways.
  
  For the 'data/*' entries, we remove the 'data/' part and '.i/.d' part from the
  beginning and the end then pass that to matcher.
  
  For the 'meta/*' entries, we remove the 'meta/' and '/00manifest.(i/d)' part 
from
  beginning and end then call matcher.visitdir() with it to make sure all the
  parent directories are also downloaded.
  
  Since the storage filtering for narrow stream clones is implemented with this
  patch, we remove the un-implemented error message, add some more tests and add
  the treemanifest case to tests too.
  
  The tests demonstrate that it works correctly.
  
  After this patch, we have now narrow stream clones working. Narrow stream 
clones
  are a very important feature for large repositories who have good internet
  connection because they use streamclones for cloning and if they do normal
  narrow clone, that takes more time then a full streamclone. Also narrow-stream
  clone will drastically speed up clone timings.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5139

AFFECTED FILES
  mercurial/store.py
  mercurial/streamclone.py
  tests/test-narrow-clone-stream.t

CHANGE DETAILS

diff --git a/tests/test-narrow-clone-stream.t b/tests/test-narrow-clone-stream.t
--- a/tests/test-narrow-clone-stream.t
+++ b/tests/test-narrow-clone-stream.t
@@ -1,7 +1,16 @@
+#testcases tree flat
+
 Tests narrow stream clones
 
   $ . "$TESTDIR/narrow-library.sh"
 
+#if tree
+  $ cat << EOF >> $HGRCPATH
+  > [experimental]
+  > treemanifest = 1
+  > EOF
+#endif
+
 Server setup
 
   $ hg init master
@@ -34,6 +43,44 @@
 
   $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include 
"dir/src/f10" --stream
   streaming all changes
-  remote: abort: support for narrow stream clones is missing
-  abort: pull failed on remote
-  [255]
+  * files to transfer, * KB of data (glob)
+  transferred * KB in * seconds (* MB/sec) (glob)
+
+  $ cd narrow
+  $ ls
+  $ hg tracked
+  I path:dir/src/f10
+
+Making sure we have the correct set of requirements
+
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  narrowhg-experimental
+  revlogv1
+  store
+  treemanifest (tree !)
+
+Making sure store has the required files
+
+  $ ls .hg/store/
+  00changelog.i
+  00manifest.i
+  data
+  fncache
+  meta (tree !)
+  narrowspec
+  undo
+  undo.backupfiles
+  undo.phaseroots
+
+Checking that repository has all the required data and not broken
+
+  $ hg verify
+  checking changesets
+  checking manifests
+  checking directory manifests (tree !)
+  crosschecking files in changesets and manifests
+  checking files
+  checked 40 changesets with 1 changes to 1 files
diff --git a/mercurial/streamclone.py b/mercurial/streamclone.py
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -545,10 +545,6 @@
 Returns a 3-tuple of (file count, file size, data iterator).
 """
 
-# temporarily raise error until we add storage level logic
-if includes or excludes:
-raise error.Abort(_("support for narrow stream clones is missing"))
-
 with repo.lock():
 
 entries = []
diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -24,6 +24,20 @@
 
 parsers = policy.importmod(r'parsers')
 
+def _matchtrackedpath(path, matcher):
+"""parses a fncache entry and returns whether the entry is tracking a path
+matched by matcher or not.
+
+If matcher is None, returns True"""
+
+if matcher is None:
+return True
+path = decodefilename(path)
+if path.startswith('data/'):
+return matcher(path[len('data/'):-len('.i')])
+elif path.startswith('meta/'):
+return matcher.visitdir(path[len('meta/'):-len('/00manifest.i')] or 
'.')
+
 # This avoids a collision between a file named foo and a dir named
 # foo.i or foo.d
 def _encodedir(path):
@@ -413,6 +427,8 @@
 
 def datafiles(self, matcher=None):
 for a, b, size in super(encodedstore, self).datafiles():
+if not _matchtrackedpath(a, matcher):
+continue
 try:
 a = decodefilename(a)
 except KeyError:
@@ -542,6 +558,8 @@
 
 def datafiles(self, matcher=None):
 for f in sorted(self.fncache):
+if not _matchtrackedpath(f, matcher):
+continue
 ef = self.encode(f)
 try:
 yield f, ef, self.getsize(ef)



To: pulkit, durin42, #hg-reviewers
Cc: mjpieters, mercurial-devel

D5138: narrow: only send includepats and excludepats if they are not empty

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a reviewer: martinvonz.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  If we send an empty includepats or excludepats argument to getbundle, it's
  translated to `['']` on the server which causes problems because even though
  it's empty, bool of that value if True and we end up creating 
differencematcher
  with narrowspec.match() which results in unexpected behavior.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5138

AFFECTED FILES
  hgext/narrow/narrowcommands.py
  mercurial/bundle2.py

CHANGE DETAILS

diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -1688,8 +1688,8 @@
 bundler.prefercompressed = False
 
 # get the inlcudes and excludes
-includepats = kwargs[r'includepats']
-excludepats = kwargs[r'excludepats']
+includepats = kwargs.get(r'includepats')
+excludepats = kwargs.get(r'excludepats')
 
 narrowstream = repo.ui.configbool('server', 'stream-narrow-clones')
 
diff --git a/hgext/narrow/narrowcommands.py b/hgext/narrow/narrowcommands.py
--- a/hgext/narrow/narrowcommands.py
+++ b/hgext/narrow/narrowcommands.py
@@ -141,8 +141,10 @@
 include, exclude = repo.narrowpats
 kwargs['oldincludepats'] = include
 kwargs['oldexcludepats'] = exclude
-kwargs['includepats'] = include
-kwargs['excludepats'] = exclude
+if include:
+kwargs['includepats'] = include
+if exclude:
+kwargs['excludepats'] = exclude
 # calculate known nodes only in ellipses cases because in non-ellipses 
cases
 # we have all the nodes
 if wireprototypes.ELLIPSESCAP in pullop.remote.capabilities():



To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5137: streamclone: new server config and some API changes for narrow stream clones

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This patch introduces a new server config `server.stream-narrow-clones` which 
if
  set to True will advertise that the server supports narrow stream clones.
  
  This patch also pass on the includes and excludes from getbundle command to
  streamclone generation code.
  
  There is a test added to show that the includepats and excludepats are 
correctly
  passed.
  
  Upcoming patches will implement storage layer filtering for streamclones and
  then we can remove the temporary error and plug in the whole logic together to
  make narrow stream clones working.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

AFFECTED FILES
  mercurial/bundle2.py
  mercurial/configitems.py
  mercurial/help/config.txt
  mercurial/streamclone.py
  tests/test-narrow-clone-stream.t

CHANGE DETAILS

diff --git a/tests/test-narrow-clone-stream.t b/tests/test-narrow-clone-stream.t
new file mode 100644
--- /dev/null
+++ b/tests/test-narrow-clone-stream.t
@@ -0,0 +1,39 @@
+Tests narrow stream clones
+
+  $ . "$TESTDIR/narrow-library.sh"
+
+Server setup
+
+  $ hg init master
+  $ cd master
+  $ mkdir dir
+  $ mkdir dir/src
+  $ cd dir/src
+  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit 
-m "Commit src $x"; done
+
+  $ cd ..
+  $ mkdir tests
+  $ cd tests
+  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit 
-m "Commit src $x"; done
+  $ cd ../../..
+
+Trying to stream clone when the server does not support it
+
+  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include 
"dir/src/f10" --stream
+  streaming all changes
+  remote: abort: server does not support narrow stream clones
+  abort: pull failed on remote
+  [255]
+
+Enable stream clone on the server
+
+  $ echo "[server]" >> master/.hg/hgrc
+  $ echo "stream-narrow-clones=True" >> master/.hg/hgrc
+
+Cloning a specific file when stream clone is supported
+
+  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include 
"dir/src/f10" --stream
+  streaming all changes
+  remote: abort: support for narrow stream clones is missing
+  abort: pull failed on remote
+  [255]
diff --git a/mercurial/streamclone.py b/mercurial/streamclone.py
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -531,7 +531,7 @@
 finally:
 fp.close()
 
-def generatev2(repo):
+def generatev2(repo, includes, excludes):
 """Emit content for version 2 of a streaming clone.
 
 the data stream consists the following entries:
@@ -544,6 +544,10 @@
 Returns a 3-tuple of (file count, file size, data iterator).
 """
 
+# temporarily raise error until we add storage level logic
+if includes or excludes:
+raise error.Abort(_("support for narrow stream clones is missing"))
+
 with repo.lock():
 
 entries = []
diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt
--- a/mercurial/help/config.txt
+++ b/mercurial/help/config.txt
@@ -1848,6 +1848,10 @@
 Older Mercurial clients only support zlib compression and this setting
 has no effect for legacy clients.
 
+``stream-narrow-clones``
+Whether the server supports streaming narrow clones or not.
+(default: False)
+
 ``uncompressed``
 Whether to allow clients to clone a repository using the
 uncompressed streaming protocol. This transfers about 40% more
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -1011,6 +1011,9 @@
 coreconfigitem('server', 'streamunbundle',
 default=False,
 )
+coreconfigitem('server', 'stream-narrow-clones',
+default=False,
+)
 coreconfigitem('server', 'uncompressed',
 default=True,
 )
diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -1687,7 +1687,17 @@
 # to avoid compression to consumers of the bundle.
 bundler.prefercompressed = False
 
-filecount, bytecount, it = streamclone.generatev2(repo)
+# get the inlcudes and excludes
+includepats = kwargs[r'includepats']
+excludepats = kwargs[r'excludepats']
+
+narrowstream = repo.ui.configbool('server', 'stream-narrow-clones')
+
+if (includepats or excludepats) and not narrowstream:
+raise error.Abort(_('server does not support narrow stream clones'))
+
+filecount, bytecount, it = streamclone.generatev2(repo, includepats,
+  excludepats)
 requirements = _formatrequirementsspec(repo.requirements)
 part = bundler.newpart('stream2', data=it)
 part.addparam('bytecount', '%d' % bytecount, mandatory=True)



To: pulkit, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list

D4850: store: pass matcher to store.datafiles()

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit updated this revision to Diff 12210.
pulkit retitled this revision from "store: pass matcher to store.datafiles() 
and filter files according to it" to "store: pass matcher to store.datafiles()".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4850?vs=11602=12210

REVISION DETAIL
  https://phab.mercurial-scm.org/D4850

AFFECTED FILES
  mercurial/store.py

CHANGE DETAILS

diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -359,17 +359,21 @@
 l.sort()
 return l
 
-def datafiles(self):
+def datafiles(self, matcher=None):
 return self._walk('data', True) + self._walk('meta', True)
 
 def topfiles(self):
 # yield manifest before changelog
 return reversed(self._walk('', False))
 
-def walk(self):
-'''yields (unencoded, encoded, size)'''
+def walk(self, matcher=None):
+'''yields (unencoded, encoded, size)
+
+if a matcher is passed, storage files of only those tracked paths
+are passed with matches the matcher
+'''
 # yield data files first
-for x in self.datafiles():
+for x in self.datafiles(matcher):
 yield x
 for x in self.topfiles():
 yield x
@@ -407,7 +411,7 @@
 self.vfs = vfsmod.filtervfs(vfs, encodefilename)
 self.opener = self.vfs
 
-def datafiles(self):
+def datafiles(self, matcher=None):
 for a, b, size in super(encodedstore, self).datafiles():
 try:
 a = decodefilename(a)
@@ -536,7 +540,7 @@
 def getsize(self, path):
 return self.rawvfs.stat(path).st_size
 
-def datafiles(self):
+def datafiles(self, matcher=None):
 for f in sorted(self.fncache):
 ef = self.encode(f)
 try:



To: pulkit, #hg-reviewers, martinvonz
Cc: indygreg, martinvonz, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5119: exchange: pass includepats and excludepats as arguments to getbundle()

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit updated this revision to Diff 12207.
pulkit edited the summary of this revision.
Herald added a reviewer: durin42.
Herald added a reviewer: martinvonz.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5119?vs=12170=12207

REVISION DETAIL
  https://phab.mercurial-scm.org/D5119

AFFECTED FILES
  hgext/narrow/narrowbundle2.py
  mercurial/exchange.py
  mercurial/wireprototypes.py

CHANGE DETAILS

diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
--- a/mercurial/wireprototypes.py
+++ b/mercurial/wireprototypes.py
@@ -162,6 +162,8 @@
 'cg': 'boolean',
 'cbattempted': 'boolean',
 'stream': 'boolean',
+'includepats': 'csv',
+'excludepats': 'csv',
 }
 
 class baseprotocolhandler(interfaceutil.Interface):
diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -40,6 +40,7 @@
 streamclone,
 url as urlmod,
 util,
+wireprototypes,
 )
 from .utils import (
 stringutil,
@@ -1633,6 +1634,13 @@
 kwargs['common'] = pullop.common
 kwargs['heads'] = pullop.heads or pullop.rheads
 
+# check server supports narrow and then adding includepats and excludepats
+servernarrow = pullop.remote.capable(wireprototypes.NARROWCAP)
+if servernarrow and pullop.includepats:
+kwargs['includepats'] = pullop.includepats
+if servernarrow and pullop.excludepats:
+kwargs['excludepats'] = pullop.excludepats
+
 if streaming:
 kwargs['cg'] = False
 kwargs['stream'] = True
diff --git a/hgext/narrow/narrowbundle2.py b/hgext/narrow/narrowbundle2.py
--- a/hgext/narrow/narrowbundle2.py
+++ b/hgext/narrow/narrowbundle2.py
@@ -260,8 +260,6 @@
 getbundleargs['depth'] = 'plain'
 getbundleargs['oldincludepats'] = 'csv'
 getbundleargs['oldexcludepats'] = 'csv'
-getbundleargs['includepats'] = 'csv'
-getbundleargs['excludepats'] = 'csv'
 getbundleargs['known'] = 'csv'
 
 # Extend changegroup serving to handle requests from narrow clients.



To: pulkit, #hg-reviewers, indygreg, durin42, martinvonz
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4851: streamclone: pass narrowing related info in _walkstreamfiles()

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit updated this revision to Diff 12209.
pulkit edited the summary of this revision.
pulkit retitled this revision from "streamclone: pass narrowing related info in 
generatev2() and _walkstreamfiles()" to "streamclone: pass narrowing related 
info in _walkstreamfiles()".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4851?vs=11603=12209

REVISION DETAIL
  https://phab.mercurial-scm.org/D4851

AFFECTED FILES
  mercurial/streamclone.py

CHANGE DETAILS

diff --git a/mercurial/streamclone.py b/mercurial/streamclone.py
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -16,6 +16,7 @@
 branchmap,
 cacheutil,
 error,
+narrowspec,
 phases,
 pycompat,
 repository,
@@ -194,8 +195,8 @@
 return True
 
 # This is it's own function so extensions can override it.
-def _walkstreamfiles(repo):
-return repo.store.walk()
+def _walkstreamfiles(repo, matcher=None):
+return repo.store.walk(matcher)
 
 def generatev1(repo):
 """Emit content for version 1 of a streaming clone.
@@ -553,8 +554,12 @@
 entries = []
 totalfilesize = 0
 
+matcher = None
+if includes or excludes:
+matcher = narrowspec.match(repo.root, includes, excludes)
+
 repo.ui.debug('scanning\n')
-for name, ename, size in _walkstreamfiles(repo):
+for name, ename, size in _walkstreamfiles(repo, matcher):
 if size:
 entries.append((_srcstore, name, _fileappend, size))
 totalfilesize += size



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[PATCH] commands: adjust metavariables as appropriate

2018-10-17 Thread Anton Shestakov
# HG changeset patch
# User Anton Shestakov 
# Date 1539781236 -28800
#  Wed Oct 17 21:00:36 2018 +0800
# Node ID 02ad74c4abc1b66e63f118f1247ccc7c97acc786
# Parent  ef0baff11aeaaa8424294982235a18c476728264
commands: adjust metavariables as appropriate

Apart from looking better in hg help command, these strings are also helpful
when generating shell completions programmatically.

diff --git a/hgext/patchbomb.py b/hgext/patchbomb.py
--- a/hgext/patchbomb.py
+++ b/hgext/patchbomb.py
@@ -478,20 +478,25 @@ emailopts = [
 ('', 'body', None, _('send patches as inline message text (default)')),
 ('a', 'attach', None, _('send patches as attachments')),
 ('i', 'inline', None, _('send patches as inline attachments')),
-('', 'bcc', [], _('email addresses of blind carbon copy recipients')),
-('c', 'cc', [], _('email addresses of copy recipients')),
+('', 'bcc', [],
+ _('email addresses of blind carbon copy recipients'), _('EMAIL')),
+('c', 'cc', [], _('email addresses of copy recipients'), _('EMAIL')),
 ('', 'confirm', None, _('ask for confirmation before sending')),
 ('d', 'diffstat', None, _('add diffstat output to messages')),
-('', 'date', '', _('use the given date as the sending date')),
-('', 'desc', '', _('use the given file as the series description')),
-('f', 'from', '', _('email address of sender')),
+('', 'date', '', _('use the given date as the sending date'), _('DATE')),
+('', 'desc', '',
+ _('use the given file as the series description'), _('FILE')),
+('f', 'from', '', _('email address of sender'), _('EMAIL')),
 ('n', 'test', None, _('print messages that would be sent')),
-('m', 'mbox', '', _('write messages to mbox file instead of sending 
them')),
-('', 'reply-to', [], _('email addresses replies should be sent to')),
-('s', 'subject', '', _('subject of first message (intro or single 
patch)')),
-('', 'in-reply-to', '', _('message identifier to reply to')),
-('', 'flag', [], _('flags to add in subject prefixes')),
-('t', 'to', [], _('email addresses of recipients'))]
+('m', 'mbox', '',
+ _('write messages to mbox file instead of sending them'), _('FILE')),
+('', 'reply-to', [],
+ _('email addresses replies should be sent to'), _('EMAIL')),
+('s', 'subject', '',
+ _('subject of first message (intro or single patch)'), _('TEXT')),
+('', 'in-reply-to', '', _('message identifier to reply to'), _('MSGID')),
+('', 'flag', [], _('flags to add in subject prefixes'), _('FLAG')),
+('t', 'to', [], _('email addresses of recipients'), _('EMAIL'))]
 
 @command('email',
 [('g', 'git', None, _('use git extended diff format')),
@@ -499,7 +504,8 @@ emailopts = [
 ('o', 'outgoing', None,
  _('send changes not found in the target repository')),
 ('b', 'bundle', None, _('send changes not in target as a binary bundle')),
-('B', 'bookmark', '', _('send changes only reachable by given bookmark')),
+('B', 'bookmark', '',
+ _('send changes only reachable by given bookmark'), _('BOOKMARK')),
 ('', 'bundlename', 'bundle',
  _('name of the bundle attachment file'), _('NAME')),
 ('r', 'rev', [], _('a revision to send'), _('REV')),
diff --git a/hgext/strip.py b/hgext/strip.py
--- a/hgext/strip.py
+++ b/hgext/strip.py
@@ -110,7 +110,7 @@ def strip(ui, repo, revs, update=True, b
   ('k', 'keep', None, _("do not modify working directory during "
 "strip")),
   ('B', 'bookmark', [], _("remove revs only reachable from given"
-  " bookmark"))],
+  " bookmark"), _('BOOKMARK'))],
   _('hg strip [-k] [-f] [-B bookmark] [-r] REV...'),
   helpcategory=command.CATEGORY_MAINTENANCE)
 def stripcmd(ui, repo, *revs, **opts):
diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
--- a/mercurial/cmdutil.py
+++ b/mercurial/cmdutil.py
@@ -149,7 +149,7 @@ diffopts2 = [
 ]
 
 mergetoolopts = [
-('t', 'tool', '', _('specify merge tool')),
+('t', 'tool', '', _('specify merge tool'), _('TOOL')),
 ]
 
 similarityopts = [
diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -1996,7 +1996,7 @@ def diff(ui, repo, *pats, **opts):
 
 @command('export',
 [('B', 'bookmark', '',
- _('export changes only reachable by given bookmark')),
+ _('export changes only reachable by given bookmark'), _('BOOKMARK')),
 ('o', 'output', '',
  _('print output to file with formatted name'), _('FORMAT')),
 ('', 'switch-parent', None, _('diff against the second parent')),
@@ -2257,7 +2257,7 @@ def forget(ui, repo, *pats, **opts):
  ('D', 'currentdate', False,
   _('record the current date as commit date')),
  ('U', 'currentuser', False,
-  _('record the current user as committer'), _('DATE'))]
+  _('record the current user as committer'))]
 + commitopts2 + 

D5134: wireprotov2: implement command for retrieving raw store files

2018-10-17 Thread indygreg (Gregory Szorc)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGed55a0077490: wireprotov2: implement command for retrieving 
raw store files (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5134?vs=12196=12204

REVISION DETAIL
  https://phab.mercurial-scm.org/D5134

AFFECTED FILES
  mercurial/help/internals/wireprotocolv2.txt
  mercurial/wireprotov2server.py
  tests/test-http-protocol.t
  tests/test-wireproto-caching.t
  tests/test-wireproto-command-capabilities.t
  tests/test-wireproto-command-rawstorefiledata.t
  tests/test-wireproto-content-redirects.t

CHANGE DETAILS

diff --git a/tests/test-wireproto-content-redirects.t 
b/tests/test-wireproto-content-redirects.t
--- a/tests/test-wireproto-content-redirects.t
+++ b/tests/test-wireproto-content-redirects.t
@@ -65,9 +65,9 @@
   s> Server: testing stub value\r\n
   s> Date: $HTTP_DATE$\r\n
   s> Content-Type: application/mercurial-cbor\r\n
-  s> Content-Length: 2131\r\n
+  s> Content-Length: 2241\r\n
   s> \r\n
-  s> 
\xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0003\xa5Hcommands\xabIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa2Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x84IbookmarksGparentsEphaseHrevisionIrevisions\xa2Hrequired\xf5DtypeDlistKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullIfilesdata\xa3Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x83NfirstchangesetGparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolJpathfilter\xa3Gdefault\xf6Hrequired\xf4DtypeDdictIrevisions\xa2Hrequired\xf5DtypeDlistKpermissions\x81DpullTrecommendedbatchsize\x19\xc3PEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa3Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullTrecommendedbatchsize\x1a\x00\x01\x86\xa0Gpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushQframingmediatypes\x81X/mercurial-exp-framing-0006Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Hredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x81\xa5DnameHtarget-aHprotocolDhttpKsnirequired\xf4Ktlsversions\x82C1.2C1.3Duris\x81Shttp://example.com/Nv1capabilitiesY\x01\xd3batch
 branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset 
compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey 
streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
+  s> 

D5136: exchange: support declaring pull depth

2018-10-17 Thread indygreg (Gregory Szorc)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGac59de55c8b4: exchange: support declaring pull depth 
(authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5136?vs=12198=12205

REVISION DETAIL
  https://phab.mercurial-scm.org/D5136

AFFECTED FILES
  mercurial/exchange.py
  mercurial/hg.py

CHANGE DETAILS

diff --git a/mercurial/hg.py b/mercurial/hg.py
--- a/mercurial/hg.py
+++ b/mercurial/hg.py
@@ -479,7 +479,7 @@
 
 def clone(ui, peeropts, source, dest=None, pull=False, revs=None,
   update=True, stream=False, branch=None, shareopts=None,
-  storeincludepats=None, storeexcludepats=None):
+  storeincludepats=None, storeexcludepats=None, depth=None):
 """Make a copy of an existing repository.
 
 Create a copy of an existing repository in a new directory.  The
@@ -749,7 +749,8 @@
 exchange.pull(local, srcpeer, revs,
   streamclonerequested=stream,
   includepats=storeincludepats,
-  excludepats=storeexcludepats)
+  excludepats=storeexcludepats,
+  depth=depth)
 elif srcrepo:
 # TODO lift restriction once exchange.push() accepts narrow
 # push.
diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1316,7 +1316,7 @@
 
 def __init__(self, repo, remote, heads=None, force=False, bookmarks=(),
  remotebookmarks=None, streamclonerequested=None,
- includepats=None, excludepats=None):
+ includepats=None, excludepats=None, depth=None):
 # repo we pull into
 self.repo = repo
 # repo we pull from
@@ -1350,6 +1350,8 @@
 self.includepats = includepats
 # Set of file patterns to exclude.
 self.excludepats = excludepats
+# Number of ancestor changesets to pull from each pulled head.
+self.depth = depth
 
 @util.propertycache
 def pulledsubset(self):
@@ -1454,7 +1456,8 @@
 pullop.rheads = set(pullop.rheads) - pullop.common
 
 def pull(repo, remote, heads=None, force=False, bookmarks=(), opargs=None,
- streamclonerequested=None, includepats=None, excludepats=None):
+ streamclonerequested=None, includepats=None, excludepats=None,
+ depth=None):
 """Fetch repository data from a remote.
 
 This is the main function used to retrieve data from a remote repository.
@@ -1475,6 +1478,9 @@
 ``includepats`` and ``excludepats`` define explicit file patterns to
 include and exclude in storage, respectively. If not defined, narrow
 patterns from the repo instance are used, if available.
+``depth`` is an integer indicating the DAG depth of history we're
+interested in. If defined, for each revision specified in ``heads``, we
+will fetch up to this many of its ancestors and data associated with them.
 
 Returns the ``pulloperation`` created for this pull.
 """
@@ -1495,6 +1501,7 @@
 pullop = pulloperation(repo, remote, heads, force, bookmarks=bookmarks,
streamclonerequested=streamclonerequested,
includepats=includepats, excludepats=excludepats,
+   depth=depth,
**pycompat.strkwargs(opargs))
 
 peerlocal = pullop.remote.local()



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5096: py3: fix test-hardlinks.t

2018-10-17 Thread mbthomas (Mark Thomas)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGe0dea186ab6e: py3: fix test-hardlinks.t (authored by 
mbthomas, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D5096?vs=12111=12206#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5096?vs=12111=12206

REVISION DETAIL
  https://phab.mercurial-scm.org/D5096

AFFECTED FILES
  contrib/python3-whitelist
  tests/test-hardlinks.t

CHANGE DETAILS

diff --git a/tests/test-hardlinks.t b/tests/test-hardlinks.t
--- a/tests/test-hardlinks.t
+++ b/tests/test-hardlinks.t
@@ -19,8 +19,9 @@
   $ cat > linkcp.py < from __future__ import absolute_import
   > import sys
-  > from mercurial import util
-  > util.copyfiles(sys.argv[1], sys.argv[2], hardlink=True)
+  > from mercurial import pycompat, util
+  > util.copyfiles(pycompat.fsencode(sys.argv[1]),
+  >pycompat.fsencode(sys.argv[2]), hardlink=True)
   > EOF
 
   $ linkcp()
diff --git a/contrib/python3-whitelist b/contrib/python3-whitelist
--- a/contrib/python3-whitelist
+++ b/contrib/python3-whitelist
@@ -214,6 +214,7 @@
 test-gpg.t
 test-graft.t
 test-grep.t
+test-hardlinks.t
 test-hg-parseurl.py
 test-hghave.t
 test-hgignore.t



To: mbthomas, pulkit, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5135: exchangev2: support for calling rawstorefiledata to retrieve raw files

2018-10-17 Thread indygreg (Gregory Szorc)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG00a4cd368e3f: exchangev2: support for calling 
rawstorefiledata to retrieve raw files (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5135?vs=12197=12203

REVISION DETAIL
  https://phab.mercurial-scm.org/D5135

AFFECTED FILES
  mercurial/exchangev2.py
  tests/test-wireproto-exchangev2.t

CHANGE DETAILS

diff --git a/tests/test-wireproto-exchangev2.t 
b/tests/test-wireproto-exchangev2.t
--- a/tests/test-wireproto-exchangev2.t
+++ b/tests/test-wireproto-exchangev2.t
@@ -963,3 +963,185 @@
   client-narrow-2/.hg/store/00manifest.i
   client-narrow-2/.hg/store/data/dir0/d.i
 #endif
+
+--stream will use rawfiledata to transfer changelog and manifestlog, then
+fall through to get files data
+
+  $ hg --debug clone --stream -U http://localhost:$HGPORT client-stream-0
+  using http://localhost:$HGPORT/
+  sending capabilities command
+  sending 1 commands
+  sending command rawstorefiledata: {
+'files': [
+  'changelog',
+  'manifestlog'
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=1275; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  updating the branch cache
+  query 1; heads
+  sending 2 commands
+  sending command heads: {}
+  sending command known: {
+'nodes': [
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=22; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=11; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=2; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=3; stream=2; streamflags=; 
type=command-response; flags=eos)
+  searching for changes
+  all remote heads known locally
+  sending 1 commands
+  sending command changesetdata: {
+'fields': set([
+  'bookmarks',
+  'parents',
+  'phase',
+  'revision'
+]),
+'revisions': [
+  {
+'heads': [
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'roots': [
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'type': 'changesetdagrange'
+  }
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=13; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  checking for updated bookmarks
+  sending 1 commands
+  sending command filesdata: {
+'fields': set([
+  'parents',
+  'revision'
+]),
+'haveparents': True,
+'revisions': [
+  {
+'nodes': [
+  '3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:',
+  '\xb7\t8\x08\x92\xb1\x93\xc1\t\x1d:\x81\x7fp`R\xe3F\x82\x1b',
+  'G\xfe\x01*\xb27\xa8\xc7\xfc\x0cx\xf9\xf2mXf\xee\xf3\xf8%',
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'type': 'changesetexplicit'
+  }
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=1133; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  (sent 5 HTTP requests and * bytes; received * bytes in responses) (glob)
+
+--stream + --include/--exclude will only obtain some files
+
+  $ hg --debug --config extensions.pullext=$TESTDIR/pullext.py clone --stream 
--include dir0/ -U http://localhost:$HGPORT client-stream-2
+  using http://localhost:$HGPORT/
+  sending capabilities command
+  sending 1 commands
+  sending command rawstorefiledata: {
+'files': [
+  'changelog',
+  'manifestlog'
+]
+  }
+  received frame(size=9; request=1; stream=2; 

D5133: wireprotov2: add response type that serializes to indefinite length bytestring

2018-10-17 Thread indygreg (Gregory Szorc)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2c55716f8a1c: wireprotov2: add response type that 
serializes to indefinite length bytestring (authored by indygreg, committed by 
).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5133?vs=12195=12202

REVISION DETAIL
  https://phab.mercurial-scm.org/D5133

AFFECTED FILES
  mercurial/wireprotoframing.py
  mercurial/wireprototypes.py

CHANGE DETAILS

diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
--- a/mercurial/wireprototypes.py
+++ b/mercurial/wireprototypes.py
@@ -389,3 +389,12 @@
 fullhashseed = attr.ib(default=None)
 serverdercerts = attr.ib(default=None)
 servercadercerts = attr.ib(default=None)
+
+@attr.s
+class indefinitebytestringresponse(object):
+"""Represents an object to be encoded to an indefinite length bytestring.
+
+Instances are initialized from an iterable of chunks, with each chunk being
+a bytes instance.
+"""
+chunks = attr.ib()
diff --git a/mercurial/wireprotoframing.py b/mercurial/wireprotoframing.py
--- a/mercurial/wireprotoframing.py
+++ b/mercurial/wireprotoframing.py
@@ -1146,6 +1146,14 @@
 for frame in emitter.send(o.data):
 yield frame
 
+elif isinstance(
+o, wireprototypes.indefinitebytestringresponse):
+for chunk in cborutil.streamencodebytestringfromiter(
+o.chunks):
+
+for frame in emitter.send(chunk):
+yield frame
+
 # A regular object is CBOR encoded.
 else:
 for chunk in cborutil.streamencode(o):



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5132: exchangev2: recognize narrow patterns when pulling

2018-10-17 Thread indygreg (Gregory Szorc)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG55836a34f41b: exchangev2: recognize narrow patterns when 
pulling (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5132?vs=12194=12201

REVISION DETAIL
  https://phab.mercurial-scm.org/D5132

AFFECTED FILES
  mercurial/exchangev2.py
  tests/pullext.py
  tests/test-wireproto-exchangev2.t

CHANGE DETAILS

diff --git a/tests/test-wireproto-exchangev2.t 
b/tests/test-wireproto-exchangev2.t
--- a/tests/test-wireproto-exchangev2.t
+++ b/tests/test-wireproto-exchangev2.t
@@ -619,3 +619,347 @@
   $ hg -R client-bookmarks bookmarks
  book-12:cd2534766bec
  book-22:cd2534766bec
+
+  $ killdaemons.py
+
+Let's set up a slightly more complicated server
+
+  $ hg init server-2
+  $ enablehttpv2 server-2
+  $ cd server-2
+  $ mkdir dir0 dir1
+  $ echo a0 > a
+  $ echo b0 > b
+  $ hg -q commit -A -m 'commit 0'
+  $ echo c0 > dir0/c
+  $ echo d0 > dir0/d
+  $ hg -q commit -A -m 'commit 1'
+  $ echo e0 > dir1/e
+  $ echo f0 > dir1/f
+  $ hg -q commit -A -m 'commit 2'
+  $ echo c1 > dir0/c
+  $ echo e1 > dir1/e
+  $ hg commit -m 'commit 3'
+  $ hg serve -p $HGPORT -d --pid-file hg.pid -E error.log
+  $ cat hg.pid > $DAEMON_PIDS
+
+  $ cd ..
+
+Narrow clone only fetches some files
+
+  $ hg --config extensions.pullext=$TESTDIR/pullext.py --debug clone -U 
--include dir0/ http://localhost:$HGPORT/ client-narrow-0
+  using http://localhost:$HGPORT/
+  sending capabilities command
+  query 1; heads
+  sending 2 commands
+  sending command heads: {}
+  sending command known: {
+'nodes': []
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=22; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=11; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=1; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=3; stream=2; streamflags=; 
type=command-response; flags=eos)
+  sending 1 commands
+  sending command changesetdata: {
+'fields': set([
+  'bookmarks',
+  'parents',
+  'phase',
+  'revision'
+]),
+'revisions': [
+  {
+'heads': [
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'roots': [],
+'type': 'changesetdagrange'
+  }
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=783; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  add changeset 3390ef850073
+  add changeset b709380892b1
+  add changeset 47fe012ab237
+  add changeset 97765fc3cd62
+  checking for updated bookmarks
+  sending 1 commands
+  sending command manifestdata: {
+'fields': set([
+  'parents',
+  'revision'
+]),
+'haveparents': True,
+'nodes': [
+  '\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN',
+  '|2 \x1a\xa3\xa1R\xa9\xe6\xa9"+?\xa8\xd0\xe3\x0f\xc2V\xe8',
+  '\x8d\xd0W<\x7f\xaf\xe2\x04F\xcc\xea\xac\x05N\xea\xa4x\x91M\xdb',
+  '113\x85\xf2!\x8b\x08^\xb2Z\x821\x1e*\xdd\x0e\xeb\x8c3'
+],
+'tree': ''
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=967; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  sending 1 commands
+  sending command filesdata: {
+'fields': set([
+  'parents',
+  'revision'
+]),
+'haveparents': True,
+'pathfilter': {
+  'include': [
+'path:dir0'
+  ]
+},
+'revisions': [
+  {
+'nodes': [
+  '3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:',
+  '\xb7\t8\x08\x92\xb1\x93\xc1\t\x1d:\x81\x7fp`R\xe3F\x82\x1b',
+  'G\xfe\x01*\xb27\xa8\xc7\xfc\x0cx\xf9\xf2mXf\xee\xf3\xf8%',
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'type': 'changesetexplicit'
+  }
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 

[PATCH] profiling: move default mode to "real" time

2018-10-17 Thread Boris Feld
# HG changeset patch
# User Boris Feld 
# Date 1534549478 -7200
#  Sat Aug 18 01:44:38 2018 +0200
# Node ID 63cac0b435c518d8007df572c94341328a2d1dde
# Parent  b95b48a55c36050852b46868f4e4f750a29d8754
# EXP-Topic profiling-mode
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
63cac0b435c5
profiling: move default mode to "real" time

Mercurial operations involve a lot of disks or network access. These impact
command runtime significantly and it seems important to report them in our
default profiling output.

Having the right default means that we don't forget them when asking people to
produces profiling traces or when doing profiling ourselves.

Moving to "real time" by default will remove the need to think about
activating it on most occasions. The "CPU" time-based profiling is still
accessible when necessary.

diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -904,7 +904,7 @@ coreconfigitem('profiling', 'statformat'
 default='hotpath',
 )
 coreconfigitem('profiling', 'time-track',
-default='cpu',
+default='real',
 )
 coreconfigitem('profiling', 'type',
 default='stat',
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5135: exchangev2: support for calling rawstorefiledata to retrieve raw files

2018-10-17 Thread durin42 (Augie Fackler)
durin42 added a comment.


  I have significant misgivings about exchanging raw disk contents. We should 
revisit this and see if we can do better in the future.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5135

To: indygreg, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


Re: [PATCH 1 of 8] formatter: more details on assertion failure

2018-10-17 Thread Boris FELD
On 11/10/2018 10:23, Gregory Szorc wrote:
> On Thu, Oct 11, 2018 at 10:05 AM Boris Feld  > wrote:
>
> # HG changeset patch
> # User Boris Feld  >
> # Date 1538408273 -7200
> #      Mon Oct 01 17:37:53 2018 +0200
> # Node ID 8eaff9311fa407d89854d6e60938200939a0a506
> # Parent  a66594c5fad422499bfaf669d182e02fe3c0be32
> # EXP-Topic revlog-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull
> https://bitbucket.org/octobus/mercurial-devel/ -r 8eaff9311fa4
> formatter: more details on assertion failure
>
>
> Queued parts 1-6. Still digesting part 7.
Is there anything we could do to helps the review process of part 7 and 8?
>  
>
>
> This is useful when the assertion fails.
>
> diff --git a/mercurial/formatter.py b/mercurial/formatter.py
> --- a/mercurial/formatter.py
> +++ b/mercurial/formatter.py
> @@ -213,7 +213,7 @@ class baseformatter(object):
>      def write(self, fields, deftext, *fielddata, **opts):
>          '''do default text output while assigning data to item'''
>          fieldkeys = fields.split()
> -        assert len(fieldkeys) == len(fielddata)
> +        assert len(fieldkeys) == len(fielddata), (fieldkeys,
> fielddata)
>          self._item.update(zip(fieldkeys, fielddata))
>      def condwrite(self, cond, fields, deftext, *fielddata, **opts):
>          '''do conditional write (primarily for plain formatter)'''
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> 
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
>
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


Re: [PATCH 1 of 8] formatter: more details on assertion failure

2018-10-17 Thread Gregory Szorc
On Wed, Oct 17, 2018 at 3:22 PM Boris FELD  wrote:

> On 11/10/2018 10:23, Gregory Szorc wrote:
>
> On Thu, Oct 11, 2018 at 10:05 AM Boris Feld 
> wrote:
>
>> # HG changeset patch
>> # User Boris Feld 
>> # Date 1538408273 -7200
>> #  Mon Oct 01 17:37:53 2018 +0200
>> # Node ID 8eaff9311fa407d89854d6e60938200939a0a506
>> # Parent  a66594c5fad422499bfaf669d182e02fe3c0be32
>> # EXP-Topic revlog-perf
>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
>> 8eaff9311fa4
>> formatter: more details on assertion failure
>>
>
> Queued parts 1-6. Still digesting part 7.
>
> Is there anything we could do to helps the review process of part 7 and 8?
>

Please resend the patches.


>
>
>>
>> This is useful when the assertion fails.
>>
>> diff --git a/mercurial/formatter.py b/mercurial/formatter.py
>> --- a/mercurial/formatter.py
>> +++ b/mercurial/formatter.py
>> @@ -213,7 +213,7 @@ class baseformatter(object):
>>  def write(self, fields, deftext, *fielddata, **opts):
>>  '''do default text output while assigning data to item'''
>>  fieldkeys = fields.split()
>> -assert len(fieldkeys) == len(fielddata)
>> +assert len(fieldkeys) == len(fielddata), (fieldkeys, fielddata)
>>  self._item.update(zip(fieldkeys, fielddata))
>>  def condwrite(self, cond, fields, deftext, *fielddata, **opts):
>>  '''do conditional write (primarily for plain formatter)'''
>> ___
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>>
>
> ___
> Mercurial-devel mailing 
> listMercurial-devel@mercurial-scm.orghttps://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5132: exchangev2: recognize narrow patterns when pulling

2018-10-17 Thread durin42 (Augie Fackler)
durin42 accepted this revision.
durin42 added inline comments.
This revision is now accepted and ready to land.

INLINE COMMENTS

> pullext.py:1
> +# pullext.py - Simple extension to test pulling
> +#

"test narrow pulling" maybe? I feel like that's mostly what it is...

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5132

To: indygreg, #hg-reviewers, durin42
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


Re: [PATCH] color: issue warning in yellow

2018-10-17 Thread Boris FELD
On 21/08/2018 15:15, Yuya Nishihara wrote:
> On Mon, 20 Aug 2018 11:14:23 -0700, Martin von Zweigbergk via Mercurial-devel 
> wrote:
>> On Mon, Aug 20, 2018 at 9:26 AM Kyle Lippincott  wrote:
>>
>>> Bright/bold yellow should be considered unavailable.  "Normal" yellow,
>>> which is closer to brown or maybe gold on many screens, is fine.
>>>
>> Ah, and we already use yellow for a few things (as you also pointed out).
>> Sorry I didn't even check that.
>>
>>
>>>   At Google, and I think other tools like clang, generally use magenta I
>>> believe, but I have no strong preference.  Keep in mind that ui.prompt is
>>> also yellow a couple lines below.
>>>
>> Good point about ui.prompt. Yellow seems like the obvious choice for
>> warnings, so do we want to change the color for the prompt?
> FWIW, I don't like the red-colored "error" since it's hard to spot in
> white-on-black screen. Well, it's readable, but not significant. A plain
> "yellow" (i.e. dark yellow) would have the same effect.

I'm not sure what is the status of this series.

Rereading the discussion, it seems like the yellow on white background
is readable. Is there anything else blocking this improvement

> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4928: sqlitestore: file storage backend using SQLite

2018-10-17 Thread indygreg (Gregory Szorc)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGfed697fa1734: sqlitestore: file storage backend using 
SQLite (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4928?vs=12191=12200

REVISION DETAIL
  https://phab.mercurial-scm.org/D4928

AFFECTED FILES
  hgext/sqlitestore.py
  tests/hghave.py
  tests/test-sqlitestore.t
  tests/test-storage.py

CHANGE DETAILS

diff --git a/tests/test-storage.py b/tests/test-storage.py
--- a/tests/test-storage.py
+++ b/tests/test-storage.py
@@ -17,6 +17,16 @@
 storage as storagetesting,
 )
 
+from hgext import (
+sqlitestore,
+)
+
+try:
+from mercurial import zstd
+zstd.__version__
+except ImportError:
+zstd = None
+
 STATE = {
 'lastindex': 0,
 'ui': uimod.ui(),
@@ -70,5 +80,42 @@
  maketransaction,
  addrawrevision)
 
+def makesqlitefile(self):
+path = STATE['vfs'].join(b'db-%d.db' % STATE['lastindex'])
+STATE['lastindex'] += 1
+
+db = sqlitestore.makedb(path)
+
+compression = b'zstd' if zstd else b'zlib'
+
+return sqlitestore.sqlitefilestore(db, b'dummy-path', compression)
+
+def addrawrevisionsqlite(self, fl, tr, node, p1, p2, linkrev, rawtext=None,
+ delta=None, censored=False, ellipsis=False,
+ extstored=False):
+flags = 0
+
+if censored:
+flags |= sqlitestore.FLAG_CENSORED
+
+if ellipsis | extstored:
+raise error.Abort(b'support for ellipsis and extstored flags not '
+  b'supported')
+
+if rawtext is not None:
+fl._addrawrevision(node, rawtext, tr, linkrev, p1, p2, flags=flags)
+elif delta is not None:
+fl._addrawrevision(node, rawtext, tr, linkrev, p1, p2,
+   storedelta=delta, flags=flags)
+else:
+raise error.Abort(b'must supply rawtext or delta arguments')
+
+sqlitefileindextests = storagetesting.makeifileindextests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+sqlitefiledatatests = storagetesting.makeifiledatatests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+sqlitefilemutationtests = storagetesting.makeifilemutationtests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+
 if __name__ == '__main__':
 silenttestrunner.main(__name__)
diff --git a/tests/test-sqlitestore.t b/tests/test-sqlitestore.t
new file mode 100644
--- /dev/null
+++ b/tests/test-sqlitestore.t
@@ -0,0 +1,110 @@
+#require sqlite
+
+  $ cat >> $HGRCPATH < [extensions]
+  > sqlitestore =
+  > EOF
+
+New repo should not use SQLite by default
+
+  $ hg init empty-no-sqlite
+  $ cat empty-no-sqlite/.hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+storage.new-repo-backend=sqlite is recognized
+
+  $ hg --config storage.new-repo-backend=sqlite init empty-sqlite
+  $ cat empty-sqlite/.hg/requires
+  dotencode
+  exp-sqlite-001
+  exp-sqlite-comp-001=zstd (zstd !)
+  exp-sqlite-comp-001-zlib (no-zstd !)
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+  $ cat >> $HGRCPATH << EOF
+  > [storage]
+  > new-repo-backend = sqlite
+  > EOF
+
+Can force compression to zlib
+
+  $ hg --config storage.sqlite.compression=zlib init empty-zlib
+  $ cat empty-zlib/.hg/requires
+  dotencode
+  exp-sqlite-001
+  exp-sqlite-comp-001=$BUNDLE2_COMPRESSIONS$
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+Can force compression to none
+
+  $ hg --config storage.sqlite.compression=none init empty-none
+  $ cat empty-none/.hg/requires
+  dotencode
+  exp-sqlite-001
+  exp-sqlite-comp-001=none
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+Can make a local commit
+
+  $ hg init local-commit
+  $ cd local-commit
+  $ echo 0 > foo
+  $ hg commit -A -m initial
+  adding foo
+
+That results in a row being inserted into various tables
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM filepath;
+  > EOF
+  1|foo
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM fileindex;
+  > EOF
+  
1|1|0|-1|-1|0|0|1||6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe
 (esc)
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM delta;
+  > EOF
+  1|1| \xd2\xaf\x8d\xd2"\x01\xdd\x8dH\xe5\xdc\xfc\xae\xd2\x81\xff\x94"\xc7|0 
(esc)
+  
+
+Tracking multiple files works
+
+  $ echo 1 > bar
+  $ hg commit -A -m 'add bar'
+  adding bar
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM filedata ORDER BY id ASC;
+  > EOF
+  
1|1|foo|0|6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe|-1|-1|0|0|1|
 (esc)
+  2|2|bar|0|\xb8\xe0/d3s\x80!\xa0e\xf9Au\xc7\xcd#\xdb_\x05\xbe|-1|-1|1|0|2| 
(esc)
+
+Multiple revisions of a file works
+
+  $ echo a >> foo
+  $ hg commit -m 'modify foo'
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM filedata ORDER BY id ASC;
+  > 

D5118: storageutil: extract most of peek_censored from revlog

2018-10-17 Thread indygreg (Gregory Szorc)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb0fbd1792e2d: storageutil: extract most of peek_censored 
from revlog (authored by indygreg, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5118?vs=12166=12199

REVISION DETAIL
  https://phab.mercurial-scm.org/D5118

AFFECTED FILES
  mercurial/revlog.py
  mercurial/utils/storageutil.py

CHANGE DETAILS

diff --git a/mercurial/utils/storageutil.py b/mercurial/utils/storageutil.py
--- a/mercurial/utils/storageutil.py
+++ b/mercurial/utils/storageutil.py
@@ -9,6 +9,7 @@
 
 import hashlib
 import re
+import struct
 
 from ..i18n import _
 from ..node import (
@@ -449,3 +450,31 @@
 delta=delta)
 
 prevrev = rev
+
+def deltaiscensored(delta, baserev, baselenfn):
+"""Determine if a delta represents censored revision data.
+
+``baserev`` is the base revision this delta is encoded against.
+``baselenfn`` is a callable receiving a revision number that resolves the
+length of the revision fulltext.
+
+Returns a bool indicating if the result of the delta represents a censored
+revision.
+"""
+# Fragile heuristic: unless new file meta keys are added alphabetically
+# preceding "censored", all censored revisions are prefixed by
+# "\1\ncensored:". A delta producing such a censored revision must be a
+# full-replacement delta, so we inspect the first and only patch in the
+# delta for this prefix.
+hlen = struct.calcsize(">lll")
+if len(delta) <= hlen:
+return False
+
+oldlen = baselenfn(baserev)
+newlen = len(delta) - hlen
+if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
+return False
+
+add = "\1\ncensored:"
+addlen = len(add)
+return newlen >= addlen and delta[hlen:hlen + addlen] == add
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -2109,23 +2109,7 @@
 if not self._censorable:
 return False
 
-# Fragile heuristic: unless new file meta keys are added alphabetically
-# preceding "censored", all censored revisions are prefixed by
-# "\1\ncensored:". A delta producing such a censored revision must be a
-# full-replacement delta, so we inspect the first and only patch in the
-# delta for this prefix.
-hlen = struct.calcsize(">lll")
-if len(delta) <= hlen:
-return False
-
-oldlen = self.rawsize(baserev)
-newlen = len(delta) - hlen
-if delta[:hlen] != mdiff.replacediffheader(oldlen, newlen):
-return False
-
-add = "\1\ncensored:"
-addlen = len(add)
-return newlen >= addlen and delta[hlen:hlen + addlen] == add
+return storageutil.deltaiscensored(delta, baserev, self.rawsize)
 
 def getstrippoint(self, minlink):
 """find the minimum rev that must be stripped to strip the linkrev



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5134: wireprotov2: implement command for retrieving raw store files

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Implementing shallow clone of the changelog is hard. We want the 4.8
  release to have a fast implementation of partial clone in wireprotov2. In
  order to achieve fast, we can't use deltas for transferring changelog and
  manifestlog data.
  
  Per discussions at the 4.8 sprint, this commit implements a somwwhat hacky
  and likely-to-be-changed-drastically-or-dropped command in wireprotov2 that
  facilitates access to raw store files, namely the changelog and manifestlog.
  Using this command, clients can perform a "stream clone" of sorts for just
  the changelog and manifestlog. This will allow clients to fetch the changelog
  and manifest revlogs, stream them to disk (which should be fast), then follow
  up filesdata requests for files revision data for a particular changeset.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5134

AFFECTED FILES
  mercurial/help/internals/wireprotocolv2.txt
  mercurial/wireprotov2server.py
  tests/test-http-protocol.t
  tests/test-wireproto-caching.t
  tests/test-wireproto-command-capabilities.t
  tests/test-wireproto-command-rawstorefiledata.t
  tests/test-wireproto-content-redirects.t

CHANGE DETAILS

diff --git a/tests/test-wireproto-content-redirects.t 
b/tests/test-wireproto-content-redirects.t
--- a/tests/test-wireproto-content-redirects.t
+++ b/tests/test-wireproto-content-redirects.t
@@ -65,9 +65,9 @@
   s> Server: testing stub value\r\n
   s> Date: $HTTP_DATE$\r\n
   s> Content-Type: application/mercurial-cbor\r\n
-  s> Content-Length: 2131\r\n
+  s> Content-Length: 2241\r\n
   s> \r\n
-  s> 
\xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0003\xa5Hcommands\xabIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa2Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x84IbookmarksGparentsEphaseHrevisionIrevisions\xa2Hrequired\xf5DtypeDlistKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullIfilesdata\xa3Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x83NfirstchangesetGparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolJpathfilter\xa3Gdefault\xf6Hrequired\xf4DtypeDdictIrevisions\xa2Hrequired\xf5DtypeDlistKpermissions\x81DpullTrecommendedbatchsize\x19\xc3PEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa3Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullTrecommendedbatchsize\x1a\x00\x01\x86\xa0Gpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushQframingmediatypes\x81X/mercurial-exp-framing-0006Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Hredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x81\xa5DnameHtarget-aHprotocolDhttpKsnirequired\xf4Ktlsversions\x82C1.2C1.3Duris\x81Shttp://example.com/Nv1capabilitiesY\x01\xd3batch
 branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset 
compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey 
streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
+  s> 

D5136: exchange: support declaring pull depth

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Upcoming commits will teach exchangev2 how to perform a shallow
  clone. This commit teaches hg.clone(), exchange.pull(), and
  exchange.pulloperation to recognize a request for a shallow clone
  by having the caller specify a numeric depth of the maximum number of
  ancestor changesets to fetch.
  
  There are certainly other ways we could control shallow-ness. But this
  one is simple to implement and is also how the narrow extension
  controls things. So it seems to make sense to start here.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5136

AFFECTED FILES
  mercurial/exchange.py
  mercurial/hg.py

CHANGE DETAILS

diff --git a/mercurial/hg.py b/mercurial/hg.py
--- a/mercurial/hg.py
+++ b/mercurial/hg.py
@@ -479,7 +479,7 @@
 
 def clone(ui, peeropts, source, dest=None, pull=False, revs=None,
   update=True, stream=False, branch=None, shareopts=None,
-  storeincludepats=None, storeexcludepats=None):
+  storeincludepats=None, storeexcludepats=None, depth=None):
 """Make a copy of an existing repository.
 
 Create a copy of an existing repository in a new directory.  The
@@ -732,7 +732,8 @@
 exchange.pull(local, srcpeer, revs,
   streamclonerequested=stream,
   includepats=storeincludepats,
-  excludepats=storeexcludepats)
+  excludepats=storeexcludepats,
+  depth=depth)
 elif srcrepo:
 # TODO lift restriction once exchange.push() accepts narrow
 # push.
diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1316,7 +1316,7 @@
 
 def __init__(self, repo, remote, heads=None, force=False, bookmarks=(),
  remotebookmarks=None, streamclonerequested=None,
- includepats=None, excludepats=None):
+ includepats=None, excludepats=None, depth=None):
 # repo we pull into
 self.repo = repo
 # repo we pull from
@@ -1350,6 +1350,8 @@
 self.includepats = includepats
 # Set of file patterns to exclude.
 self.excludepats = excludepats
+# Number of ancestor changesets to pull from each pulled head.
+self.depth = depth
 
 @util.propertycache
 def pulledsubset(self):
@@ -1454,7 +1456,8 @@
 pullop.rheads = set(pullop.rheads) - pullop.common
 
 def pull(repo, remote, heads=None, force=False, bookmarks=(), opargs=None,
- streamclonerequested=None, includepats=None, excludepats=None):
+ streamclonerequested=None, includepats=None, excludepats=None,
+ depth=None):
 """Fetch repository data from a remote.
 
 This is the main function used to retrieve data from a remote repository.
@@ -1475,6 +1478,9 @@
 ``includepats`` and ``excludepats`` define explicit file patterns to
 include and exclude in storage, respectively. If not defined, narrow
 patterns from the repo instance are used, if available.
+``depth`` is an integer indicating the DAG depth of history we're
+interested in. If defined, for each revision specified in ``heads``, we
+will fetch up to this many of its ancestors and data associated with them.
 
 Returns the ``pulloperation`` created for this pull.
 """
@@ -1495,6 +1501,7 @@
 pullop = pulloperation(repo, remote, heads, force, bookmarks=bookmarks,
streamclonerequested=streamclonerequested,
includepats=includepats, excludepats=excludepats,
+   depth=depth,
**pycompat.strkwargs(opargs))
 
 peerlocal = pullop.remote.local()



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5135: exchangev2: support for calling rawstorefiledata to retrieve raw files

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is somewhat hacky. For that I apologize.
  
  At the 4.8 Sprint, we decided we wanted to land support in wireprotov2 for 
doing
  a partial clone with changelog and manifestlog bootstrapped from a "stream 
clone"
  like primitive.
  
  This commit implements the client-side bits necessary to facilitate that.
  
  If the new server-side command for obtaining raw files data is available, we
  call it to get the raw files for the changelog and manifestlog. Then we
  fall through to an incremental pull. But when fetching files data, instead
  of using the list of a changesets and manifests that we fetched via the
  "changesetdata" command, we do a linear scan of the repo and resolve the
  changeset and manifest nodes along with the manifest linkrevs.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5135

AFFECTED FILES
  mercurial/exchangev2.py
  tests/test-wireproto-exchangev2.t

CHANGE DETAILS

diff --git a/tests/test-wireproto-exchangev2.t 
b/tests/test-wireproto-exchangev2.t
--- a/tests/test-wireproto-exchangev2.t
+++ b/tests/test-wireproto-exchangev2.t
@@ -963,3 +963,185 @@
   client-narrow-2/.hg/store/00manifest.i
   client-narrow-2/.hg/store/data/dir0/d.i
 #endif
+
+--stream will use rawfiledata to transfer changelog and manifestlog, then
+fall through to get files data
+
+  $ hg --debug clone --stream -U http://localhost:$HGPORT client-stream-0
+  using http://localhost:$HGPORT/
+  sending capabilities command
+  sending 1 commands
+  sending command rawstorefiledata: {
+'files': [
+  'changelog',
+  'manifestlog'
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=1275; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  updating the branch cache
+  query 1; heads
+  sending 2 commands
+  sending command heads: {}
+  sending command known: {
+'nodes': [
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=22; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=11; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=2; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=3; stream=2; streamflags=; 
type=command-response; flags=eos)
+  searching for changes
+  all remote heads known locally
+  sending 1 commands
+  sending command changesetdata: {
+'fields': set([
+  'bookmarks',
+  'parents',
+  'phase',
+  'revision'
+]),
+'revisions': [
+  {
+'heads': [
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'roots': [
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'type': 'changesetdagrange'
+  }
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=13; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  checking for updated bookmarks
+  sending 1 commands
+  sending command filesdata: {
+'fields': set([
+  'parents',
+  'revision'
+]),
+'haveparents': True,
+'revisions': [
+  {
+'nodes': [
+  '3\x90\xef\x85\x00s\xfb\xc2\xf0\xdf\xff"D4,\x8e\x92)\x01:',
+  '\xb7\t8\x08\x92\xb1\x93\xc1\t\x1d:\x81\x7fp`R\xe3F\x82\x1b',
+  'G\xfe\x01*\xb27\xa8\xc7\xfc\x0cx\xf9\xf2mXf\xee\xf3\xf8%',
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'type': 'changesetexplicit'
+  }
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=1133; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; 

D5133: wireprotov2: add response type that serializes to indefinite length bytestring

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This will be needed in a future patch.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5133

AFFECTED FILES
  mercurial/wireprotoframing.py
  mercurial/wireprototypes.py

CHANGE DETAILS

diff --git a/mercurial/wireprototypes.py b/mercurial/wireprototypes.py
--- a/mercurial/wireprototypes.py
+++ b/mercurial/wireprototypes.py
@@ -389,3 +389,12 @@
 fullhashseed = attr.ib(default=None)
 serverdercerts = attr.ib(default=None)
 servercadercerts = attr.ib(default=None)
+
+@attr.s
+class indefinitebytestringresponse(object):
+"""Represents an object to be encoded to an indefinite length bytestring.
+
+Instances are initialized from an iterable of chunks, with each chunk being
+a bytes instance.
+"""
+chunks = attr.ib()
diff --git a/mercurial/wireprotoframing.py b/mercurial/wireprotoframing.py
--- a/mercurial/wireprotoframing.py
+++ b/mercurial/wireprotoframing.py
@@ -1146,6 +1146,14 @@
 for frame in emitter.send(o.data):
 yield frame
 
+elif isinstance(
+o, wireprototypes.indefinitebytestringresponse):
+for chunk in cborutil.streamencodebytestringfromiter(
+o.chunks):
+
+for frame in emitter.send(chunk):
+yield frame
+
 # A regular object is CBOR encoded.
 else:
 for chunk in cborutil.streamencode(o):



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5132: exchangev2: recognize narrow patterns when pulling

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  pulloperation instances were recently taught to record file
  include and exclude patterns to facilitate narrow file transfer.
  Teaching the exchangev2 code to transfer a subset of files is
  as simple as constructing a narrow matcher from these patterns and
  filtering all seen file paths through it.
  
  Keep in mind that this change only influences file data: we're
  still fetching all changeset and manifest data. So, there's still
  a ton of "partial clone" to implement in exchangev2.
  
  On a personal note, I derive gratification that this feature requires
  very few lines of new code to implement.
  
  To test this, we implemented a minimal extension which allows us to specify
  --include/--exclude to clone. While the narrow extension provides these
  arguments, I explicitly wanted to test this functionality without the
  narrow extension enabled, as that extension monkeypatches various things
  and I want to isolate the behavior of core Mercurial.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5132

AFFECTED FILES
  mercurial/exchangev2.py
  tests/pullext.py
  tests/test-wireproto-exchangev2.t

CHANGE DETAILS

diff --git a/tests/test-wireproto-exchangev2.t 
b/tests/test-wireproto-exchangev2.t
--- a/tests/test-wireproto-exchangev2.t
+++ b/tests/test-wireproto-exchangev2.t
@@ -619,3 +619,347 @@
   $ hg -R client-bookmarks bookmarks
  book-12:cd2534766bec
  book-22:cd2534766bec
+
+  $ killdaemons.py
+
+Let's set up a slightly more complicated server
+
+  $ hg init server-2
+  $ enablehttpv2 server-2
+  $ cd server-2
+  $ mkdir dir0 dir1
+  $ echo a0 > a
+  $ echo b0 > b
+  $ hg -q commit -A -m 'commit 0'
+  $ echo c0 > dir0/c
+  $ echo d0 > dir0/d
+  $ hg -q commit -A -m 'commit 1'
+  $ echo e0 > dir1/e
+  $ echo f0 > dir1/f
+  $ hg -q commit -A -m 'commit 2'
+  $ echo c1 > dir0/c
+  $ echo e1 > dir1/e
+  $ hg commit -m 'commit 3'
+  $ hg serve -p $HGPORT -d --pid-file hg.pid -E error.log
+  $ cat hg.pid > $DAEMON_PIDS
+
+  $ cd ..
+
+Narrow clone only fetches some files
+
+  $ hg --config extensions.pullext=$TESTDIR/pullext.py --debug clone -U 
--include dir0/ http://localhost:$HGPORT/ client-narrow-0
+  using http://localhost:$HGPORT/
+  sending capabilities command
+  query 1; heads
+  sending 2 commands
+  sending command heads: {}
+  sending command known: {
+'nodes': []
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=22; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  received frame(size=11; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=1; request=3; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=3; stream=2; streamflags=; 
type=command-response; flags=eos)
+  sending 1 commands
+  sending command changesetdata: {
+'fields': set([
+  'bookmarks',
+  'parents',
+  'phase',
+  'revision'
+]),
+'revisions': [
+  {
+'heads': [
+  '\x97v_\xc3\xcdbO\xd1\xfa\x01v\x93,!\xff\xd1j\xdfC.'
+],
+'roots': [],
+'type': 'changesetdagrange'
+  }
+]
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=783; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=0; request=1; stream=2; streamflags=; 
type=command-response; flags=eos)
+  add changeset 3390ef850073
+  add changeset b709380892b1
+  add changeset 47fe012ab237
+  add changeset 97765fc3cd62
+  checking for updated bookmarks
+  sending 1 commands
+  sending command manifestdata: {
+'fields': set([
+  'parents',
+  'revision'
+]),
+'haveparents': True,
+'nodes': [
+  '\x99/Gy\x02\x9a=\xf8\xd0fm\x00\xbb\x92OicN',
+  '|2 \x1a\xa3\xa1R\xa9\xe6\xa9"+?\xa8\xd0\xe3\x0f\xc2V\xe8',
+  '\x8d\xd0W<\x7f\xaf\xe2\x04F\xcc\xea\xac\x05N\xea\xa4x\x91M\xdb',
+  '113\x85\xf2!\x8b\x08^\xb2Z\x821\x1e*\xdd\x0e\xeb\x8c3'
+],
+'tree': ''
+  }
+  received frame(size=9; request=1; stream=2; streamflags=stream-begin; 
type=stream-settings; flags=eos)
+  received frame(size=11; request=1; stream=2; streamflags=encoded; 
type=command-response; flags=continuation)
+  received frame(size=967; request=1; stream=2; streamflags=encoded; 

D5131: remotefilelog: add some docstring

2018-10-17 Thread durin42 (Augie Fackler)
durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5131

AFFECTED FILES
  hgext/remotefilelog/__init__.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -6,6 +6,17 @@
 # GNU General Public License version 2 or any later version.
 """remotefilelog causes Mercurial to lazilly fetch file contents (EXPERIMENTAL)
 
+This extension is HIGHLY EXPERIMENTAL. There are NO BACKWARDS COMPATIBILITY
+GUARANTEES. This means that repositories created with this extension may
+only be usable with the exact version of this extension/Mercurial that was
+used. The extension attempts to enforce this in order to prevent repository
+corruption.
+
+remotefilelog works by fetching file contents lazily and storing them
+in a cache on the client rather than in revlogs. This allows enormous
+histories to be transferred only partially, making them easier to
+operate on.
+
 Configs:
 
 ``packs.maxchainlen`` specifies the maximum delta chain length in pack 
files



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[Bug 6005] New: command declaration broken for extensions

2018-10-17 Thread mercurial-bugs
https://bz.mercurial-scm.org/show_bug.cgi?id=6005

Bug ID: 6005
   Summary: command declaration broken for extensions
   Product: Mercurial
   Version: default branch
  Hardware: PC
OS: Linux
Status: UNCONFIRMED
  Severity: bug
  Priority: wish
 Component: Mercurial
  Assignee: bugzi...@mercurial-scm.org
  Reporter: lothiral...@gmail.com
CC: mercurial-devel@mercurial-scm.org

Changeset fa88170c10bb changes the way we declare command name that should be
present in the shortlist.

Previously, such names were prefixed with "^", now we use an explicit
declaration in the command decorator.

This change has the unfortunate side effect of breaking all extensions
declaring command with leading "^".

We should keep the compatibility around for at least a cycle, with a developer
warning.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[Bug 6004] New: title: statprof skips many lines when displaying hotpath

2018-10-17 Thread mercurial-bugs
https://bz.mercurial-scm.org/show_bug.cgi?id=6004

Bug ID: 6004
   Summary: title: statprof skips many lines when displaying
hotpath
   Product: Mercurial
   Version: default branch
  Hardware: PC
OS: Linux
Status: UNCONFIRMED
  Severity: bug
  Priority: wish
 Component: Mercurial
  Assignee: bugzi...@mercurial-scm.org
  Reporter: lothiral...@gmail.com
CC: mercurial-devel@mercurial-scm.org

Since the python3 compatibility series, hotpath display is broken. Most lines
are skipped from the output. Only "grey" line appears, the white and red ones
do not.

Backing out the full series fix the issue:

2864f8d3fcd6 + b594db74dc13 + 7df42042636d + 9d3034348c4f + 113adb1b3f24 +
4b7eb862692e

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


[Bug 6003] New: statprof.py no longer gather results

2018-10-17 Thread mercurial-bugs
https://bz.mercurial-scm.org/show_bug.cgi?id=6003

Bug ID: 6003
   Summary: statprof.py no longer gather results
   Product: Mercurial
   Version: default branch
  Hardware: PC
OS: Linux
Status: UNCONFIRMED
  Severity: bug
  Priority: wish
 Component: Mercurial
  Assignee: bugzi...@mercurial-scm.org
  Reporter: lothiral...@gmail.com
CC: mercurial-devel@mercurial-scm.org

After changeset 2864f8d3fcd6, hg --profile no longer display any output.
Digging a bit, it seems to not gather any data during the run.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5130: lfs: autoload the extension when cloning from repo with lfs enabled

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg added a comment.


  > ! In https://phab.mercurial-scm.org/D5130#76736, @mharbison72 wrote:
  >  Will do.  How hard do you think it will be to do the same thing for pull?
  
  I don't think it should be too difficult to implement. But I have possibly 
controversial opinions about the UI. e.g. I'm not a fan of mutating repo 
requirements after repo creation without the user's explicit consent. (But not 
being able to pull from a remote that recently added LFS is a bad experience as 
well.)
  
  > Also, I need to see if this can be made to work with cloning from a local 
repo.
  
  I'm **really** not a fan of local / hardlink clones. They are very brittle. 
When you start throwing extra repo features like LFS into the mix, it becomes 
even more dangerous. Especially when we start talking about adding modifying 
`.hg/requires` automatically. The fundamental problem is the hardlink clone is 
strictly separate but it "inherits" some state from its parent. If we start 
varying behavior of the two repos, interaction between them can be wonky. In 
general, for advanced features, we should go through the "hg clone --pull" 
interface to create a clean break and minimize potential for wonkiness.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5130

To: mharbison72, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5130: lfs: autoload the extension when cloning from repo with lfs enabled

2018-10-17 Thread mharbison72 (Matt Harbison)
mharbison72 added a comment.


  In https://phab.mercurial-scm.org/D5130#76727, @indygreg wrote:
  
  > As a follow-up, it would be nice to have the LFS wire protocol capabilities 
documented in `mercurial/help/internals/wireprotocol.txt`, especially since it 
isn't obvious how the now 2 capabilities are different.
  
  
  Will do.  How hard do you think it will be to do the same thing for pull?  
Also, I need to see if this can be made to work with cloning from a local repo.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5130

To: mharbison72, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4931: narrow: only send the narrowspecs back if ACL in play

2018-10-17 Thread pulkit (Pulkit Goyal)
pulkit updated this revision to Diff 12192.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4931?vs=11774=12192

REVISION DETAIL
  https://phab.mercurial-scm.org/D4931

AFFECTED FILES
  mercurial/exchange.py
  tests/test-narrow-rebase.t
  tests/test-narrow-widen-no-ellipsis.t

CHANGE DETAILS

diff --git a/tests/test-narrow-widen-no-ellipsis.t 
b/tests/test-narrow-widen-no-ellipsis.t
--- a/tests/test-narrow-widen-no-ellipsis.t
+++ b/tests/test-narrow-widen-no-ellipsis.t
@@ -137,7 +137,6 @@
   adding file changes
   added 5 changesets with 4 changes to 2 files
   new changesets *:* (glob)
-  3 local changesets published
   (run 'hg update' to get a working copy)
   $ hg update -r 'desc("add wider")'
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
diff --git a/tests/test-narrow-rebase.t b/tests/test-narrow-rebase.t
--- a/tests/test-narrow-rebase.t
+++ b/tests/test-narrow-rebase.t
@@ -86,7 +86,6 @@
 
   $ hg update -q 'desc("conflicting outside/f1")'
   $ hg phase -f -d .
-  no phases changed
   $ hg rebase -d 'desc("modify outside/f1")'
   rebasing 4:707c035aadb6 "conflicting outside/f1"
   abort: conflict in file 'outside/f1' is outside narrow clone
diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -1900,6 +1900,7 @@
 new_args = {}
 new_args.update(kwargs)
 new_args[r'narrow'] = True
+new_args[r'narrow_acl'] = True
 new_args[r'includepats'] = req_includes
 if req_excludes:
 new_args[r'excludepats'] = req_excludes
@@ -2164,7 +2165,8 @@
 if 'treemanifest' in repo.requirements:
 part.addparam('treemanifest', '1')
 
-if kwargs.get(r'narrow', False) and (include or exclude):
+if (kwargs.get(r'narrow', False) and kwargs.get('narrow_acl', False)
+and (include or exclude)):
 narrowspecpart = bundler.newpart('narrow:spec')
 if include:
 narrowspecpart.addparam(



To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4928: sqlitestore: file storage backend using SQLite

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg updated this revision to Diff 12191.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D4928?vs=12167=12191

REVISION DETAIL
  https://phab.mercurial-scm.org/D4928

AFFECTED FILES
  hgext/sqlitestore.py
  tests/hghave.py
  tests/test-sqlitestore.t
  tests/test-storage.py

CHANGE DETAILS

diff --git a/tests/test-storage.py b/tests/test-storage.py
--- a/tests/test-storage.py
+++ b/tests/test-storage.py
@@ -17,6 +17,16 @@
 storage as storagetesting,
 )
 
+from hgext import (
+sqlitestore,
+)
+
+try:
+from mercurial import zstd
+zstd.__version__
+except ImportError:
+zstd = None
+
 STATE = {
 'lastindex': 0,
 'ui': uimod.ui(),
@@ -70,5 +80,42 @@
  maketransaction,
  addrawrevision)
 
+def makesqlitefile(self):
+path = STATE['vfs'].join(b'db-%d.db' % STATE['lastindex'])
+STATE['lastindex'] += 1
+
+db = sqlitestore.makedb(path)
+
+compression = b'zstd' if zstd else b'zlib'
+
+return sqlitestore.sqlitefilestore(db, b'dummy-path', compression)
+
+def addrawrevisionsqlite(self, fl, tr, node, p1, p2, linkrev, rawtext=None,
+ delta=None, censored=False, ellipsis=False,
+ extstored=False):
+flags = 0
+
+if censored:
+flags |= sqlitestore.FLAG_CENSORED
+
+if ellipsis | extstored:
+raise error.Abort(b'support for ellipsis and extstored flags not '
+  b'supported')
+
+if rawtext is not None:
+fl._addrawrevision(node, rawtext, tr, linkrev, p1, p2, flags=flags)
+elif delta is not None:
+fl._addrawrevision(node, rawtext, tr, linkrev, p1, p2,
+   storedelta=delta, flags=flags)
+else:
+raise error.Abort(b'must supply rawtext or delta arguments')
+
+sqlitefileindextests = storagetesting.makeifileindextests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+sqlitefiledatatests = storagetesting.makeifiledatatests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+sqlitefilemutationtests = storagetesting.makeifilemutationtests(
+makesqlitefile, maketransaction, addrawrevisionsqlite)
+
 if __name__ == '__main__':
 silenttestrunner.main(__name__)
diff --git a/tests/test-sqlitestore.t b/tests/test-sqlitestore.t
new file mode 100644
--- /dev/null
+++ b/tests/test-sqlitestore.t
@@ -0,0 +1,110 @@
+#require sqlite
+
+  $ cat >> $HGRCPATH < [extensions]
+  > sqlitestore =
+  > EOF
+
+New repo should not use SQLite by default
+
+  $ hg init empty-no-sqlite
+  $ cat empty-no-sqlite/.hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+storage.new-repo-backend=sqlite is recognized
+
+  $ hg --config storage.new-repo-backend=sqlite init empty-sqlite
+  $ cat empty-sqlite/.hg/requires
+  dotencode
+  exp-sqlite-001
+  exp-sqlite-comp-001=zstd (zstd !)
+  exp-sqlite-comp-001-zlib (no-zstd !)
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+  $ cat >> $HGRCPATH << EOF
+  > [storage]
+  > new-repo-backend = sqlite
+  > EOF
+
+Can force compression to zlib
+
+  $ hg --config storage.sqlite.compression=zlib init empty-zlib
+  $ cat empty-zlib/.hg/requires
+  dotencode
+  exp-sqlite-001
+  exp-sqlite-comp-001=$BUNDLE2_COMPRESSIONS$
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+Can force compression to none
+
+  $ hg --config storage.sqlite.compression=none init empty-none
+  $ cat empty-none/.hg/requires
+  dotencode
+  exp-sqlite-001
+  exp-sqlite-comp-001=none
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+Can make a local commit
+
+  $ hg init local-commit
+  $ cd local-commit
+  $ echo 0 > foo
+  $ hg commit -A -m initial
+  adding foo
+
+That results in a row being inserted into various tables
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM filepath;
+  > EOF
+  1|foo
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM fileindex;
+  > EOF
+  
1|1|0|-1|-1|0|0|1||6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe
 (esc)
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM delta;
+  > EOF
+  1|1| \xd2\xaf\x8d\xd2"\x01\xdd\x8dH\xe5\xdc\xfc\xae\xd2\x81\xff\x94"\xc7|0 
(esc)
+  
+
+Tracking multiple files works
+
+  $ echo 1 > bar
+  $ hg commit -A -m 'add bar'
+  adding bar
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM filedata ORDER BY id ASC;
+  > EOF
+  
1|1|foo|0|6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe|-1|-1|0|0|1|
 (esc)
+  2|2|bar|0|\xb8\xe0/d3s\x80!\xa0e\xf9Au\xc7\xcd#\xdb_\x05\xbe|-1|-1|1|0|2| 
(esc)
+
+Multiple revisions of a file works
+
+  $ echo a >> foo
+  $ hg commit -m 'modify foo'
+
+  $ sqlite3 .hg/store/db.sqlite << EOF
+  > SELECT * FROM filedata ORDER BY id ASC;
+  > EOF
+  
1|1|foo|0|6/\xef(L\xe2\xca\x02\xae\xcc\x8d\xe6\xd5\xe8\xa1\xc3\xaf\x05V\xfe|-1|-1|0|0|1|
 (esc)
+  

D4767: exchangev2: recognize narrow patterns when pulling

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg abandoned this revision.
indygreg added a comment.


  I'll resubmit this as part of another series.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4767

To: indygreg, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4766: fastcheckout: arguments to make checkout narrow

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg abandoned this revision.
indygreg added a comment.


  Per discussions at 4.8 sprint, we're going to hold off on this.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4766

To: indygreg, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D4765: fastcheckout: extension for obtaining a checkout quickly

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg abandoned this revision.
indygreg added a comment.


  Per discussions at 4.8 sprint, we're going to hold off on this.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4765

To: indygreg, #hg-reviewers
Cc: mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5130: lfs: autoload the extension when cloning from repo with lfs enabled

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg added a comment.


  Thank you for finishing what I started and implementing this properly (by 
adding a separate server-side capability to distinguish between LFS states).
  
  As a follow-up, it would be nice to have the LFS wire protocol capabilities 
documented in `mercurial/help/internals/wireprotocol.txt`, especially since it 
isn't obvious how the now 2 capabilities are different.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5130

To: mharbison72, indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5130: lfs: autoload the extension when cloning from repo with lfs enabled

2018-10-17 Thread mharbison72 (Matt Harbison)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG6637b079ae45: lfs: autoload the extension when cloning from 
repo with lfs enabled (authored by mharbison72, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D5130?vs=12189=12190#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5130?vs=12189=12190

REVISION DETAIL
  https://phab.mercurial-scm.org/D5130

AFFECTED FILES
  hgext/lfs/wrapper.py
  mercurial/hg.py
  mercurial/localrepo.py
  tests/test-lfs-serve-access.t
  tests/test-lfs-serve.t

CHANGE DETAILS

diff --git a/tests/test-lfs-serve.t b/tests/test-lfs-serve.t
--- a/tests/test-lfs-serve.t
+++ b/tests/test-lfs-serve.t
@@ -304,10 +304,10 @@
   $ grep 'lfs' .hg/requires $SERVER_REQUIRES
   $TESTTMP/server/.hg/requires:lfs
 
-TODO: fail more gracefully.
-
-  $ hg clone -q http://localhost:$HGPORT $TESTTMP/client4_clone
-  abort: HTTP Error 500: Internal Server Error
+  $ hg clone http://localhost:$HGPORT $TESTTMP/client4_clone
+  (remote is using large file support (lfs), but it is explicitly disabled in 
the local configuration)
+  abort: repository requires features unknown to this Mercurial: lfs!
+  (see https://mercurial-scm.org/wiki/MissingRequirement for more information)
   [255]
   $ grep 'lfs' $TESTTMP/client4_clone/.hg/requires $SERVER_REQUIRES
   grep: $TESTTMP/client4_clone/.hg/requires: $ENOENT$
@@ -661,8 +661,6 @@
   $ cat $TESTTMP/errors.log | grep '^[A-Z]'
   Traceback (most recent call last):
   ValueError: no common changegroup version
-  Traceback (most recent call last):
-  ValueError: no common changegroup version
 #else
   $ cat $TESTTMP/errors.log
 #endif
diff --git a/tests/test-lfs-serve-access.t b/tests/test-lfs-serve-access.t
--- a/tests/test-lfs-serve-access.t
+++ b/tests/test-lfs-serve-access.t
@@ -42,6 +42,7 @@
 Downloads fail...
 
   $ hg clone http://localhost:$HGPORT httpclone
+  (remote is using large file support (lfs); lfs will be enabled for this 
repository)
   requesting all changes
   adding changesets
   adding manifests
@@ -76,6 +77,7 @@
   $ hg clone --debug http://localhost:$HGPORT/subdir/mount/point cloned2
   using http://localhost:$HGPORT/subdir/mount/point
   sending capabilities command
+  (remote is using large file support (lfs); lfs will be enabled for this 
repository)
   query 1; heads
   sending batch command
   requesting all changes
@@ -88,7 +90,6 @@
   adding file changes
   adding lfs.bin revisions
   added 1 changesets with 1 changes to 1 files
-  calling hook pretxnchangegroup.lfs: hgext.lfs.checkrequireslfs
   bundle2-input-part: total payload size 648
   bundle2-input-part: "listkeys" (params: 1 mandatory) supported
   bundle2-input-part: "phase-heads" supported
@@ -239,6 +240,7 @@
 Test an I/O error in localstore.verify() (Batch API) with GET
 
   $ hg clone http://localhost:$HGPORT1 httpclone2
+  (remote is using large file support (lfs); lfs will be enabled for this 
repository)
   requesting all changes
   adding changesets
   adding manifests
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -2895,6 +2895,9 @@
 if createopts.get('narrowfiles'):
 requirements.add(repository.NARROW_REQUIREMENT)
 
+if createopts.get('lfs'):
+requirements.add('lfs')
+
 return requirements
 
 def filterknowncreateopts(ui, createopts):
@@ -2913,6 +2916,7 @@
 """
 known = {
 'backend',
+'lfs',
 'narrowfiles',
 'sharedrepo',
 'sharedrelative',
@@ -2931,6 +2935,9 @@
 
 backend
The storage backend to use.
+lfs
+   Repository will be created with ``lfs`` requirement. The lfs extension
+   will automatically be loaded when the repository is accessed.
 narrowfiles
Set up repository to support narrow file storage.
 sharedrepo
diff --git a/mercurial/hg.py b/mercurial/hg.py
--- a/mercurial/hg.py
+++ b/mercurial/hg.py
@@ -578,6 +578,23 @@
 
 createopts['narrowfiles'] = True
 
+if srcpeer.capable(b'lfs-serve'):
+# Repository creation honors the config if it disabled the extension, 
so
+# we can't just announce that lfs will be enabled.  This check avoids
+# saying that lfs will be enabled, and then saying it's an unknown
+# feature.  The lfs creation option is set in either case so that a
+# requirement is added.  If the extension is explicitly disabled but 
the
+# requirement is set, the clone aborts early, before transferring any
+# data.
+createopts['lfs'] = True
+
+if extensions.disabledext('lfs'):
+ui.status(_('(remote is using large file support (lfs), but it is '
+'explicitly disabled in the local configuration)\n'))
+else:
+ui.status(_('(remote is using large file support (lfs); lfs will '
+'be enabled for 

D4782: remotefilelog: import pruned-down remotefilelog extension from hg-experimental

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg added a comment.


  I need to give the final state of the code a final one-over. I will do that 
after breakfast.
  
  One thing that is missing is a detailed docstring for the extension. That can 
be added as a follow-up. I think said docstring should discourage use of the 
extension and explicitly state that there is a possibility much of the 
functionality in the extension never stabilizes without major BC breakage. The 
sqlitestore extension may have some language worth borrowing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4782

To: durin42, #hg-reviewers
Cc: indygreg, spectral, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5123: remotefilelogserver: remove pack-serving functionality

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg accepted this revision.
indygreg added a comment.
This revision is now accepted and ready to land.


  The optics to removing this are a bit bad because efficient pack transfer can 
lead to significant performance gains. While I think the use of packs in 
Mercurial for exchange will occur at some time, the path to stabilization in 
core is not clear and will almost certainly see a BC break from this code. I'm 
fine removing this functionality from RFL for now. We can always add something 
similar later.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5123

To: durin42, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5129: remotefilelog: rename wireproto methods and remaining capabilities

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg accepted this revision.
indygreg added a comment.
This revision is now accepted and ready to land.


  Yuck. But the names don't matter much for an experimental feature. And the 
wireproto v1 peer batching code dictates certain things with regards to naming. 
So our hands are a bit tied here :/

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5129

To: durin42, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5128: remotefilelog: consolidate and rename on-disk store requirement

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg accepted this revision.
indygreg added a comment.
This revision is now accepted and ready to land.


  \o/

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5128

To: durin42, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


D5127: remotefilelog: consolidate and rename bundle2 capability

2018-10-17 Thread indygreg (Gregory Szorc)
indygreg accepted this revision.
indygreg added a comment.
This revision is now accepted and ready to land.


  Nice catch on the overloaded capability name! Constants are better for 
expressing intent.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5127

To: durin42, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel