date:20181113

D5064: style: run black on a subset of mercurial

2018-11-13 Thread av6 (Anton Shestakov)

av6 added a comment.


  I look at the changes and see nitpicks at best. On the one hand, black proved 
better than any linter that we can already write consistent code. On the other, 
if black were a linter... I'd switch to flake8, which at least is configurable 
.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5064

To: mjpieters, indygreg, durin42, #hg-reviewers
Cc: av6, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] tests: document a known failing interaction between narrow and lfs

2018-11-13 Thread Matt Harbison

# HG changeset patch
# User Matt Harbison 
# Date 1542171263 18000
#  Tue Nov 13 23:54:23 2018 -0500
# Node ID 9b3e22e87fc38c5e2ffd66b5ba044be810cdd500
# Parent  d2c997b8001ff5fff0ae3d042362f321a707dd55
tests: document a known failing interaction between narrow and lfs

This is one of the two remaining aborts I found looking into issue5794.  I've
got no idea what's wrong with the hook, since the changes there fixed the other
two problems noted in that bug report.  It seems like it might go away when the
narrow issue is fixed, but let's make sure this doesn't get lost.

The stacktrace for the hook seems to indicate that the missing file *is* in ctx:

  remote: Traceback (most recent call last):
  remote:   File "c:\Users\Matt\projects\hg\hgext\lfs\__init__.py", line 253, 
in checkrequireslfs
  remote: if any(f in ctx and match(f) and ctx[f].islfs() for f in 
ctx.files()):
  remote:   File "c:\Users\Matt\projects\hg\hgext\lfs\__init__.py", line 253, 
in 
  remote: if any(f in ctx and match(f) and ctx[f].islfs() for f in 
ctx.files()):
  remote:   File "c:\Users\Matt\projects\hg\hgext\lfs\wrapper.py", line 191, in 
filectxislfs
  remote: return _islfs(self.filelog(), self.filenode())
  remote:   File "c:\Users\Matt\projects\hg\mercurial\context.py", line 631, in 
filenode
  remote: return self._filenode
  remote:   File "c:\Users\Matt\projects\hg\mercurial\util.py", line 1528, in 
__get__
  remote: result = self.func(obj)
  remote:   File "c:\Users\Matt\projects\hg\mercurial\context.py", line 579, in 
_filenode
  remote: return self._filelog.lookup(self._fileid)
  remote:   File "c:\Users\Matt\projects\hg\mercurial\filelog.py", line 68, in 
lookup
  remote: self._revlog.indexfile)
  remote:   File "c:\Users\Matt\projects\hg\mercurial\utils\storageutil.py", 
line 218, in fileidlookup
  remote: raise error.LookupError(fileid, identifier, _('no match found'))
  remote: LookupError: data/inside2/f.i@f59b4e021835: no match found

diff --git a/tests/test-narrow-exchange.t b/tests/test-narrow-exchange.t
--- a/tests/test-narrow-exchange.t
+++ b/tests/test-narrow-exchange.t
@@ -1,3 +1,11 @@
+#testcases lfs-on lfs-off
+
+#if lfs-on
+  $ cat >> $HGRCPATH < [extensions]
+  > lfs =
+  > EOF
+#endif
 
   $ . "$TESTDIR/narrow-library.sh"
 
@@ -201,10 +209,17 @@ not also in narrower repo
   (run 'hg update' to get a working copy)
 TODO: this should tell the user that their narrow clone does not have the
 necessary content to be able to push to the target
-  $ hg push ssh://user@dummy/narrow2
+
+TODO: lfs shouldn't abort like this
+  $ hg push ssh://user@dummy/narrow2 || true
   pushing to ssh://user@dummy/narrow2
   searching for changes
   remote: adding changesets
   remote: adding manifests
   remote: adding file changes
   remote: added 1 changesets with 0 changes to 0 files
+  remote: error: pretxnchangegroup.lfs hook raised an exception: 
data/inside2/f.i@f59b4e021835: no match found (lfs-on !)
+  remote: transaction abort! (lfs-on !)
+  remote: rollback completed (lfs-on !)
+  remote: abort: data/inside2/f.i@f59b4e021835: no match found! (lfs-on !)
+  abort: stream ended unexpectedly (got 0 bytes, expected 4) (lfs-on !)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5243: resolve: fix mark-check when a file was deleted on one side (issue6020)

2018-11-13 Thread spectral (Kyle Lippincott)

spectral planned changes to this revision.
spectral added a comment.


  (not sure when I'll get to the changes, but removing from Needs Review, sorry 
I delayed so long doing that)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5243

To: spectral, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5268: shelve: use matcher to restrict prefetch to just the modified files

2018-11-13 Thread spectral (Kyle Lippincott)

spectral created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Shelve currently operates by:
  
  - make a temp commit
  - identify all the bases necessary to shelve, put them in the bundle
  - use exportfile to export the temp commit to the bundle ('file' here means 
"export to this fd", not "export this file")
  - remove the temp commit
  
  exportfile calls prefetchfiles, and prefetchfiles uses a matcher to restrict
  what files it's going to prefetch; if it's not provided, it's alwaysmatcher.
  This means that `hg shelve` in a remotefilelog repo can possibly download the
  file contents of everything in the repository, even when it doesn't need to. 
It
  luckily is restricted to the narrowspec (if there is one), but this is still a
  lot of downloading that's just unnecessary, especially if there's a "smart"
  VCS-aware filesystem involved.
  
  exportfile is called with exactly one revision to emit, so we're just
  restricting it to prefetching the files from that revision. The base revisions
  having separate files should not be a concern since they're handled already;
  example:
  
  commit 10 is draft and modifies foo/a.txt and foo/b.txt
  commit 11 is draft and modifies foo/a.txt
  my working directory that I'm shelving modifies foo/b.txt
  
  By the time we get to exportfile, commit 10 and 11 are already handled, so the
  matcher only specifying foo/b.txt does not cause any problems. I verified this
  by doing an `hg unbundle` on the bundle that shelve produces, and getting the
  full contents of those commits back out, instead of just the files that were
  modified in the shelve.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5268

AFFECTED FILES
  hgext/shelve.py

CHANGE DETAILS

diff --git a/hgext/shelve.py b/hgext/shelve.py
--- a/hgext/shelve.py
+++ b/hgext/shelve.py
@@ -430,8 +430,12 @@
 shelvedfile(repo, name, 'shelve').writeinfo(info)
 bases = list(mutableancestors(repo[node]))
 shelvedfile(repo, name, 'hg').writebundle(bases, node)
+# Create a matcher so that prefetch doesn't attempt to fetch the entire
+# repository pointlessly.
+match = scmutil.matchfiles(repo, repo[node].files())
 with shelvedfile(repo, name, patchextension).opener('wb') as fp:
-cmdutil.exportfile(repo, [node], fp, opts=mdiff.diffopts(git=True))
+cmdutil.exportfile(repo, [node], fp, opts=mdiff.diffopts(git=True),
+   match=match)
 
 def _includeunknownfiles(repo, pats, opts, extra):
 s = repo.status(match=scmutil.match(repo[None], pats, opts),



To: spectral, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5064: style: run black on a subset of mercurial

2018-11-13 Thread indygreg (Gregory Szorc)

indygreg added a comment.


  Where do we stand on the intent to mass reformat the code base?
  
  I'm not super thrilled at some of black's decisions (like using double quotes 
for all strings and merging imports into the same line which leads to excessive 
code churn later). But the exact style doesn't matter as much as having it be 
consistent. So I'm willing to go with black if we feel it's the best tool for 
the job. I'm not sure what the alternatives are these days.

INLINE COMMENTS

> mergeutil.py:18
> +raise error.Abort(
> +_("unresolved merge conflicts " "(see 'hg help resolve')")
> +)

I'm surprised by this result. I'd like to think the reformatting tool would be 
smarter than this.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5064

To: mjpieters, indygreg, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@40614: 5 new changesets

2018-11-13 Thread Mercurial Commits

5 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/13d4ad8d7801
changeset:   40610:13d4ad8d7801
user:Pulkit Goyal 
date:Tue Nov 13 17:41:26 2018 +0300
summary: py3: fix keyword arguments handling in hgext/remotefilelog/

https://www.mercurial-scm.org/repo/hg/rev/5e3b3f88c2e4
changeset:   40611:5e3b3f88c2e4
user:Pulkit Goyal 
date:Tue Nov 13 17:41:54 2018 +0300
summary: py3: make second argument of fdopen() a str

https://www.mercurial-scm.org/repo/hg/rev/3fa4183e7803
changeset:   40612:3fa4183e7803
user:Pulkit Goyal 
date:Tue Nov 13 18:07:21 2018 +0300
summary: py3: use node.hex(h.digest()) instead of h.hexdigest()

https://www.mercurial-scm.org/repo/hg/rev/9769e0f6ffe0
changeset:   40613:9769e0f6ffe0
user:Pulkit Goyal 
date:Tue Nov 13 18:08:17 2018 +0300
summary: py3: don't use dict.iterkeys()

https://www.mercurial-scm.org/repo/hg/rev/aa588bf40a08
changeset:   40614:aa588bf40a08
bookmark:@
tag: tip
user:Pulkit Goyal 
date:Tue Nov 13 18:08:55 2018 +0300
summary: py3: add b suffix to make sure file is opened in bytes mode

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5170: branchmap: pass changelog into revbranchcache.branchinfo()

2018-11-13 Thread indygreg (Gregory Szorc)

indygreg added a comment.


  I think this patch can be abandoned because of other work late in the 4.8 
cycle?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5170

To: pulkit, #hg-reviewers
Cc: indygreg, yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] templater: check invalid use of list expression properly (issue5920)

2018-11-13 Thread Augie Fackler

queued, thanks

> On Nov 13, 2018, at 8:39 AM, Yuya Nishihara  wrote:
> 
> # HG changeset patch
> # User Yuya Nishihara 
> # Date 1542114930 -32400
> #  Tue Nov 13 22:15:30 2018 +0900
> # Node ID 5c7dff08eea9b3f05995380e612546c9bc7a17ad
> # Parent  526ee887c4d512e4d2812edaef30908eba3fafc4
> templater: check invalid use of list expression properly (issue5920)
> 
> The error message is still cryptic, but it should be better.
> 
> diff --git a/mercurial/templater.py b/mercurial/templater.py
> --- a/mercurial/templater.py
> +++ b/mercurial/templater.py
> @@ -374,9 +374,7 @@ def compileexp(exp, context, curmethods)
> if not exp:
> raise error.ParseError(_("missing argument"))
> t = exp[0]
> -if t in curmethods:
> -return curmethods[t](exp, context)
> -raise error.ParseError(_("unknown method '%s'") % t)
> +return curmethods[t](exp, context)
> 
> # template evaluation
> 
> @@ -496,6 +494,10 @@ def _buildfuncargs(exp, context, curmeth
> def buildkeyvaluepair(exp, content):
> raise error.ParseError(_("can't use a key-value pair in this context"))
> 
> +def buildlist(exp, context):
> +raise error.ParseError(_("can't use a list in this context"),
> +   hint=_('check place of comma and parens'))
> +
> # methods to interpret function arguments or inner expressions (e.g. {_(x)})
> exprmethods = {
> "integer": lambda e, c: (templateutil.runinteger, e[1]),
> @@ -508,6 +510,7 @@ exprmethods = {
> "%": buildmap,
> "func": buildfunc,
> "keyvalue": buildkeyvaluepair,
> +"list": buildlist,
> "+": lambda e, c: buildarithmetic(e, c, lambda a, b: a + b),
> "-": lambda e, c: buildarithmetic(e, c, lambda a, b: a - b),
> "negate": buildnegate,
> diff --git a/tests/test-template-basic.t b/tests/test-template-basic.t
> --- a/tests/test-template-basic.t
> +++ b/tests/test-template-basic.t
> @@ -188,7 +188,8 @@ Call function which takes named argument
> 
>   $ hg debugtemplate '{" "|separate}'
>   $ hg debugtemplate '{("not", "an", "argument", "list")|separate}'
> -  hg: parse error: unknown method 'list'
> +  hg: parse error: can't use a list in this context
> +  (check place of comma and parens)
>   [255]
> 
> Second branch starting at nullrev:
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH stable v2] tests: fix wireproto redirection test on systems without tls1.2

2018-11-13 Thread Gregory Szorc

On Tue, Nov 13, 2018 at 4:51 PM Augie Fackler  wrote:

> # HG changeset patch
> # User Augie Fackler 
> # Date 1542156468 18000
> #  Tue Nov 13 19:47:48 2018 -0500
> # Branch stable
> # Node ID 5c9d315298fc3c2576a0beeffd994e7f3a8187ed
> # Parent  7e2c58b08e74514541101f5049eea8a1ff4ba4a7
> tests: fix wireproto redirection test on systems without tls1.2
>

Queued for stable. Thanks.


>
> Our automated package builder has some ancient configuration that
> lacks modern TLS, which is how we noticed this.
>
> Tested: the test now passes on both macOS High Sierra (has tls1.2) and
> Ubuntu Trusty (which does not).
>
> diff --git a/tests/test-wireproto-content-redirects.t
> b/tests/test-wireproto-content-redirects.t
> --- a/tests/test-wireproto-content-redirects.t
> +++ b/tests/test-wireproto-content-redirects.t
> @@ -68,17 +68,20 @@ Redirect targets advertised when configu
>s> Content-Length: 2259\r\n
>s> \r\n
>s>
>  
> \xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0003\xa5Hcommands\xacIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa2Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x84IbookmarksGparentsEphaseHrevisionIrevisions\xa2Hrequired\xf5DtypeDlistKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x83HlinknodeGparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullIfilesdata\xa3Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x84NfirstchangesetHlinknodeGparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolJpathfilter\xa3Gdefault\xf6Hrequired\xf4DtypeDdictIrevisions\xa2Hrequired\xf5DtypeDlistKpermissions\x81DpullTrecommendedbatchsize\x19\xc3PEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa3Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullTrecommendedbatchsize\x1a\x00\x01\x86\xa0Gpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushPrawstorefiledata\xa2Dargs\xa2Efiles\xa2Hrequired\xf5DtypeDlistJpathfilter\xa3Gdefault\xf6Hrequired\xf4DtypeDlistKpermissions\x81DpullQframingmediatypes\x81X/mercurial-exp-framing-0006Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Hredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x81\xa5DnameHtarget-aHprotocolDhttpKsnirequired\xf4Ktlsversions\x82C1.2C1.3Duris\x81Shttp://
> example.com/Nv1capabilitiesY\x01\xd3batch
>  branchmap
> $USUAL_BUNDLE2_CAPS$ changegroupsubset compression=$BUNDLE2_COMPRESSIONS$
> getbundle httpheader=1024 httpmediatype=0.1rx,0.1tx,0.2tx known lookup
> pushkey streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN
> unbundlehash
> -  (remote redirect target target-a is compatible)
> +  (remote redirect target target-a is compatible) (tls1.2 !)
> +  (remote redirect target target-a requires unsupported TLS versions:
> 1.2, 1.3) (no-tls1.2 !)
>sending capabilities command
>s> POST /api/exp-http-v2-0003/ro/capabilities HTTP/1.1\r\n
>s> Accept-Encoding: identity\r\n
>s> accept: application/mercurial-exp-framing-0006\r\n
>s> content-type: application/mercurial-exp-framing-0006\r\n
> -  s> content-length: 111\r\n
> +  s> content-length: 111\r\n (tls1.2 !)
> +  s> content-length: 102\r\n (no-tls1.2 !)
>s> host: $LOCALIP:$HGPORT\r\n (glob)
>s> user-agent: Mercurial debugwireproto\r\n
>s> \r\n
> -  s>
>  
> \x1c\x00\x00\x01\x00\x01\x01\x82\xa1Pcontentencodings\x81HidentityC\x00\x00\x01\x00\x01\x00\x11\xa2DnameLcapabilitiesHredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x81Htarget-a
> +  s>
>  
> \x1c\x00\x00\x01\x00\x01\x01\x82\xa1Pcontentencodings\x81HidentityC\x00\x00\x01\x00\x01\x00\x11\xa2DnameLcapabilitiesHredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x81Htarget-a
> (tls1.2 !)
> +  s>
>  
> \x1c\x00\x00\x01\x00\x01\x01\x82\xa1Pcontentencodings\x81Hidentity:\x00\x00\x01\x00\x01\x00\x11\xa2DnameLcapabilitiesHredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x80
> (no-tls1.2 !)
>s> makefile('rb', None)
>s> HTTP/1.1 200 OK\r\n
>s> Server: testing stub value\r\n
> ___
> Mercurial-devel mailing

[PATCH stable v2] tests: fix wireproto redirection test on systems without tls1.2

2018-11-13 Thread Augie Fackler

# HG changeset patch
# User Augie Fackler 
# Date 1542156468 18000
#  Tue Nov 13 19:47:48 2018 -0500
# Branch stable
# Node ID 5c9d315298fc3c2576a0beeffd994e7f3a8187ed
# Parent  7e2c58b08e74514541101f5049eea8a1ff4ba4a7
tests: fix wireproto redirection test on systems without tls1.2

Our automated package builder has some ancient configuration that
lacks modern TLS, which is how we noticed this.

Tested: the test now passes on both macOS High Sierra (has tls1.2) and
Ubuntu Trusty (which does not).

diff --git a/tests/test-wireproto-content-redirects.t 
b/tests/test-wireproto-content-redirects.t
--- a/tests/test-wireproto-content-redirects.t
+++ b/tests/test-wireproto-content-redirects.t
@@ -68,17 +68,20 @@ Redirect targets advertised when configu
   s> Content-Length: 2259\r\n
   s> \r\n
   s> 
\xa3GapibaseDapi/Dapis\xa1Pexp-http-v2-0003\xa5Hcommands\xacIbranchmap\xa2Dargs\xa0Kpermissions\x81DpullLcapabilities\xa2Dargs\xa0Kpermissions\x81DpullMchangesetdata\xa2Dargs\xa2Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x84IbookmarksGparentsEphaseHrevisionIrevisions\xa2Hrequired\xf5DtypeDlistKpermissions\x81DpullHfiledata\xa2Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x83HlinknodeGparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDpath\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullIfilesdata\xa3Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x84NfirstchangesetHlinknodeGparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolJpathfilter\xa3Gdefault\xf6Hrequired\xf4DtypeDdictIrevisions\xa2Hrequired\xf5DtypeDlistKpermissions\x81DpullTrecommendedbatchsize\x19\xc3PEheads\xa2Dargs\xa1Jpubliconly\xa3Gdefault\xf4Hrequired\xf4DtypeDboolKpermissions\x81DpullEknown\xa2Dargs\xa1Enodes\xa3Gdefault\x80Hrequired\xf4DtypeDlistKpermissions\x81DpullHlistkeys\xa2Dargs\xa1Inamespace\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullFlookup\xa2Dargs\xa1Ckey\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullLmanifestdata\xa3Dargs\xa4Ffields\xa4Gdefault\xd9\x01\x02\x80Hrequired\xf4DtypeCsetKvalidvalues\xd9\x01\x02\x82GparentsHrevisionKhaveparents\xa3Gdefault\xf4Hrequired\xf4DtypeDboolEnodes\xa2Hrequired\xf5DtypeDlistDtree\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpullTrecommendedbatchsize\x1a\x00\x01\x86\xa0Gpushkey\xa2Dargs\xa4Ckey\xa2Hrequired\xf5DtypeEbytesInamespace\xa2Hrequired\xf5DtypeEbytesCnew\xa2Hrequired\xf5DtypeEbytesCold\xa2Hrequired\xf5DtypeEbytesKpermissions\x81DpushPrawstorefiledata\xa2Dargs\xa2Efiles\xa2Hrequired\xf5DtypeDlistJpathfilter\xa3Gdefault\xf6Hrequired\xf4DtypeDlistKpermissions\x81DpullQframingmediatypes\x81X/mercurial-exp-framing-0006Rpathfilterprefixes\xd9\x01\x02\x82Epath:Lrootfilesin:Nrawrepoformats\x82LgeneraldeltaHrevlogv1Hredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x81\xa5DnameHtarget-aHprotocolDhttpKsnirequired\xf4Ktlsversions\x82C1.2C1.3Duris\x81Shttp://example.com/Nv1capabilitiesY\x01\xd3batch
 branchmap $USUAL_BUNDLE2_CAPS$ changegroupsubset 
compression=$BUNDLE2_COMPRESSIONS$ getbundle httpheader=1024 
httpmediatype=0.1rx,0.1tx,0.2tx known lookup pushkey 
streamreqs=generaldelta,revlogv1 unbundle=HG10GZ,HG10BZ,HG10UN unbundlehash
-  (remote redirect target target-a is compatible)
+  (remote redirect target target-a is compatible) (tls1.2 !)
+  (remote redirect target target-a requires unsupported TLS versions: 1.2, 
1.3) (no-tls1.2 !)
   sending capabilities command
   s> POST /api/exp-http-v2-0003/ro/capabilities HTTP/1.1\r\n
   s> Accept-Encoding: identity\r\n
   s> accept: application/mercurial-exp-framing-0006\r\n
   s> content-type: application/mercurial-exp-framing-0006\r\n
-  s> content-length: 111\r\n
+  s> content-length: 111\r\n (tls1.2 !)
+  s> content-length: 102\r\n (no-tls1.2 !)
   s> host: $LOCALIP:$HGPORT\r\n (glob)
   s> user-agent: Mercurial debugwireproto\r\n
   s> \r\n
-  s> 
\x1c\x00\x00\x01\x00\x01\x01\x82\xa1Pcontentencodings\x81HidentityC\x00\x00\x01\x00\x01\x00\x11\xa2DnameLcapabilitiesHredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x81Htarget-a
+  s> 
\x1c\x00\x00\x01\x00\x01\x01\x82\xa1Pcontentencodings\x81HidentityC\x00\x00\x01\x00\x01\x00\x11\xa2DnameLcapabilitiesHredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x81Htarget-a
 (tls1.2 !)
+  s> 
\x1c\x00\x00\x01\x00\x01\x01\x82\xa1Pcontentencodings\x81Hidentity:\x00\x00\x01\x00\x01\x00\x11\xa2DnameLcapabilitiesHredirect\xa2Fhashes\x82Fsha256Dsha1Gtargets\x80
 (no-tls1.2 !)
   s> makefile('rb', None)
   s> HTTP/1.1 200 OK\r\n
   s> Server: testing stub value\r\n
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH stable] tests: fix wireproto redirection test on systems without tls1.2

2018-11-13 Thread Augie Fackler



> On Nov 13, 2018, at 15:43, Augie Fackler  wrote:
> 
> # HG changeset patch
> # User Augie Fackler 
> # Date 1542141748 18000
> #  Tue Nov 13 15:42:28 2018 -0500
> # Branch stable
> # Node ID 53b7b24c957b0ecbfb8c20dab0c820b07b12ddd0
> # Parent  7e2c58b08e74514541101f5049eea8a1ff4ba4a7
> tests: fix wireproto redirection test on systems without tls1.2

ugh, seeing a new failure on old-TLS systems, ignore this. I'll get a v2 out 
tomorrow. :(

> 
> Our automated package builder has some ancient configuration that
> lacks modern TLS, which is how we noticed this.
> 
> diff --git a/tests/test-wireproto-content-redirects.t 
> b/tests/test-wireproto-content-redirects.t
> --- a/tests/test-wireproto-content-redirects.t
> +++ b/tests/test-wireproto-content-redirects.t
> @@ -30,6 +30,7 @@
>0   0 992f4779029a3df8d0666d00bb924f69634e2641 
>  
> 
>1   1 a988fb43583e871d1ed5750ee074c6d840bbbfc8 
> 992f4779029a3df8d0666d00bb924f69634e2641 
> 
> 
> +#if tls1.2
>   $ hg --config simplecache.redirectsfile=redirects.py serve -p $HGPORT -d 
> --pid-file hg.pid -E error.log
>   $ cat hg.pid > $DAEMON_PIDS
> 
> @@ -357,6 +358,7 @@ Redirect targets advertised when configu
> }
>   ]
>   (sent 2 HTTP requests and * bytes; received * bytes in responses) (glob)
> +#endif
> 
> Unknown protocol is filtered from compatible targets
> 
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@40609: 63 new changesets

2018-11-13 Thread Mercurial Commits

63 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/840cd57cde32
changeset:   40547:840cd57cde32
parent:  40543:7bffbbe03e90
user:Yuya Nishihara 
date:Sat Nov 03 19:42:50 2018 +0900
summary: ui: add config knob to redirect status messages to stderr (API)

https://www.mercurial-scm.org/repo/hg/rev/040447dc3c62
changeset:   40548:040447dc3c62
user:Augie Fackler 
date:Tue Nov 06 10:26:33 2018 -0500
summary: remotefilelog: fix various whitespace issues in docstring

https://www.mercurial-scm.org/repo/hg/rev/d6ec45b79277
changeset:   40549:d6ec45b79277
user:Augie Fackler 
date:Tue Nov 06 10:41:00 2018 -0500
summary: tests: fix config knob in test-narrow-clone-stream.t

https://www.mercurial-scm.org/repo/hg/rev/164b2e77f9a5
changeset:   40550:164b2e77f9a5
user:Boris Feld 
date:Wed Oct 03 10:53:29 2018 +0200
summary: perf: introduce a perfrevlogwrite command

https://www.mercurial-scm.org/repo/hg/rev/4dd7edeb3da9
changeset:   40551:4dd7edeb3da9
user:Boris Feld 
date:Wed Oct 03 11:04:57 2018 +0200
summary: perf: offer full details in perfrevlogwrite

https://www.mercurial-scm.org/repo/hg/rev/c63081cd3902
changeset:   40552:c63081cd3902
user:Boris Feld 
date:Tue Nov 06 00:57:34 2018 +0100
summary: perf: only display the total time for perfrevlogwrite if quiet

https://www.mercurial-scm.org/repo/hg/rev/631011ff6771
changeset:   40553:631011ff6771
user:Boris Feld 
date:Fri Oct 19 17:23:29 2018 +0200
summary: perf: add the notion of "source" to perfrevlogwrite

https://www.mercurial-scm.org/repo/hg/rev/6c2357029364
changeset:   40554:6c2357029364
user:Boris Feld 
date:Mon Nov 05 15:15:02 2018 +0100
summary: perf: add `parent-1` as possible source for perfrevlogwrite

https://www.mercurial-scm.org/repo/hg/rev/b5b3dd4e40c3
changeset:   40555:b5b3dd4e40c3
user:Boris Feld 
date:Mon Nov 05 15:15:18 2018 +0100
summary: perf: add `parent-2` as possible source for perfrevlogwrite

https://www.mercurial-scm.org/repo/hg/rev/e14d44772fb3
changeset:   40556:e14d44772fb3
user:Boris Feld 
date:Mon Nov 05 15:19:44 2018 +0100
summary: perf: add `parent-smallest` as possible source for perfrevlogwrite

https://www.mercurial-scm.org/repo/hg/rev/355ae096faef
changeset:   40557:355ae096faef
user:Boris Feld 
date:Mon Nov 05 15:24:09 2018 +0100
summary: perf: add `storage` as possible source for perfrevlogwrite

https://www.mercurial-scm.org/repo/hg/rev/4756a33d0d31
changeset:   40558:4756a33d0d31
user:Boris Feld 
date:Mon Nov 05 15:57:31 2018 +0100
summary: perf: add a lazydeltabase option to perfrevlogwrite

https://www.mercurial-scm.org/repo/hg/rev/26bab5c03e4c
changeset:   40559:26bab5c03e4c
user:Augie Fackler 
date:Tue Nov 06 11:52:41 2018 -0500
summary: py3: ratchet caught two more passing tests

https://www.mercurial-scm.org/repo/hg/rev/8c1f36bf2d3e
changeset:   40560:8c1f36bf2d3e
user:Augie Fackler 
date:Tue Nov 06 11:18:12 2018 -0500
summary: revlog: add a comment to help clang-format produce less-awful 
results

https://www.mercurial-scm.org/repo/hg/rev/5c14bf0c5be3
changeset:   40561:5c14bf0c5be3
user:Augie Fackler 
date:Tue Nov 06 11:19:35 2018 -0500
summary: revlog: add blank line in comment to help clang-format

https://www.mercurial-scm.org/repo/hg/rev/e5ad3ef90aa1
changeset:   40562:e5ad3ef90aa1
user:Augie Fackler 
date:Tue Nov 06 11:22:16 2018 -0500
summary: revlog: give formatting to clang-format

https://www.mercurial-scm.org/repo/hg/rev/2ad56a9b983b
changeset:   40563:2ad56a9b983b
user:Augie Fackler 
date:Wed Nov 07 12:15:36 2018 -0500
summary: tests: work around `tac` not being portable

https://www.mercurial-scm.org/repo/hg/rev/0df4d93fdc27
changeset:   40564:0df4d93fdc27
user:Martin von Zweigbergk 
date:Wed Nov 07 10:29:38 2018 -0800
summary: tests: replace `tac` reimplementation by `sort -r`

https://www.mercurial-scm.org/repo/hg/rev/71b8ad0ef3e0
changeset:   40565:71b8ad0ef3e0
user:Augie Fackler 
date:Wed Nov 07 14:21:39 2018 -0500
summary: tests: fix up some import statements caught by Python 3

https://www.mercurial-scm.org/repo/hg/rev/b9557567cc3f
changeset:   40566:b9557567cc3f
user:Danny Hooper 
date:Tue Nov 06 15:50:41 2018 -0800
summary: fix: add suboption for configuring execution order of tools

https://www.mercurial-scm.org/repo/hg/rev/8785d66edd6e
changeset:   40567:8785d66edd6e
user:Kyle Lippincott 
date:Wed Nov 07 15:41:18 2018 -0800
summary: tests: fix a couple typos in test-resolve.t comments and add a 
comment

https://www.mercurial-scm.org/repo/hg/rev/b93157f69f46
changeset:   40568:b93157f69f46
user:Kyle Lippincott 
date:Wed Nov 07 15:42:56

Re: [PATCH] tests: fix wireproto redirection test on systems without tls1.2

2018-11-13 Thread Augie Fackler

ignore this version - forgot to flag for stable

> On Nov 13, 2018, at 15:43, Augie Fackler  wrote:
> 
> # HG changeset patch
> # User Augie Fackler 
> # Date 1542141748 18000
> #  Tue Nov 13 15:42:28 2018 -0500
> # Branch stable
> # Node ID 53b7b24c957b0ecbfb8c20dab0c820b07b12ddd0
> # Parent  7e2c58b08e74514541101f5049eea8a1ff4ba4a7
> tests: fix wireproto redirection test on systems without tls1.2
> 
> Our automated package builder has some ancient configuration that
> lacks modern TLS, which is how we noticed this.
> 
> diff --git a/tests/test-wireproto-content-redirects.t 
> b/tests/test-wireproto-content-redirects.t
> --- a/tests/test-wireproto-content-redirects.t
> +++ b/tests/test-wireproto-content-redirects.t
> @@ -30,6 +30,7 @@
>0   0 992f4779029a3df8d0666d00bb924f69634e2641 
>  
> 
>1   1 a988fb43583e871d1ed5750ee074c6d840bbbfc8 
> 992f4779029a3df8d0666d00bb924f69634e2641 
> 
> 
> +#if tls1.2
>   $ hg --config simplecache.redirectsfile=redirects.py serve -p $HGPORT -d 
> --pid-file hg.pid -E error.log
>   $ cat hg.pid > $DAEMON_PIDS
> 
> @@ -357,6 +358,7 @@ Redirect targets advertised when configu
> }
>   ]
>   (sent 2 HTTP requests and * bytes; received * bytes in responses) (glob)
> +#endif
> 
> Unknown protocol is filtered from compatible targets
> 
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 6022] New: histedit curses ui has prefixes in changeset data

2018-11-13 Thread mercurial-bugs

https://bz.mercurial-scm.org/show_bug.cgi?id=6022

Bug ID: 6022
   Summary: histedit curses ui has prefixes in changeset data
   Product: Mercurial
   Version: 4.8
  Hardware: All
OS: Linux
Status: UNCONFIRMED
  Severity: bug
  Priority: normal
 Component: histedit
  Assignee: bugzi...@mercurial-scm.org
  Reporter: gregory.sz...@gmail.com
CC: mercurial-devel@mercurial-scm.org

Using histedit with the curses interface, the changeset data at the bottom of
the window is prefixed with weird characters. The top portion of the UI
(complete with coloring and highlighting) looks fine.

Perhaps my terminal settings are wonky. But I find it weird the top half works
but the bottom doesn't.

Here is a paste of what is rendered on my screen:

?: help, k/up: move up, j/down: move down, space: select, v: view patch
d: drop, e: edit, f: fold, m: mess, p: pick, r: roll
pgup/K: move patch up, pgdn/J: move patch down, c: commit, q: abort
  #0  mess   11723:ce81021c88a5   robustcheckout: record more granular metrics
for overall times; r?sheehan
  #1  pick   11724:f09c578e7d88   robustcheckout: use commandexecutor API for
"lookup" command; r?sheehan
  #2  pick   11725:14867cbdeaca   INCOMPLETE robustcheckout: support partial
checkouts (bug 1505407)


lqk
xchangeset: 11723:ce81021c88a5 
  x
xuser:  gps
  x
xbookmark: 
  x
xfiles:
hgext/robustcheckout/__init__.py,hgext/robustcheckout/tests/test-perfherder.t  
  x
xsummary:   robustcheckout: record more granular metrics for overall times;
r?sheehan x
xno overlap
  x
mqj

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH stable] tests: fix wireproto redirection test on systems without tls1.2

2018-11-13 Thread Augie Fackler

# HG changeset patch
# User Augie Fackler 
# Date 1542141748 18000
#  Tue Nov 13 15:42:28 2018 -0500
# Branch stable
# Node ID 53b7b24c957b0ecbfb8c20dab0c820b07b12ddd0
# Parent  7e2c58b08e74514541101f5049eea8a1ff4ba4a7
tests: fix wireproto redirection test on systems without tls1.2

Our automated package builder has some ancient configuration that
lacks modern TLS, which is how we noticed this.

diff --git a/tests/test-wireproto-content-redirects.t 
b/tests/test-wireproto-content-redirects.t
--- a/tests/test-wireproto-content-redirects.t
+++ b/tests/test-wireproto-content-redirects.t
@@ -30,6 +30,7 @@
0   0 992f4779029a3df8d0666d00bb924f69634e2641 
 

1   1 a988fb43583e871d1ed5750ee074c6d840bbbfc8 
992f4779029a3df8d0666d00bb924f69634e2641 

 
+#if tls1.2
   $ hg --config simplecache.redirectsfile=redirects.py serve -p $HGPORT -d 
--pid-file hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
@@ -357,6 +358,7 @@ Redirect targets advertised when configu
 }
   ]
   (sent 2 HTTP requests and * bytes; received * bytes in responses) (glob)
+#endif
 
 Unknown protocol is filtered from compatible targets
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] tests: fix wireproto redirection test on systems without tls1.2

2018-11-13 Thread Augie Fackler

# HG changeset patch
# User Augie Fackler 
# Date 1542141748 18000
#  Tue Nov 13 15:42:28 2018 -0500
# Branch stable
# Node ID 53b7b24c957b0ecbfb8c20dab0c820b07b12ddd0
# Parent  7e2c58b08e74514541101f5049eea8a1ff4ba4a7
tests: fix wireproto redirection test on systems without tls1.2

Our automated package builder has some ancient configuration that
lacks modern TLS, which is how we noticed this.

diff --git a/tests/test-wireproto-content-redirects.t 
b/tests/test-wireproto-content-redirects.t
--- a/tests/test-wireproto-content-redirects.t
+++ b/tests/test-wireproto-content-redirects.t
@@ -30,6 +30,7 @@
0   0 992f4779029a3df8d0666d00bb924f69634e2641 
 

1   1 a988fb43583e871d1ed5750ee074c6d840bbbfc8 
992f4779029a3df8d0666d00bb924f69634e2641 

 
+#if tls1.2
   $ hg --config simplecache.redirectsfile=redirects.py serve -p $HGPORT -d 
--pid-file hg.pid -E error.log
   $ cat hg.pid > $DAEMON_PIDS
 
@@ -357,6 +358,7 @@ Redirect targets advertised when configu
 }
   ]
   (sent 2 HTTP requests and * bytes; received * bytes in responses) (glob)
+#endif
 
 Unknown protocol is filtered from compatible targets
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5267: revlog: automatically read from opened file handles

2018-11-13 Thread indygreg (Gregory Szorc)

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The revlog reading code commonly opens a new file handle for
  reading on demand. There is support for passing a file handle
  to revlog.revision(). But it is marked as an internal argument.
  
  When revlogs are written, we write() data as it is available. But
  we don't flush() data until all revisions are written.
  
  Putting these two traits together, it is possible for an in-process
  revlog reader during active writes to trigger the opening of a new
  file handle on a file with unflushed writes. The reader won't have
  access to all "available" revlog data (as it hasn't been flushed).
  And with the introduction of the previous patch, this can lead to
  the revlog raising an error due to a partial read.
  
  I witnessed this behavior when applying changegroup data (via
  `hg pull`) before issue6006 was fixed via different means. Having
  this and the previous patch in play would have helped cause errors
  earlier rather than manifesting as hash verification failures.
  
  While this has been a long-standing issue, I believe the relatively
  new delta computation code has tickled it into being more common.
  This is because the new delta computation code will compute deltas
  in more scenarios. This can lead to revlog reading. While the delta
  computation code is probably supposed to reuse file handles, it
  appears it isn't doing so in all circumstances.
  
  But the issue runs deeper than that. Theoretically, any code can
  access revision data during revlog writes. It appears we were just
  getting lucky that it wasn't. (The "add revision callback" passed to
  addgroup() provides an avenue to do this.)
  
  If I changed the revlog's behavior to not cache the full revision
  text or to clear caches after revision insertion during addgroup(),
  I was able to produce crashes 100% of the time when writing changelog
  revisions. This is because changelog's add revision callback attempts
  to resolve the revision data to access the changed files list. And
  without the revision's fulltext being cached, we performed a revlog
  read, which required opening a new file handle. This attempted to read
  unflushed data, leading to a partial read and a crash.
  
  This commit teaches the revlog to store the file handles used for
  writing multiple revisions during addgroup(). It also teaches the
  code for resolving a file handle when reading to use these handles,
  if available. This ensures that *any* reads (regardless of their
  source) use the active writing file handles, if available. These
  file handles have access to the unflushed data because they wrote it.
  This allows reads to complete without issue.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5267

AFFECTED FILES
  mercurial/revlog.py

CHANGE DETAILS

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -375,6 +375,9 @@
 # custom flags.
 self._flagprocessors = dict(_flagprocessors)
 
+# 2-tuple of file handles being used for active writing.
+self._writinghandles = None
+
 mmapindexthreshold = None
 v = REVLOG_DEFAULT_VERSION
 opts = getattr(opener, 'options', None)
@@ -505,8 +508,21 @@
 @contextlib.contextmanager
 def _datareadfp(self, existingfp=None):
 """file object suitable to read data"""
+# Use explicit file handle, if given.
 if existingfp is not None:
 yield existingfp
+
+# Use a file handle being actively used for writes, if available.
+# There is some danger to doing this because reads will seek the
+# file. However, _writeentry() performs a SEEK_END before all writes,
+# so we should be safe.
+elif self._writinghandles:
+if self._inline:
+yield self._writinghandles[0]
+else:
+yield self._writinghandles[1]
+
+# Otherwise open a new file handle.
 else:
 if self._inline:
 func = self._indexfp
@@ -1750,6 +1766,9 @@
 if fp:
 fp.flush()
 fp.close()
+# We can't use the cached file handle after close(). So prevent
+# its usage.
+self._writinghandles = None
 
 with self._indexfp('r') as ifh, self._datafp('w') as dfh:
 for r in self:
@@ -1996,7 +2015,9 @@
 # if the file was seeked to before the end. See issue4943 for more.
 #
 # We work around this issue by inserting a seek() before writing.
-# Note: This is likely not necessary on Python 3.
+# Note: This is likely not necessary on Python 3. However, because
+# the file handle is reused for reads and may be seeked there, we need
+# to be careful before changing this.
 ifh.seek(0,

D5265: revlog: use single file handle when de-inlining revlog

2018-11-13 Thread indygreg (Gregory Szorc)

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  _getsegmentforrevs() will eventually call into _datareadfp() to
  resolve a file handle to read revision data. If no file handle
  is passed into _getsegmentforrevs(), it opens a new one.
  
  Explicit is better than implicit.
  
  This commit changes _enforceinlinesize() to open a file handle
  explicitly when converting inline revlogs to split revlogs and
  to pass this file handle into _getsegmentforrevs().
  
  I haven't measured, but this change should improve performance,
  as we no longer reopen the revlog for reading for every revision
  in the revlog when it is converted from inline to split. Instead,
  we open it at most once and use it for the duration of the
  operation. That being said, I /think/ the chunk cache may mitigate
  the number of file opens required.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5265

AFFECTED FILES
  mercurial/revlog.py

CHANGE DETAILS

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1732,9 +1732,9 @@
 fp.flush()
 fp.close()
 
-with self._datafp('w') as df:
+with self._indexfp('r') as ifh, self._datafp('w') as dfh:
 for r in self:
-df.write(self._getsegmentforrevs(r, r)[1])
+dfh.write(self._getsegmentforrevs(r, r, df=ifh)[1])
 
 with self._indexfp('w') as fp:
 self.version &= ~FLAG_INLINE_DATA



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5266: revlog: detect incomplete revlog reads

2018-11-13 Thread indygreg (Gregory Szorc)

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  _readsegment() is supposed to return N bytes of revlog revision
  data starting at a file offset. Surprisingly, its behavior before
  this patch never verified that it actually read and returned N
  bytes! Instead, it would perform the read(), then return whatever
  data was available. And even more surprisingly, nothing in the
  call chain appears to have been validating that it received all
  the data it was expecting.
  
  This behavior could lead to partial or incomplete revision chunks
  being operated on. This could result in e.g. cached deltas being
  applied against incomplete base revisions. The delta application
  process would happily perform this operation. Only hash
  verification would detect the corruption and save us.
  
  This commit changes the behavior of raw revlog reading to validate
  that we actually read() the number of bytes that were requested.
  We will raise a more specific error faster, rather than possibly
  have it go undetected or manifest later in the call stack, at
  delta application or hash verification.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5266

AFFECTED FILES
  mercurial/revlog.py

CHANGE DETAILS

diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -1342,6 +1342,8 @@
 original seek position will NOT be restored.
 
 Returns a str or buffer of raw byte data.
+
+Raises if the requested number of bytes could not be read.
 """
 # Cache data both forward and backward around the requested
 # data, in a fixed size window. This helps speed up operations
@@ -1353,9 +1355,26 @@
 with self._datareadfp(df) as df:
 df.seek(realoffset)
 d = df.read(reallength)
+
 self._cachesegment(realoffset, d)
 if offset != realoffset or reallength != length:
-return util.buffer(d, offset - realoffset, length)
+startoffset = offset - realoffset
+if len(d) - startoffset < length:
+raise error.RevlogError(
+_('partial read of revlog %s; expected %d bytes from '
+  'offset %d, got %d') %
+(self.indexfile if self._inline else self.datafile,
+ length, realoffset, len(d) - startoffset))
+
+return util.buffer(d, startoffset, length)
+
+if len(d) < length:
+raise error.RevlogError(
+_('partial read of revlog %s; expected %d bytes from offset '
+  '%d, got %d') %
+(self.indexfile if self._inline else self.datafile,
+ length, offset, len(d)))
+
 return d
 
 def _getsegment(self, offset, length, df=None):



To: indygreg, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5263: py3: add b suffix to make sure file is opened in bytes mode

2018-11-13 Thread pulkit (Pulkit Goyal)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGaa588bf40a08: py3: add b suffix to make sure file is opened 
in bytes mode (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5263?vs=12520=12526

REVISION DETAIL
  https://phab.mercurial-scm.org/D5263

AFFECTED FILES
  hgext/remotefilelog/basepack.py
  hgext/remotefilelog/basestore.py
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/fileserverclient.py
  hgext/remotefilelog/remotefilelogserver.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/remotefilelogserver.py 
b/hgext/remotefilelog/remotefilelogserver.py
--- a/hgext/remotefilelog/remotefilelogserver.py
+++ b/hgext/remotefilelog/remotefilelogserver.py
@@ -234,7 +234,7 @@
 
 f = None
 try:
-f = util.atomictempfile(filecachepath, "w")
+f = util.atomictempfile(filecachepath, "wb")
 f.write(text)
 except (IOError, OSError):
 # Don't abort if the user only has permission to read,
@@ -246,7 +246,7 @@
 finally:
 os.umask(oldumask)
 else:
-with open(filecachepath, "r") as f:
+with open(filecachepath, "rb") as f:
 text = f.read()
 return text
 
diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -482,7 +482,7 @@
 
 def close(self):
 if fetches:
-msg = ("%s files fetched over %d fetches - " +
+msg = ("%d files fetched over %d fetches - " +
"(%d misses, %0.2f%% hit ratio) over %0.2fs\n") % (
fetched,
fetches,
diff --git a/hgext/remotefilelog/debugcommands.py 
b/hgext/remotefilelog/debugcommands.py
--- a/hgext/remotefilelog/debugcommands.py
+++ b/hgext/remotefilelog/debugcommands.py
@@ -176,7 +176,7 @@
 
 def parsefileblob(path, decompress):
 raw = None
-f = open(path, "r")
+f = open(path, "rb")
 try:
 raw = f.read()
 finally:
diff --git a/hgext/remotefilelog/basestore.py b/hgext/remotefilelog/basestore.py
--- a/hgext/remotefilelog/basestore.py
+++ b/hgext/remotefilelog/basestore.py
@@ -255,7 +255,7 @@
 they want to be kept alive in the store.
 """
 repospath = os.path.join(self._path, "repos")
-with open(repospath, 'a') as reposfile:
+with open(repospath, 'ab') as reposfile:
 reposfile.write(os.path.dirname(path) + "\n")
 
 repospathstat = os.stat(repospath)
@@ -270,7 +270,7 @@
 return True
 
 if self._validatecachelog:
-with open(self._validatecachelog, 'a+') as f:
+with open(self._validatecachelog, 'ab+') as f:
 f.write("corrupt %s during %s\n" % (path, action))
 
 os.rename(path, path + ".corrupt")
diff --git a/hgext/remotefilelog/basepack.py b/hgext/remotefilelog/basepack.py
--- a/hgext/remotefilelog/basepack.py
+++ b/hgext/remotefilelog/basepack.py
@@ -373,8 +373,8 @@
 suffix=self.PACKSUFFIX + '-tmp')
 self.idxfp, self.idxpath = opener.mkstemp(
 suffix=self.INDEXSUFFIX + '-tmp')
-self.packfp = os.fdopen(self.packfp, r'w+')
-self.idxfp = os.fdopen(self.idxfp, r'w+')
+self.packfp = os.fdopen(self.packfp, r'wb+')
+self.idxfp = os.fdopen(self.idxfp, r'wb+')
 self.sha = hashlib.sha1()
 self._closed = False
 



To: pulkit, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5262: py3: don't use dict.iterkeys()

2018-11-13 Thread pulkit (Pulkit Goyal)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG9769e0f6ffe0: py3: dont use dict.iterkeys() (authored 
by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5262?vs=12519=12525

REVISION DETAIL
  https://phab.mercurial-scm.org/D5262

AFFECTED FILES
  hgext/remotefilelog/basepack.py
  hgext/remotefilelog/fileserverclient.py
  hgext/remotefilelog/remotefilelog.py
  hgext/remotefilelog/repack.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/repack.py b/hgext/remotefilelog/repack.py
--- a/hgext/remotefilelog/repack.py
+++ b/hgext/remotefilelog/repack.py
@@ -507,7 +507,7 @@
 total=len(byfile))
 
 ancestors = {}
-nodes = list(node for node in entries.iterkeys())
+nodes = list(node for node in entries)
 nohistory = []
 for i, node in enumerate(nodes):
 if node in ancestors:
@@ -643,7 +643,7 @@
 count = 0
 for filename, entries in sorted(byfile.iteritems()):
 ancestors = {}
-nodes = list(node for node in entries.iterkeys())
+nodes = list(node for node in entries)
 
 for node in nodes:
 if node in ancestors:
diff --git a/hgext/remotefilelog/remotefilelog.py 
b/hgext/remotefilelog/remotefilelog.py
--- a/hgext/remotefilelog/remotefilelog.py
+++ b/hgext/remotefilelog/remotefilelog.py
@@ -419,7 +419,7 @@
 # Breadth first traversal to build linkrev graph
 parentrevs = collections.defaultdict(list)
 revmap = {}
-queue = collections.deque(((None, n) for n in parentsmap.iterkeys()
+queue = collections.deque(((None, n) for n in parentsmap
  if n not in allparents))
 while queue:
 prevrev, current = queue.pop()
diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -340,7 +340,7 @@
 missingid = cache.receiveline()
 if not missingid:
 missedset = set(missed)
-for missingid in idmap.iterkeys():
+for missingid in idmap:
 if not missingid in missedset:
 missed.append(missingid)
 self.ui.warn(_("warning: cache connection closed early - " +
diff --git a/hgext/remotefilelog/basepack.py b/hgext/remotefilelog/basepack.py
--- a/hgext/remotefilelog/basepack.py
+++ b/hgext/remotefilelog/basepack.py
@@ -468,7 +468,7 @@
 # Precompute the location of each entry
 locations = {}
 count = 0
-for node in sorted(self.entries.iterkeys()):
+for node in sorted(self.entries):
 location = count * self.INDEXENTRYLENGTH
 locations[node] = location
 count += 1



To: pulkit, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5259: py3: fix keyword arguments handling in hgext/remotefilelog/

2018-11-13 Thread pulkit (Pulkit Goyal)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG13d4ad8d7801: py3: fix keyword arguments handling in 
hgext/remotefilelog/ (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5259?vs=12516=12522

REVISION DETAIL
  https://phab.mercurial-scm.org/D5259

AFFECTED FILES
  hgext/remotefilelog/__init__.py
  hgext/remotefilelog/basestore.py
  hgext/remotefilelog/contentstore.py
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/fileserverclient.py
  hgext/remotefilelog/metadatastore.py
  hgext/remotefilelog/remotefilectx.py
  hgext/remotefilelog/shallowbundle.py
  hgext/remotefilelog/shallowutil.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/shallowutil.py 
b/hgext/remotefilelog/shallowutil.py
--- a/hgext/remotefilelog/shallowutil.py
+++ b/hgext/remotefilelog/shallowutil.py
@@ -105,7 +105,7 @@
 def reportpackmetrics(ui, prefix, *stores):
 dicts = [s.getmetrics() for s in stores]
 dict = prefixkeys(sumdicts(*dicts), prefix + '_')
-ui.log(prefix + "_packsizes", "", **dict)
+ui.log(prefix + "_packsizes", "", **pycompat.strkwargs(dict))
 
 def _parsepackmeta(metabuf):
 """parse datapack meta, bytes () -> dict
diff --git a/hgext/remotefilelog/shallowbundle.py 
b/hgext/remotefilelog/shallowbundle.py
--- a/hgext/remotefilelog/shallowbundle.py
+++ b/hgext/remotefilelog/shallowbundle.py
@@ -146,7 +146,7 @@
 try:
 # if serving, only send files the clients has patterns for
 if source == 'serve':
-bundlecaps = kwargs.get('bundlecaps')
+bundlecaps = kwargs.get(r'bundlecaps')
 includepattern = None
 excludepattern = None
 for cap in (bundlecaps or []):
diff --git a/hgext/remotefilelog/remotefilectx.py 
b/hgext/remotefilelog/remotefilectx.py
--- a/hgext/remotefilelog/remotefilectx.py
+++ b/hgext/remotefilelog/remotefilectx.py
@@ -15,6 +15,7 @@
 context,
 error,
 phases,
+pycompat,
 util,
 )
 from . import shallowutil
@@ -218,11 +219,11 @@
 return linknode
 
 commonlogkwargs = {
-'revs': ' '.join([hex(cl.node(rev)) for rev in revs]),
-'fnode': hex(fnode),
-'filepath': path,
-'user': shallowutil.getusername(repo.ui),
-'reponame': shallowutil.getreponame(repo.ui),
+r'revs': ' '.join([hex(cl.node(rev)) for rev in revs]),
+r'fnode': hex(fnode),
+r'filepath': path,
+r'user': shallowutil.getusername(repo.ui),
+r'reponame': shallowutil.getreponame(repo.ui),
 }
 
 repo.ui.log('linkrevfixup', 'adjusting linknode', **commonlogkwargs)
@@ -315,7 +316,7 @@
 finally:
 elapsed = time.time() - start
 repo.ui.log('linkrevfixup', logmsg, elapsed=elapsed * 1000,
-**commonlogkwargs)
+**pycompat.strkwargs(commonlogkwargs))
 
 def _verifylinknode(self, revs, linknode):
 """
@@ -408,7 +409,7 @@
 
 def annotate(self, *args, **kwargs):
 introctx = self
-prefetchskip = kwargs.pop('prefetchskip', None)
+prefetchskip = kwargs.pop(r'prefetchskip', None)
 if prefetchskip:
 # use introrev so prefetchskip can be accurately tested
 introrev = self.introrev()
diff --git a/hgext/remotefilelog/metadatastore.py 
b/hgext/remotefilelog/metadatastore.py
--- a/hgext/remotefilelog/metadatastore.py
+++ b/hgext/remotefilelog/metadatastore.py
@@ -11,12 +11,12 @@
 super(unionmetadatastore, self).__init__(*args, **kwargs)
 
 self.stores = args
-self.writestore = kwargs.get('writestore')
+self.writestore = kwargs.get(r'writestore')
 
 # If allowincomplete==True then the union store can return partial
 # ancestor lists, otherwise it will throw a KeyError if a full
 # history can't be found.
-self.allowincomplete = kwargs.get('allowincomplete', False)
+self.allowincomplete = kwargs.get(r'allowincomplete', False)
 
 def getancestors(self, name, node, known=None):
 """Returns as many ancestors as we're aware of.
diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -18,6 +18,7 @@
 from mercurial.node import bin, hex, nullid
 from mercurial import (
 error,
+pycompat,
 revlog,
 sshpeer,
 util,
@@ -119,7 +120,7 @@
 def _callstream(self, command, **opts):
 supertype = super(remotefilepeer, self)
 if not util.safehasattr(supertype, '_sendrequest'):
-self._updatecallstreamopts(command, opts)
+self._updatecallstreamopts(command, pycompat.byteskwargs(opts))
 return super(remotefilepeer, self)._callstream(command, **opts)

D5261: py3: use node.hex(h.digest()) instead of h.hexdigest()

2018-11-13 Thread pulkit (Pulkit Goyal)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG3fa4183e7803: py3: use node.hex(h.digest()) instead of 
h.hexdigest() (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5261?vs=12518=12524

REVISION DETAIL
  https://phab.mercurial-scm.org/D5261

AFFECTED FILES
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/fileserverclient.py
  hgext/remotefilelog/shallowutil.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/shallowutil.py 
b/hgext/remotefilelog/shallowutil.py
--- a/hgext/remotefilelog/shallowutil.py
+++ b/hgext/remotefilelog/shallowutil.py
@@ -17,6 +17,7 @@
 from mercurial.i18n import _
 from mercurial import (
 error,
+node,
 pycompat,
 revlog,
 util,
@@ -35,11 +36,11 @@
 return constants.SHALLOWREPO_REQUIREMENT in repo.requirements
 
 def getcachekey(reponame, file, id):
-pathhash = hashlib.sha1(file).hexdigest()
+pathhash = node.hex(hashlib.sha1(file).digest())
 return os.path.join(reponame, pathhash[:2], pathhash[2:], id)
 
 def getlocalkey(file, id):
-pathhash = hashlib.sha1(file).hexdigest()
+pathhash = node.hex(hashlib.sha1(file).digest())
 return os.path.join(pathhash, id)
 
 def getcachepath(ui, allowempty=False):
diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -18,6 +18,7 @@
 from mercurial.node import bin, hex, nullid
 from mercurial import (
 error,
+node,
 pycompat,
 revlog,
 sshpeer,
@@ -44,11 +45,11 @@
 _downloading = _('downloading')
 
 def getcachekey(reponame, file, id):
-pathhash = hashlib.sha1(file).hexdigest()
+pathhash = node.hex(hashlib.sha1(file).digest())
 return os.path.join(reponame, pathhash[:2], pathhash[2:], id)
 
 def getlocalkey(file, id):
-pathhash = hashlib.sha1(file).hexdigest()
+pathhash = node.hex(hashlib.sha1(file).digest())
 return os.path.join(pathhash, id)
 
 def peersetup(ui, peer):
diff --git a/hgext/remotefilelog/debugcommands.py 
b/hgext/remotefilelog/debugcommands.py
--- a/hgext/remotefilelog/debugcommands.py
+++ b/hgext/remotefilelog/debugcommands.py
@@ -15,6 +15,7 @@
 from mercurial import (
 error,
 filelog,
+node as nodemod,
 revlog,
 )
 from . import (
@@ -52,7 +53,7 @@
 
 def buildtemprevlog(repo, file):
 # get filename key
-filekey = hashlib.sha1(file).hexdigest()
+filekey = nodemod.hex(hashlib.sha1(file).digest())
 filedir = os.path.join(repo.path, 'store/data', filekey)
 
 # sort all entries based on linkrev
@@ -344,7 +345,7 @@
 ui.write("%s  %s  %s  %s\n" % (
 hashformatter(node),
 hashformatter(deltabasenode),
-hashlib.sha1(delta).hexdigest(),
+nodemod.hex(hashlib.sha1(delta).digest()),
 len(delta)))
 
 def debughistorypack(ui, path):



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5260: py3: make second argument of fdopen() a str

2018-11-13 Thread pulkit (Pulkit Goyal)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG5e3b3f88c2e4: py3: make second argument of fdopen() a str 
(authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5260?vs=12517=12523

REVISION DETAIL
  https://phab.mercurial-scm.org/D5260

AFFECTED FILES
  hgext/remotefilelog/basepack.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/basepack.py b/hgext/remotefilelog/basepack.py
--- a/hgext/remotefilelog/basepack.py
+++ b/hgext/remotefilelog/basepack.py
@@ -373,8 +373,8 @@
 suffix=self.PACKSUFFIX + '-tmp')
 self.idxfp, self.idxpath = opener.mkstemp(
 suffix=self.INDEXSUFFIX + '-tmp')
-self.packfp = os.fdopen(self.packfp, 'w+')
-self.idxfp = os.fdopen(self.idxfp, 'w+')
+self.packfp = os.fdopen(self.packfp, r'w+')
+self.idxfp = os.fdopen(self.idxfp, r'w+')
 self.sha = hashlib.sha1()
 self._closed = False
 



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5263: py3: add b suffix to make sure file is opened in bytes mode

2018-11-13 Thread indygreg (Gregory Szorc)

indygreg added a comment.


  Ideally we'd be using the vfs layer for I/O. But that is scope bloat and I'm 
not going to make you rewrite RFL to do things more correctly.

INLINE COMMENTS

> fileserverclient.py:485
>  if fetches:
> -msg = ("%s files fetched over %d fetches - " +
> +msg = ("%d files fetched over %d fetches - " +
> "(%d misses, %0.2f%% hit ratio) over %0.2fs\n") % (

This change is unrelated. But meh.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5263

To: pulkit, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5262: py3: don't use dict.iterkeys()

2018-11-13 Thread indygreg (Gregory Szorc)

indygreg added inline comments.

INLINE COMMENTS

> repack.py:510
>  ancestors = {}
> -nodes = list(node for node in entries.iterkeys())
> +nodes = list(node for node in entries)
>  nohistory = []

This can be reduced to `list(entries)`.

> repack.py:646
>  ancestors = {}
> -nodes = list(node for node in entries.iterkeys())
> +nodes = list(node for node in entries)
>  

Ditto.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5262

To: pulkit, #hg-reviewers
Cc: indygreg, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Using internal phase for history rewriting

2018-11-13 Thread Gregory Szorc

At the 4.8 sprint, we decided that we wanted to use "internal" phases for
hiding rewritten changesets. From the notes:

* We will use internal phase to hide rewritten changesets
 * hg pull / hg unbundle receiving an internal phase changeset will bump
the phase and unhide the changeset
 * We will continue to write backup bundles. So restoring hidden changesets
will mean applying backup bundle - same as today
 * There are some complications related to interaction of internal phase
and obsmarkers. To be figured out later.
 * We will enable internal phase by default in 4.9 cycle hopefully
 * Need to teach `hg debugupgraderepo` to enable internal phases on
existing repos

While not captured in the notes, my recollection is we would get this
enabled by default as early as possible in 4.9 and make a determination
closer to release time if it were stable enough to ship.

Has any work towards this feature been made? Is there anything I can do to
help this feature move forward?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5264: store: raise ProgrammingError if unable to decode a storage path

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Right now, the function magically return False which is dangerous, so let's
  raise ProgrammingError.
  
  Suggested by Augie in https://phab.mercurial-scm.org/D5139.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5264

AFFECTED FILES
  mercurial/store.py

CHANGE DETAILS

diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -38,6 +38,8 @@
 elif path.startswith('meta/'):
 return matcher.visitdir(path[len('meta/'):-len('/00manifest.i')] or 
'.')
 
+raise error.ProgrammingError("cannot decode path %s" % path)
+
 # This avoids a collision between a file named foo and a dir named
 # foo.i or foo.d
 def _encodedir(path):



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 12 of 14] sparse-revlog: introduce native (C) implementation of slicechunktodensity

2018-11-13 Thread Boris FELD


On 12/11/2018 15:50, Yuya Nishihara wrote:
> On Mon, 12 Nov 2018 10:55:47 +0100, Boris Feld wrote:
>> # HG changeset patch
>> # User Boris Feld 
>> # Date 1541785632 -3600
>> #  Fri Nov 09 18:47:12 2018 +0100
>> # Node ID 4a1104eade1dfb1697517d60d2c5fd7a98b8c7f0
>> # Parent  0ea42453fa491793d1e145f5093b65e84cb65e97
>> # EXP-Topic sparse-perf
>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
>> 4a1104eade1d
>> sparse-revlog: introduce native (C) implementation of slicechunktodensity
> Just quickly scanned this patch, no other patches in this series nor "sparse"
> logic is reviewed.
>
>> +struct Gap {
>> +long size;
>> +long idx;
>> +};
>> +
>> +static int gap_compare(const void *left, const void *right)
>> +{
>> +return ((struct Gap *)left)->size - ((struct Gap *)right)->size;
>> +}
>> +static int long_compare(const void *left, const void *right)
>> +{
>> +return *(long *)left - *(long *)right;
>> +}
> Nit: (const  *) as the argument is const void*.
>
> If we're sure 'left - right' never exceeds the int range, it might be better
> to explicitly cast the result to (int) to silence possible warning.
>
>> +static PyObject *index_slicechunktodensity(indexObject *self, PyObject 
>> *args)
>> +{
>> +/* method arguments */
>> +PyObject *list_revs = NULL; /* revisions in the chain */
>> +double targetdensity = 0.5; /* min density to achieve */
>> +long mingapsize = 0;/* threshold to ignore gaps */
>> +
>> +/* other core variables */
>> +long i;  /* used for various iteration */
>> +PyObject *result = NULL; /* the final return of the function */
>> +
>> +/* generic information about the delta chain being slice */
>> +Py_ssize_t num_revs = 0; /* size of the full delta chain */
>> +Py_ssize_t *revs = NULL; /* native array of revision in the chain */
>> +long chainpayload = 0;   /* sum of all delta in the chain */
>> +long deltachainspan = 0; /* distance from first byte to last byte */
>> +
>> +/* variable used for slicing the delta chain */
>> +long readdata = 0;  /* amount of data currently planned to be read */
>> +double density = 0; /* ration of payload data compared to read ones */
>> +struct Gap *gaps = NULL; /* array of notable gap in the chain */
>> +long num_gaps = 0; /* total number of notable gap recorded so far */
>> +Py_ssize_t *selected_indices = NULL; /* indices of gap skipped over */
>> +long num_selected = 0;   /* number of gaps skipped */
> Maybe i, num_rgaps, num_selected should be Py_ssize_t? In general, long can
> be narrower than ssize_t.
>
>> +PyObject *chunk = NULL;  /* individual slice */
>> +PyObject *allchunks = PyList_New(num_selected); /* all slices */
> Needs to make sure that allchunks isn't NULL.
>
> And it's probably better to just initialize the list with literal 0, since
> we have no for-loop to fill in the list items up to num_selected.
>
>> +/* parsing argument */
>> +if (!PyArg_ParseTuple(args, "O!dl", _Type, _revs,
>> +  , )) {
>> +goto bail;
>> +}
>> +
>> +/* If the delta chain contains a single element, we do not need slicing
>> + */
>> +num_revs = PyList_GET_SIZE(list_revs);
>> +if (num_revs <= 1) {
>> +result = PyTuple_Pack(1, list_revs);
>> +goto done;
>> +}
>> +
>> +/* Turn the python list into a native integer array (for efficiency) */
>> +revs = (Py_ssize_t *)malloc((num_revs) * sizeof(Py_ssize_t));
> num_revs * sizeof(...) can overflow. Using calloc() is safer if the zeroing
> cost isn't significant.
>
>> +if (revs == NULL) {
>> +PyErr_NoMemory();
>> +goto bail;
>> +}
>> +for (i = 0; i < num_revs; i++) {
>> +Py_ssize_t revnum = PyInt_AsLong(PyList_GET_ITEM(list_revs, i));
>> +if (revnum == -1 && PyErr_Occurred()) {
>> +goto bail;
>> +}
>> +revs[i] = revnum;
>> +}
> Are we sure revnum is in valid range?
What do you mean ?
>
>> +
>> +/* Compute and check various property of the unsliced delta chain */
>> +deltachainspan = index_segment_span(self, revs[0], revs[num_revs - 1]);
>> +
>> +if (deltachainspan <= mingapsize) {
>> +result = PyTuple_Pack(1, list_revs);
>> +goto done;
>> +}
>> +chainpayload = 0;
>> +for (i = 0; i < num_revs; i++) {
>> +chainpayload += index_get_length(self, revs[i]);
>> +}
>> +
>> +readdata = deltachainspan;
>> +density = 1.0;
>> +
>> +if (0 < deltachainspan) {
>> +density = (double)chainpayload / (double)deltachainspan;
>> +};
>> +
>> +if (density >= targetdensity) {
>> +result = PyTuple_Pack(1, list_revs);
>> +goto done;
>> +}
>> +
>> +/* if chain is too sparse, look for

D5263: py3: add b suffix to make sure file is opened in bytes mode

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5263

AFFECTED FILES
  hgext/remotefilelog/basepack.py
  hgext/remotefilelog/basestore.py
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/fileserverclient.py
  hgext/remotefilelog/remotefilelogserver.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/remotefilelogserver.py 
b/hgext/remotefilelog/remotefilelogserver.py
--- a/hgext/remotefilelog/remotefilelogserver.py
+++ b/hgext/remotefilelog/remotefilelogserver.py
@@ -234,7 +234,7 @@
 
 f = None
 try:
-f = util.atomictempfile(filecachepath, "w")
+f = util.atomictempfile(filecachepath, "wb")
 f.write(text)
 except (IOError, OSError):
 # Don't abort if the user only has permission to read,
@@ -246,7 +246,7 @@
 finally:
 os.umask(oldumask)
 else:
-with open(filecachepath, "r") as f:
+with open(filecachepath, "rb") as f:
 text = f.read()
 return text
 
diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -482,7 +482,7 @@
 
 def close(self):
 if fetches:
-msg = ("%s files fetched over %d fetches - " +
+msg = ("%d files fetched over %d fetches - " +
"(%d misses, %0.2f%% hit ratio) over %0.2fs\n") % (
fetched,
fetches,
diff --git a/hgext/remotefilelog/debugcommands.py 
b/hgext/remotefilelog/debugcommands.py
--- a/hgext/remotefilelog/debugcommands.py
+++ b/hgext/remotefilelog/debugcommands.py
@@ -176,7 +176,7 @@
 
 def parsefileblob(path, decompress):
 raw = None
-f = open(path, "r")
+f = open(path, "rb")
 try:
 raw = f.read()
 finally:
diff --git a/hgext/remotefilelog/basestore.py b/hgext/remotefilelog/basestore.py
--- a/hgext/remotefilelog/basestore.py
+++ b/hgext/remotefilelog/basestore.py
@@ -255,7 +255,7 @@
 they want to be kept alive in the store.
 """
 repospath = os.path.join(self._path, "repos")
-with open(repospath, 'a') as reposfile:
+with open(repospath, 'ab') as reposfile:
 reposfile.write(os.path.dirname(path) + "\n")
 
 repospathstat = os.stat(repospath)
@@ -270,7 +270,7 @@
 return True
 
 if self._validatecachelog:
-with open(self._validatecachelog, 'a+') as f:
+with open(self._validatecachelog, 'ab+') as f:
 f.write("corrupt %s during %s\n" % (path, action))
 
 os.rename(path, path + ".corrupt")
diff --git a/hgext/remotefilelog/basepack.py b/hgext/remotefilelog/basepack.py
--- a/hgext/remotefilelog/basepack.py
+++ b/hgext/remotefilelog/basepack.py
@@ -373,8 +373,8 @@
 suffix=self.PACKSUFFIX + '-tmp')
 self.idxfp, self.idxpath = opener.mkstemp(
 suffix=self.INDEXSUFFIX + '-tmp')
-self.packfp = os.fdopen(self.packfp, r'w+')
-self.idxfp = os.fdopen(self.idxfp, r'w+')
+self.packfp = os.fdopen(self.packfp, r'wb+')
+self.idxfp = os.fdopen(self.idxfp, r'wb+')
 self.sha = hashlib.sha1()
 self._closed = False
 



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5262: py3: don't use dict.iterkeys()

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  dict.iterkeys() is not present on Python 3.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5262

AFFECTED FILES
  hgext/remotefilelog/basepack.py
  hgext/remotefilelog/fileserverclient.py
  hgext/remotefilelog/remotefilelog.py
  hgext/remotefilelog/repack.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/repack.py b/hgext/remotefilelog/repack.py
--- a/hgext/remotefilelog/repack.py
+++ b/hgext/remotefilelog/repack.py
@@ -507,7 +507,7 @@
 total=len(byfile))
 
 ancestors = {}
-nodes = list(node for node in entries.iterkeys())
+nodes = list(node for node in entries)
 nohistory = []
 for i, node in enumerate(nodes):
 if node in ancestors:
@@ -643,7 +643,7 @@
 count = 0
 for filename, entries in sorted(byfile.iteritems()):
 ancestors = {}
-nodes = list(node for node in entries.iterkeys())
+nodes = list(node for node in entries)
 
 for node in nodes:
 if node in ancestors:
diff --git a/hgext/remotefilelog/remotefilelog.py 
b/hgext/remotefilelog/remotefilelog.py
--- a/hgext/remotefilelog/remotefilelog.py
+++ b/hgext/remotefilelog/remotefilelog.py
@@ -419,7 +419,7 @@
 # Breadth first traversal to build linkrev graph
 parentrevs = collections.defaultdict(list)
 revmap = {}
-queue = collections.deque(((None, n) for n in parentsmap.iterkeys()
+queue = collections.deque(((None, n) for n in parentsmap
  if n not in allparents))
 while queue:
 prevrev, current = queue.pop()
diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -340,7 +340,7 @@
 missingid = cache.receiveline()
 if not missingid:
 missedset = set(missed)
-for missingid in idmap.iterkeys():
+for missingid in idmap:
 if not missingid in missedset:
 missed.append(missingid)
 self.ui.warn(_("warning: cache connection closed early - " +
diff --git a/hgext/remotefilelog/basepack.py b/hgext/remotefilelog/basepack.py
--- a/hgext/remotefilelog/basepack.py
+++ b/hgext/remotefilelog/basepack.py
@@ -468,7 +468,7 @@
 # Precompute the location of each entry
 locations = {}
 count = 0
-for node in sorted(self.entries.iterkeys()):
+for node in sorted(self.entries):
 location = count * self.INDEXENTRYLENGTH
 locations[node] = location
 count += 1



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5261: py3: use node.hex(h.digest()) instead of h.hexdigest()

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  hashlib.sha1.hexdigest() returns str on Python 3.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5261

AFFECTED FILES
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/fileserverclient.py
  hgext/remotefilelog/shallowutil.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/shallowutil.py 
b/hgext/remotefilelog/shallowutil.py
--- a/hgext/remotefilelog/shallowutil.py
+++ b/hgext/remotefilelog/shallowutil.py
@@ -17,6 +17,7 @@
 from mercurial.i18n import _
 from mercurial import (
 error,
+node,
 pycompat,
 revlog,
 util,
@@ -35,11 +36,11 @@
 return constants.SHALLOWREPO_REQUIREMENT in repo.requirements
 
 def getcachekey(reponame, file, id):
-pathhash = hashlib.sha1(file).hexdigest()
+pathhash = node.hex(hashlib.sha1(file).digest())
 return os.path.join(reponame, pathhash[:2], pathhash[2:], id)
 
 def getlocalkey(file, id):
-pathhash = hashlib.sha1(file).hexdigest()
+pathhash = node.hex(hashlib.sha1(file).digest())
 return os.path.join(pathhash, id)
 
 def getcachepath(ui, allowempty=False):
diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -18,6 +18,7 @@
 from mercurial.node import bin, hex, nullid
 from mercurial import (
 error,
+node,
 pycompat,
 revlog,
 sshpeer,
@@ -44,11 +45,11 @@
 _downloading = _('downloading')
 
 def getcachekey(reponame, file, id):
-pathhash = hashlib.sha1(file).hexdigest()
+pathhash = node.hex(hashlib.sha1(file).digest())
 return os.path.join(reponame, pathhash[:2], pathhash[2:], id)
 
 def getlocalkey(file, id):
-pathhash = hashlib.sha1(file).hexdigest()
+pathhash = node.hex(hashlib.sha1(file).digest())
 return os.path.join(pathhash, id)
 
 def peersetup(ui, peer):
diff --git a/hgext/remotefilelog/debugcommands.py 
b/hgext/remotefilelog/debugcommands.py
--- a/hgext/remotefilelog/debugcommands.py
+++ b/hgext/remotefilelog/debugcommands.py
@@ -15,6 +15,7 @@
 from mercurial import (
 error,
 filelog,
+node as nodemod,
 revlog,
 )
 from . import (
@@ -52,7 +53,7 @@
 
 def buildtemprevlog(repo, file):
 # get filename key
-filekey = hashlib.sha1(file).hexdigest()
+filekey = nodemod.hex(hashlib.sha1(file).digest())
 filedir = os.path.join(repo.path, 'store/data', filekey)
 
 # sort all entries based on linkrev
@@ -344,7 +345,7 @@
 ui.write("%s  %s  %s  %s\n" % (
 hashformatter(node),
 hashformatter(deltabasenode),
-hashlib.sha1(delta).hexdigest(),
+nodemod.hex(hashlib.sha1(delta).digest()),
 len(delta)))
 
 def debughistorypack(ui, path):



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5259: py3: fix keyword arguments handling in hgext/remotefilelog/

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Keys of kwargs on Python 3 should be strings. This patch fixes them by 
appending
  r'' prefixes, and using pycompat.byteskwargs() and pycompat.strkwargs().

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5259

AFFECTED FILES
  hgext/remotefilelog/__init__.py
  hgext/remotefilelog/basestore.py
  hgext/remotefilelog/contentstore.py
  hgext/remotefilelog/debugcommands.py
  hgext/remotefilelog/fileserverclient.py
  hgext/remotefilelog/metadatastore.py
  hgext/remotefilelog/remotefilectx.py
  hgext/remotefilelog/shallowbundle.py
  hgext/remotefilelog/shallowutil.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/shallowutil.py 
b/hgext/remotefilelog/shallowutil.py
--- a/hgext/remotefilelog/shallowutil.py
+++ b/hgext/remotefilelog/shallowutil.py
@@ -105,7 +105,7 @@
 def reportpackmetrics(ui, prefix, *stores):
 dicts = [s.getmetrics() for s in stores]
 dict = prefixkeys(sumdicts(*dicts), prefix + '_')
-ui.log(prefix + "_packsizes", "", **dict)
+ui.log(prefix + "_packsizes", "", **pycompat.strkwargs(dict))
 
 def _parsepackmeta(metabuf):
 """parse datapack meta, bytes () -> dict
diff --git a/hgext/remotefilelog/shallowbundle.py 
b/hgext/remotefilelog/shallowbundle.py
--- a/hgext/remotefilelog/shallowbundle.py
+++ b/hgext/remotefilelog/shallowbundle.py
@@ -146,7 +146,7 @@
 try:
 # if serving, only send files the clients has patterns for
 if source == 'serve':
-bundlecaps = kwargs.get('bundlecaps')
+bundlecaps = kwargs.get(r'bundlecaps')
 includepattern = None
 excludepattern = None
 for cap in (bundlecaps or []):
diff --git a/hgext/remotefilelog/remotefilectx.py 
b/hgext/remotefilelog/remotefilectx.py
--- a/hgext/remotefilelog/remotefilectx.py
+++ b/hgext/remotefilelog/remotefilectx.py
@@ -15,6 +15,7 @@
 context,
 error,
 phases,
+pycompat,
 util,
 )
 from . import shallowutil
@@ -218,11 +219,11 @@
 return linknode
 
 commonlogkwargs = {
-'revs': ' '.join([hex(cl.node(rev)) for rev in revs]),
-'fnode': hex(fnode),
-'filepath': path,
-'user': shallowutil.getusername(repo.ui),
-'reponame': shallowutil.getreponame(repo.ui),
+r'revs': ' '.join([hex(cl.node(rev)) for rev in revs]),
+r'fnode': hex(fnode),
+r'filepath': path,
+r'user': shallowutil.getusername(repo.ui),
+r'reponame': shallowutil.getreponame(repo.ui),
 }
 
 repo.ui.log('linkrevfixup', 'adjusting linknode', **commonlogkwargs)
@@ -315,7 +316,7 @@
 finally:
 elapsed = time.time() - start
 repo.ui.log('linkrevfixup', logmsg, elapsed=elapsed * 1000,
-**commonlogkwargs)
+**pycompat.strkwargs(commonlogkwargs))
 
 def _verifylinknode(self, revs, linknode):
 """
@@ -408,7 +409,7 @@
 
 def annotate(self, *args, **kwargs):
 introctx = self
-prefetchskip = kwargs.pop('prefetchskip', None)
+prefetchskip = kwargs.pop(r'prefetchskip', None)
 if prefetchskip:
 # use introrev so prefetchskip can be accurately tested
 introrev = self.introrev()
diff --git a/hgext/remotefilelog/metadatastore.py 
b/hgext/remotefilelog/metadatastore.py
--- a/hgext/remotefilelog/metadatastore.py
+++ b/hgext/remotefilelog/metadatastore.py
@@ -11,12 +11,12 @@
 super(unionmetadatastore, self).__init__(*args, **kwargs)
 
 self.stores = args
-self.writestore = kwargs.get('writestore')
+self.writestore = kwargs.get(r'writestore')
 
 # If allowincomplete==True then the union store can return partial
 # ancestor lists, otherwise it will throw a KeyError if a full
 # history can't be found.
-self.allowincomplete = kwargs.get('allowincomplete', False)
+self.allowincomplete = kwargs.get(r'allowincomplete', False)
 
 def getancestors(self, name, node, known=None):
 """Returns as many ancestors as we're aware of.
diff --git a/hgext/remotefilelog/fileserverclient.py 
b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -18,6 +18,7 @@
 from mercurial.node import bin, hex, nullid
 from mercurial import (
 error,
+pycompat,
 revlog,
 sshpeer,
 util,
@@ -119,7 +120,7 @@
 def _callstream(self, command, **opts):
 supertype = super(remotefilepeer, self)
 if not util.safehasattr(supertype, '_sendrequest'):
-self._updatecallstreamopts(command, opts)
+self._updatecallstreamopts(command, pycompat.byteskwargs(opts))
 return super(remotefilepeer, self)._callstream(command,

D5260: py3: make second argument of fdopen() a str

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  1. skip-blame as just r'' prefixes

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5260

AFFECTED FILES
  hgext/remotefilelog/basepack.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/basepack.py b/hgext/remotefilelog/basepack.py
--- a/hgext/remotefilelog/basepack.py
+++ b/hgext/remotefilelog/basepack.py
@@ -373,8 +373,8 @@
 suffix=self.PACKSUFFIX + '-tmp')
 self.idxfp, self.idxpath = opener.mkstemp(
 suffix=self.INDEXSUFFIX + '-tmp')
-self.packfp = os.fdopen(self.packfp, 'w+')
-self.idxfp = os.fdopen(self.idxfp, 'w+')
+self.packfp = os.fdopen(self.packfp, r'w+')
+self.idxfp = os.fdopen(self.idxfp, r'w+')
 self.sha = hashlib.sha1()
 self._closed = False
 



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5039: py3: add pycache to .hgignore

2018-11-13 Thread pulkit (Pulkit Goyal)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGee9981bc8b44: py3: add __pycache__ to .hgignore (authored 
by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5039?vs=12514=12515

REVISION DETAIL
  https://phab.mercurial-scm.org/D5039

AFFECTED FILES
  contrib/python3-whitelist
  tests/test-custom-filters.t

CHANGE DETAILS

diff --git a/tests/test-custom-filters.t b/tests/test-custom-filters.t
--- a/tests/test-custom-filters.t
+++ b/tests/test-custom-filters.t
@@ -27,6 +27,7 @@
   > .hgignore
   > prefix.py
   > prefix.pyc
+  > __pycache__/
   > EOF
 
   $ cat > stuff.txt

D3771: py3: make contrib/perf.py compatible to Python 3

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit abandoned this revision.
pulkit added a comment.


  Not applicable anymore.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D3771

To: pulkit, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D4887: narrow: don't include the manifests while widening a narrow clone

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit abandoned this revision.
pulkit added a comment.


  Abandoing this one and https://phab.mercurial-scm.org/D4886 as @martinvonz's 
better fix landed as https://phab.mercurial-scm.org/D4895.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4887

To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D4849: store: introduce a function to get tracked path from a fncache entry

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit abandoned this revision.
pulkit added a comment.


  This was an incorrect implementation. A better implementation has been pushed 
as part of https://phab.mercurial-scm.org/D5139.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4849

To: pulkit, #hg-reviewers, indygreg
Cc: indygreg, yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5173: py3: make debugindexstats output printing compatible

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit abandoned this revision.
pulkit added a comment.


  I remember reading @yuja patches which took care of it.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5173

To: pulkit, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 4] logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)

2018-11-13 Thread Yuya Nishihara

# HG changeset patch
# User Yuya Nishihara 
# Date 1541908558 -32400
#  Sun Nov 11 12:55:58 2018 +0900
# Node ID 2f72218f9cfe8799617126ab453876f007d13b28
# Parent  355cce9374b1f3ca2ecee3c29957b37447b565a4
logtoprocess: drop support for ui.log() call with invalid msg arguments (BC)

Before, the logtoprocess extension put a formatted message into $MSG1, and
its arguments to $MSG2... If the specified arguments couldn't be formatted
because of a caller bug, an unformatted message was passed in to $MSG1
instead of exploding. This behavior doesn't make sense.

Since I'm planning to formalize the ui.log() interface such that we'll no
longer have to extend the ui class, I want to remove any features not
conforming to the ui.log() API. So this patch removes the support for
ill-formed arguments, and $MSG{n} (where n > 1) parameters which seems
useless as long as the message can be formatted. The $MSG1 variable isn't
renamed for the maximum compatibility.

In future patches, a formatted msg will be passed to a processlogger object,
instead of overriding the ui.log() function.

.. bc::

   The logtoprocess extension no longer supports invalid ``ui.log()``
   arguments. A log message is always formatted and passed in to the
   ``$MSG1`` environment variable.

diff --git a/hgext/logtoprocess.py b/hgext/logtoprocess.py
--- a/hgext/logtoprocess.py
+++ b/hgext/logtoprocess.py
@@ -9,21 +9,21 @@
 This extension lets you specify a shell command per ui.log() event,
 sending all remaining arguments to as environment variables to that command.
 
-Each positional argument to the method results in a `MSG[N]` key in the
-environment, starting at 1 (so `MSG1`, `MSG2`, etc.). Each keyword argument
-is set as a `OPT_UPPERCASE_KEY` variable (so the key is uppercased, and
-prefixed with `OPT_`). The original event name is passed in the `EVENT`
-environment variable, and the process ID of mercurial is given in `HGPID`.
+Positional arguments construct a log message, which is passed in the `MSG1`
+environment variables. Each keyword argument is set as a `OPT_UPPERCASE_KEY`
+variable (so the key is uppercased, and prefixed with `OPT_`). The original
+event name is passed in the `EVENT` environment variable, and the process ID
+of mercurial is given in `HGPID`.
 
-So given a call `ui.log('foo', 'bar', 'baz', spam='eggs'), a script configured
-for the `foo` event can expect an environment with `MSG1=bar`, `MSG2=baz`, and
-`OPT_SPAM=eggs`.
+So given a call `ui.log('foo', 'bar %s\n', 'baz', spam='eggs'), a script
+configured for the `foo` event can expect an environment with `MSG1=bar baz`,
+and `OPT_SPAM=eggs`.
 
 Scripts are configured in the `[logtoprocess]` section, each key an event name.
 For example::
 
   [logtoprocess]
-  commandexception = echo "$MSG2$MSG3" > /var/log/mercurial_exceptions.log
+  commandexception = echo "$MSG1" > /var/log/mercurial_exceptions.log
 
 would log the warning message and traceback of any failed command dispatch.
 
@@ -60,25 +60,11 @@ def uisetup(ui):
 """
 script = self.config('logtoprocess', event)
 if script:
-if msg:
-# try to format the log message given the remaining
-# arguments
-try:
-# Format the message as blackbox does
-formatted = msg[0] % msg[1:]
-except (TypeError, KeyError):
-# Failed to apply the arguments, ignore
-formatted = msg[0]
-messages = (formatted,) + msg[1:]
-else:
-messages = msg
 env = {
 b'EVENT': event,
 b'HGPID': os.getpid(),
+b'MSG1': msg[0] % msg[1:],
 }
-# positional arguments are listed as MSG[N] keys in the
-# environment
-env.update((b'MSG%d' % i, m) for i, m in enumerate(messages, 
1))
 # keyword arguments get prefixed with OPT_ and uppercased
 env.update((b'OPT_%s' % key.upper(), value)
for key, value in 
pycompat.byteskwargs(opts).items())
diff --git a/tests/test-logtoprocess.t b/tests/test-logtoprocess.t
--- a/tests/test-logtoprocess.t
+++ b/tests/test-logtoprocess.t
@@ -29,18 +29,14 @@ Test if logtoprocess correctly captures 
   > [logtoprocess]
   > command=(echo 'logtoprocess command output:';
   > echo "\$EVENT";
-  > echo "\$MSG1";
-  > echo "\$MSG2") > $TESTTMP/command.log
+  > echo "\$MSG1") > $TESTTMP/command.log
   > commandfinish=(echo 'logtoprocess commandfinish output:';
   > echo "\$EVENT";
   > echo "\$MSG1";
-  > echo "\$MSG2";
-  > echo "\$MSG3";
   > echo "canonical: \$OPT_CANONICAL_COMMAND") > $TESTTMP/commandfinish.log
   > foo=(echo 'logtoprocess foo output:';
   > echo "\$EVENT";
-  > echo "\$MSG1";
-  > echo "\$MSG2") >

[PATCH 3 of 4] py3: byte-stringify inline extension in test-logtoprocess.t

2018-11-13 Thread Yuya Nishihara

# HG changeset patch
# User Yuya Nishihara 
# Date 1541907338 -32400
#  Sun Nov 11 12:35:38 2018 +0900
# Node ID 355cce9374b1f3ca2ecee3c29957b37447b565a4
# Parent  f05dc9a1613a2086e8effb1dda5f446d52d5237f
py3: byte-stringify inline extension in test-logtoprocess.t

diff --git a/contrib/python3-whitelist b/contrib/python3-whitelist
--- a/contrib/python3-whitelist
+++ b/contrib/python3-whitelist
@@ -315,6 +315,7 @@ test-log-exthook.t
 test-log-linerange.t
 test-log.t
 test-logexchange.t
+test-logtoprocess.t
 test-lrucachedict.py
 test-mactext.t
 test-mailmap.t
diff --git a/tests/test-logtoprocess.t b/tests/test-logtoprocess.t
--- a/tests/test-logtoprocess.t
+++ b/tests/test-logtoprocess.t
@@ -14,12 +14,12 @@ Test if logtoprocess correctly captures 
   > command = registrar.command(cmdtable)
   > configtable = {}
   > configitem = registrar.configitem(configtable)
-  > configitem('logtoprocess', 'foo',
+  > configitem(b'logtoprocess', b'foo',
   > default=None,
   > )
   > @command(b'foobar', [])
   > def foo(ui, repo):
-  > ui.log('foo', 'a message: %s\n', 'spam')
+  > ui.log(b'foo', b'a message: %s\n', b'spam')
   > EOF
   $ cp $HGRCPATH $HGRCPATH.bak
   $ cat >> $HGRCPATH << EOF
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 4] logtoprocess: leverage procutil.shellenviron() to stringify variables (BC)

2018-11-13 Thread Yuya Nishihara

# HG changeset patch
# User Yuya Nishihara 
# Date 1541906843 -32400
#  Sun Nov 11 12:27:23 2018 +0900
# Node ID acd1e5000af16058913ff547e08a35d4583a6dac
# Parent  5c7dff08eea9b3f05995380e612546c9bc7a17ad
logtoprocess: leverage procutil.shellenviron() to stringify variables (BC)

This should make the extension more Py3 friendly. The environment variables
of the main process are copied to the dict by shellenviron().

.. bc::

   Boolean options passed to the logtoprocess extension are now formatted
   as ``0`` or ``1`` instead of ``None``, ``False``, or ``True``.

diff --git a/hgext/logtoprocess.py b/hgext/logtoprocess.py
--- a/hgext/logtoprocess.py
+++ b/hgext/logtoprocess.py
@@ -73,16 +73,16 @@ def uisetup(ui):
 # positional arguments are listed as MSG[N] keys in the
 # environment
 msgpairs = (
-('MSG{0:d}'.format(i), str(m))
+('MSG{0:d}'.format(i), m)
 for i, m in enumerate(messages, 1))
 # keyword arguments get prefixed with OPT_ and uppercased
 optpairs = (
-('OPT_{0}'.format(key.upper()), str(value))
+('OPT_{0}'.format(key.upper()), value)
 for key, value in opts.iteritems())
-env = dict(itertools.chain(procutil.shellenviron().items(),
-   msgpairs, optpairs),
-   EVENT=event, HGPID=str(os.getpid()))
-procutil.runbgcommand(script, env, shell=True)
+env = dict(itertools.chain(msgpairs, optpairs),
+   EVENT=event, HGPID=os.getpid())
+fullenv = procutil.shellenviron(env)
+procutil.runbgcommand(script, fullenv, shell=True)
 return super(logtoprocessui, self).log(event, *msg, **opts)
 
 # Replace the class for this instance and all clones created from it:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 4] logtoprocess: rewrite dict building in py3-compatible way

2018-11-13 Thread Yuya Nishihara

# HG changeset patch
# User Yuya Nishihara 
# Date 1541907194 -32400
#  Sun Nov 11 12:33:14 2018 +0900
# Node ID f05dc9a1613a2086e8effb1dda5f446d52d5237f
# Parent  acd1e5000af16058913ff547e08a35d4583a6dac
logtoprocess: rewrite dict building in py3-compatible way

diff --git a/hgext/logtoprocess.py b/hgext/logtoprocess.py
--- a/hgext/logtoprocess.py
+++ b/hgext/logtoprocess.py
@@ -34,9 +34,11 @@ not ensure that they exit cleanly.
 
 from __future__ import absolute_import
 
-import itertools
 import os
 
+from mercurial import (
+pycompat,
+)
 from mercurial.utils import (
 procutil,
 )
@@ -70,17 +72,16 @@ def uisetup(ui):
 messages = (formatted,) + msg[1:]
 else:
 messages = msg
+env = {
+b'EVENT': event,
+b'HGPID': os.getpid(),
+}
 # positional arguments are listed as MSG[N] keys in the
 # environment
-msgpairs = (
-('MSG{0:d}'.format(i), m)
-for i, m in enumerate(messages, 1))
+env.update((b'MSG%d' % i, m) for i, m in enumerate(messages, 
1))
 # keyword arguments get prefixed with OPT_ and uppercased
-optpairs = (
-('OPT_{0}'.format(key.upper()), value)
-for key, value in opts.iteritems())
-env = dict(itertools.chain(msgpairs, optpairs),
-   EVENT=event, HGPID=os.getpid())
+env.update((b'OPT_%s' % key.upper(), value)
+   for key, value in 
pycompat.byteskwargs(opts).items())
 fullenv = procutil.shellenviron(env)
 procutil.runbgcommand(script, fullenv, shell=True)
 return super(logtoprocessui, self).log(event, *msg, **opts)
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] templater: check invalid use of list expression properly (issue5920)

2018-11-13 Thread Yuya Nishihara

# HG changeset patch
# User Yuya Nishihara 
# Date 1542114930 -32400
#  Tue Nov 13 22:15:30 2018 +0900
# Node ID 5c7dff08eea9b3f05995380e612546c9bc7a17ad
# Parent  526ee887c4d512e4d2812edaef30908eba3fafc4
templater: check invalid use of list expression properly (issue5920)

The error message is still cryptic, but it should be better.

diff --git a/mercurial/templater.py b/mercurial/templater.py
--- a/mercurial/templater.py
+++ b/mercurial/templater.py
@@ -374,9 +374,7 @@ def compileexp(exp, context, curmethods)
 if not exp:
 raise error.ParseError(_("missing argument"))
 t = exp[0]
-if t in curmethods:
-return curmethods[t](exp, context)
-raise error.ParseError(_("unknown method '%s'") % t)
+return curmethods[t](exp, context)
 
 # template evaluation
 
@@ -496,6 +494,10 @@ def _buildfuncargs(exp, context, curmeth
 def buildkeyvaluepair(exp, content):
 raise error.ParseError(_("can't use a key-value pair in this context"))
 
+def buildlist(exp, context):
+raise error.ParseError(_("can't use a list in this context"),
+   hint=_('check place of comma and parens'))
+
 # methods to interpret function arguments or inner expressions (e.g. {_(x)})
 exprmethods = {
 "integer": lambda e, c: (templateutil.runinteger, e[1]),
@@ -508,6 +510,7 @@ exprmethods = {
 "%": buildmap,
 "func": buildfunc,
 "keyvalue": buildkeyvaluepair,
+"list": buildlist,
 "+": lambda e, c: buildarithmetic(e, c, lambda a, b: a + b),
 "-": lambda e, c: buildarithmetic(e, c, lambda a, b: a - b),
 "negate": buildnegate,
diff --git a/tests/test-template-basic.t b/tests/test-template-basic.t
--- a/tests/test-template-basic.t
+++ b/tests/test-template-basic.t
@@ -188,7 +188,8 @@ Call function which takes named argument
 
   $ hg debugtemplate '{" "|separate}'
   $ hg debugtemplate '{("not", "an", "argument", "list")|separate}'
-  hg: parse error: unknown method 'list'
+  hg: parse error: can't use a list in this context
+  (check place of comma and parens)
   [255]
 
 Second branch starting at nullrev:
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5039: py3: add pycache to .hgignore

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit updated this revision to Diff 12514.
pulkit retitled this revision from "py3: add more py3 specific output to test" 
to "py3: add __pycache__ to .hgignore".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5039?vs=11973=12514

REVISION DETAIL
  https://phab.mercurial-scm.org/D5039

AFFECTED FILES
  contrib/python3-whitelist
  tests/test-custom-filters.t

CHANGE DETAILS

diff --git a/tests/test-custom-filters.t b/tests/test-custom-filters.t
--- a/tests/test-custom-filters.t
+++ b/tests/test-custom-filters.t
@@ -27,6 +27,7 @@
   > .hgignore
   > prefix.py
   > prefix.pyc
+  > __pycache__/
   > EOF
 
   $ cat > stuff.txt

Re: [PATCH 11 of 14] sparse-revlog: add a `_trimchunk` function in C

2018-11-13 Thread Yuya Nishihara

On Mon, 12 Nov 2018 10:55:46 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld 
> # Date 1541785523 -3600
> #  Fri Nov 09 18:45:23 2018 +0100
> # Node ID 0ea42453fa491793d1e145f5093b65e84cb65e97
> # Parent  036a7425b869b5b8e3e7f528ed6d992c48ebb52e
> # EXP-Topic sparse-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> 0ea42453fa49
> sparse-revlog: add a `_trimchunk` function in C
> 
> We are about to implement a native version of `slicechunktodensity`. For
> clarity, we introduce the helper functions first.
> 
> This function is a native implementation of the python function `_trimchunk`
> in `mercurial/revlogutils/deltas.py`.
> 
> diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
> --- a/mercurial/cext/revlog.c
> +++ b/mercurial/cext/revlog.c
> @@ -1018,6 +1018,29 @@ static inline long index_segment_span(in
>   index_get_length(self, end_rev));
>  }
>  
> +/* returns revs[startidx:endidx] without empty trailing revs */
> +static PyObject *_trimchunk(indexObject *self, PyObject *revs, long startidx,
> +long endidx)
> +{
> + while (endidx > 1 && endidx > startidx) {
> + PyObject *rev = PyList_GET_ITEM(revs, endidx - 1);
> + if (rev == NULL) {
> + return NULL;
> + }

PyList_GET_ITEM() should never fail.

> + Py_ssize_t r = PyInt_AsLong(rev);
> + if (r == -1 && PyErr_Occurred()) {
> + return NULL;
> + }
> + if (index_get_length(self, r - 1) != 0) {
> + break;
> + }
> + endidx -= 1;
> + }
> + PyObject *chunk = PyList_GetSlice(revs, startidx, endidx);
> + Py_INCREF(chunk);
> + return chunk;

Excessive incref? PyList_GetSlice() is documented to return new reference.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 07 of 14] sparse-revlog: rework the way we enforce chunk size limit

2018-11-13 Thread Yuya Nishihara

On Mon, 12 Nov 2018 10:55:42 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld 
> # Date 1541782717 -3600
> #  Fri Nov 09 17:58:37 2018 +0100
> # Node ID b77a6b74ef31e1b3706c1c6127a15eede0334f71
> # Parent  ddafb271512fc26de60da5dceffc1509bb023d66
> # EXP-Topic sparse-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> b77a6b74ef31
> sparse-revlog: rework the way we enforce chunk size limit

> 
> We move from a O(N) algorithm to a O(log(N)) algorithm.
> 
> The previous algorithm was traversing the whole delta chain, looking for the
> exact point where it became too big. This would result in most of the delta
> chain to be traversed.
> 
> Instead, we now use a "binary" approach, slicing the chain in two until we
> have a chunk of the appropriate size.
> 
> We still keep the previous algorithm for the snapshots part. There are few of
> them and they are large bits of data distant from each other. So the previous
> algorithm should work well in that case.
> 
> To take a practical example of restoring manifest revision '59547c40bc4c' for
> a reference NetBeans repository (using sparse-revlog). The media time of the
> step `slice-sparse-chain` of `perfrevlogrevision` improve from 1.109 ms to
> 0.660 ms.
> 
> diff --git a/mercurial/revlogutils/deltas.py b/mercurial/revlogutils/deltas.py
> --- a/mercurial/revlogutils/deltas.py
> +++ b/mercurial/revlogutils/deltas.py
> @@ -176,18 +176,22 @@ def _slicechunktosize(revlog, revs, targ
>  [[3], [5]]
>  """
>  assert targetsize is None or 0 <= targetsize
> -if targetsize is None or segmentspan(revlog, revs) <= targetsize:
> +startdata = revlog.start(revs[0])
> +enddata = revlog.end(revs[-1])
> +fullspan = enddata - startdata
> +if targetsize is None or fullspan <= targetsize:
>  yield revs
>  return
>  
>  startrevidx = 0
> -startdata = revlog.start(revs[0])
>  endrevidx = 0
>  iterrevs = enumerate(revs)
>  next(iterrevs) # skip first rev.
> +# first step: get snapshots out of the way
>  for idx, r in iterrevs:
>  span = revlog.end(r) - startdata
> -if span <= targetsize:
> +snapshot = revlog.issnapshot(r)

This should break the doctest if this module were included in test-doctest.py.

> +if span <= targetsize and snapshot:
>  endrevidx = idx
>  else:
>  chunk = _trimchunk(revlog, revs, startrevidx, endrevidx + 1)
> @@ -196,6 +200,29 @@ def _slicechunktosize(revlog, revs, targ
>  startrevidx = idx
>  startdata = revlog.start(r)
>  endrevidx = idx
> +if not snapshot:
> +break
> +
> +# for the others, we use binary slicing to quickly converge toward valid
> +# chunks (otherwise, we might end up looking for start/end of many
> +# revisions)
> +nbitem = len(revs)
> +while (enddata - startdata) > targetsize:
> +endrevidx = nbitem
> +if nbitem - startrevidx <= 1:
> +break # protect against individual chunk larger than limit
> +localenddata = revlog.end(revs[endrevidx - 1])
> +span = localenddata - startdata
> +while (localenddata - startdata) > targetsize:
> +if endrevidx - startrevidx <= 1:
> +break # protect against individual chunk larger than limit
> +endrevidx -= (endrevidx - startrevidx) // 2
> +localenddata = revlog.end(revs[endrevidx -1])
> +span = localenddata - startdata

The variable 'span' isn't used.

> +yield _trimchunk(revlog, revs, startrevidx, endrevidx)
> +startrevidx = endrevidx
> +startdata = revlog.start(revs[startrevidx])

Do we have enough test coverage for this logic?

FWIW, I'm a little confused that 'endrevidx' only gets smaller and smaller,
since I thought this would do binary-search the split point. That turned out
to be wrong. We look for approximate positions to split.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 06 of 14] sparse-revlog: stop using a heap to track selected gap

2018-11-13 Thread Yuya Nishihara

On Mon, 12 Nov 2018 10:55:41 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld 
> # Date 1541689636 -3600
> #  Thu Nov 08 16:07:16 2018 +0100
> # Node ID ddafb271512fc26de60da5dceffc1509bb023d66
> # Parent  8ebe5520cc4ae87f6fccba20897d292489a651db
> # EXP-Topic sparse-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> ddafb271512f
> sparse-revlog: stop using a heap to track selected gap

Queued up to this patch, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5146: histedit: import chistedit curses UI from hg-experimental

2018-11-13 Thread durin42 (Augie Fackler)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGc36175456350: histedit: import chistedit curses UI from 
hg-experimental (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5146?vs=12322=12513

REVISION DETAIL
  https://phab.mercurial-scm.org/D5146

AFFECTED FILES
  hgext/histedit.py
  mercurial/ui.py

CHANGE DETAILS

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1245,7 +1245,11 @@
 "chunkselector": [
 "text",
 "curses",
-]
+],
+"histedit": [
+"text",
+"curses",
+],
 }
 
 # Feature-specific interface
diff --git a/hgext/histedit.py b/hgext/histedit.py
--- a/hgext/histedit.py
+++ b/hgext/histedit.py
@@ -183,7 +183,11 @@
 
 from __future__ import absolute_import
 
+import fcntl
+import functools
 import os
+import struct
+import termios
 
 from mercurial.i18n import _
 from mercurial import (
@@ -198,6 +202,7 @@
 extensions,
 hg,
 lock,
+logcmdutil,
 merge as mergemod,
 mergeutil,
 node,
@@ -235,6 +240,9 @@
 configitem('histedit', 'singletransaction',
 default=False,
 )
+configitem('ui', 'interface.histedit',
+default=None,
+)
 
 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' 
for
 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
@@ -915,6 +923,562 @@
 raise error.Abort(msg, hint=hint)
 return repo[roots[0]].node()
 
+# Curses Support
+try:
+import curses
+except ImportError:
+curses = None
+
+KEY_LIST = ['pick', 'edit', 'fold', 'drop', 'mess', 'roll']
+ACTION_LABELS = {
+'fold': '^fold',
+'roll': '^roll',
+}
+
+COLOR_HELP, COLOR_SELECTED, COLOR_OK, COLOR_WARN  = 1, 2, 3, 4
+
+E_QUIT, E_HISTEDIT = 1, 2
+E_PAGEDOWN, E_PAGEUP, E_LINEUP, E_LINEDOWN, E_RESIZE = 3, 4, 5, 6, 7
+MODE_INIT, MODE_PATCH, MODE_RULES, MODE_HELP = 0, 1, 2, 3
+
+KEYTABLE = {
+'global': {
+'h': 'next-action',
+'KEY_RIGHT': 'next-action',
+'l': 'prev-action',
+'KEY_LEFT':  'prev-action',
+'q': 'quit',
+'c': 'histedit',
+'C': 'histedit',
+'v': 'showpatch',
+'?': 'help',
+},
+MODE_RULES: {
+'d': 'action-drop',
+'e': 'action-edit',
+'f': 'action-fold',
+'m': 'action-mess',
+'p': 'action-pick',
+'r': 'action-roll',
+' ': 'select',
+'j': 'down',
+'k': 'up',
+'KEY_DOWN':  'down',
+'KEY_UP':'up',
+'J': 'move-down',
+'K': 'move-up',
+'KEY_NPAGE': 'move-down',
+'KEY_PPAGE': 'move-up',
+'0': 'goto',  # Used for 0..9
+},
+MODE_PATCH: {
+' ': 'page-down',
+'KEY_NPAGE': 'page-down',
+'KEY_PPAGE': 'page-up',
+'j': 'line-down',
+'k': 'line-up',
+'KEY_DOWN':  'line-down',
+'KEY_UP':'line-up',
+'J': 'down',
+'K': 'up',
+},
+MODE_HELP: {
+},
+}
+
+def screen_size():
+return struct.unpack('hh', fcntl.ioctl(1, termios.TIOCGWINSZ, ''))
+
+class histeditrule(object):
+def __init__(self, ctx, pos, action='pick'):
+self.ctx = ctx
+self.action = action
+self.origpos = pos
+self.pos = pos
+self.conflicts = []
+
+def __str__(self):
+# Some actions ('fold' and 'roll') combine a patch with a previous one.
+# Add a marker showing which patch they apply to, and also omit the
+# description for 'roll' (since it will get discarded). Example 
display:
+#
+#  #10 pick   316392:06a16c25c053   add option to skip tests
+#  #11 ^roll  316393:71313c964cc5
+#  #12 pick   316394:ab31f3973b0d   include mfbt for mozilla-config.h
+#  #13 ^fold  316395:14ce5803f4c3   fix warnings
+#
+# The carets point to the changeset being folded into ("roll this
+# changeset into the changeset above").
+action = ACTION_LABELS.get(self.action, self.action)
+h = self.ctx.hex()[0:12]
+r = self.ctx.rev()
+desc = self.ctx.description().splitlines()[0].strip()
+if self.action == 'roll':
+desc = ''
+return "#{0:<2} {1:<6} {2}:{3}   {4}".format(
+self.origpos, action, r, h, desc)
+
+def checkconflicts(self, other):
+if other.pos > self.pos and other.origpos <= self.origpos:
+if set(other.ctx.files()) & set(self.ctx.files()) != set():
+self.conflicts.append(other)
+return self.conflicts
+
+if other in self.conflicts:

D5146: histedit: import chistedit curses UI from hg-experimental

2018-11-13 Thread pulkit (Pulkit Goyal)

pulkit accepted this revision.
pulkit added a comment.


  Queueing this, many thanks for importing this in core.
  
  Can you add some documentation about the new config option to enable curses 
interface as followup?
  
  Also, how do you feel about histedit using curses interface if 
`ui.interface=curses` is set?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5146

To: durin42, #hg-reviewers, martinvonz, pulkit
Cc: martinvonz, pulkit, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5258: manifest: perform cheap checks before potentially allocating memory

2018-11-13 Thread durin42 (Augie Fackler)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGda4478ca0e32: manifest: perform cheap checks before 
potentially allocating memory (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5258?vs=12507=12512

REVISION DETAIL
  https://phab.mercurial-scm.org/D5258

AFFECTED FILES
  mercurial/cext/manifest.c

CHANGE DETAILS

diff --git a/mercurial/cext/manifest.c b/mercurial/cext/manifest.c
--- a/mercurial/cext/manifest.c
+++ b/mercurial/cext/manifest.c
@@ -137,13 +137,13 @@
return MANIFEST_TOO_SHORT_LINE;
}
next++; /* advance past newline */
-   if (!realloc_if_full(self)) {
-   return MANIFEST_OOM; /* no memory */
-   }
if (prev && strcmp(prev, data) > -1) {
/* This data isn't sorted, so we have to abort. */
return MANIFEST_NOT_SORTED;
}
+   if (!realloc_if_full(self)) {
+   return MANIFEST_OOM; /* no memory */
+   }
l = self->lines + ((self->numlines)++);
l->start = data;
l->len = next - data;



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5257: manifest: also reject obviously-too-short lines when parsing lines

2018-11-13 Thread durin42 (Augie Fackler)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGf27f8e9ef1e7: manifest: also reject obviously-too-short 
lines when parsing lines (authored by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5257?vs=12506=12511

REVISION DETAIL
  https://phab.mercurial-scm.org/D5257

AFFECTED FILES
  mercurial/cext/manifest.c

CHANGE DETAILS

diff --git a/mercurial/cext/manifest.c b/mercurial/cext/manifest.c
--- a/mercurial/cext/manifest.c
+++ b/mercurial/cext/manifest.c
@@ -39,6 +39,7 @@
 #define MANIFEST_NOT_SORTED -2
 #define MANIFEST_MALFORMED -3
 #define MANIFEST_BOGUS_FILENAME -4
+#define MANIFEST_TOO_SHORT_LINE -5
 
 /* get the length of the path for a line */
 static size_t pathlen(line *l)
@@ -126,6 +127,15 @@
if (!next) {
return MANIFEST_MALFORMED;
}
+   if ((next - data) < 22) {
+   /* We should have at least 22 bytes in a line:
+  1 byte filename
+  1 NUL
+  20 bytes of hash
+  so we can give up here.
+   */
+   return MANIFEST_TOO_SHORT_LINE;
+   }
next++; /* advance past newline */
if (!realloc_if_full(self)) {
return MANIFEST_OOM; /* no memory */
@@ -202,6 +212,11 @@
PyExc_ValueError,
"Manifest had an entry with a zero-length filename.");
break;
+   case MANIFEST_TOO_SHORT_LINE:
+   PyErr_Format(
+   PyExc_ValueError,
+   "Manifest had implausibly-short line.");
+   break;
default:
PyErr_Format(PyExc_ValueError,
 "Unknown problem parsing manifest.");



To: durin42, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5235: revlog: replace PyInt_AS_LONG with a more portable helper function

2018-11-13 Thread durin42 (Augie Fackler)

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGfa33196088c4: revlog: replace PyInt_AS_LONG with a more 
portable helper function (authored by durin42, committed by ).

CHANGED PRIOR TO COMMIT
  https://phab.mercurial-scm.org/D5235?vs=12502=12509#toc

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D5235?vs=12502=12509

REVISION DETAIL
  https://phab.mercurial-scm.org/D5235

AFFECTED FILES
  mercurial/cext/revlog.c
  mercurial/cext/util.h

CHANGE DETAILS

diff --git a/mercurial/cext/util.h b/mercurial/cext/util.h
--- a/mercurial/cext/util.h
+++ b/mercurial/cext/util.h
@@ -58,4 +58,17 @@
return _PyDict_NewPresized(((1 + expected_size) / 2) * 3);
 }
 
+/* Convert a PyInt or PyLong to a long. Returns false if there is an
+   error, in which case an exception will already have been set. */
+static inline bool pylong_to_long(PyObject *pylong, long *out)
+{
+   *out = PyLong_AsLong(pylong);
+   /* Fast path to avoid hitting PyErr_Occurred if the value was obviously
+* not an error. */
+   if (*out != -1) {
+   return true;
+   }
+   return PyErr_Occurred() == NULL;
+}
+
 #endif /* _HG_UTIL_H_ */
diff --git a/mercurial/cext/revlog.c b/mercurial/cext/revlog.c
--- a/mercurial/cext/revlog.c
+++ b/mercurial/cext/revlog.c
@@ -24,7 +24,6 @@
 #define PyInt_Check PyLong_Check
 #define PyInt_FromLong PyLong_FromLong
 #define PyInt_FromSsize_t PyLong_FromSsize_t
-#define PyInt_AS_LONG PyLong_AS_LONG
 #define PyInt_AsLong PyLong_AsLong
 #endif
 
@@ -161,10 +160,17 @@
 int maxrev)
 {
if (rev >= self->length) {
+   long tmp;
PyObject *tuple =
PyList_GET_ITEM(self->added, rev - self->length);
-   ps[0] = (int)PyInt_AS_LONG(PyTuple_GET_ITEM(tuple, 5));
-   ps[1] = (int)PyInt_AS_LONG(PyTuple_GET_ITEM(tuple, 6));
+   if (!pylong_to_long(PyTuple_GET_ITEM(tuple, 5), )) {
+   return -1;
+   }
+   ps[0] = (int)tmp;
+   if (!pylong_to_long(PyTuple_GET_ITEM(tuple, 6), )) {
+   return -1;
+   }
+   ps[1] = (int)tmp;
} else {
const char *data = index_deref(self, rev);
ps[0] = getbe32(data + 24);
@@ -464,7 +470,10 @@
if (iter == NULL)
return -2;
while ((iter_item = PyIter_Next(iter))) {
-   iter_item_long = PyInt_AS_LONG(iter_item);
+   if (!pylong_to_long(iter_item, _item_long)) {
+   Py_DECREF(iter_item);
+   return -2;
+   }
Py_DECREF(iter_item);
if (iter_item_long < min_idx)
min_idx = iter_item_long;
@@ -853,7 +862,11 @@
if (rev >= self->length) {
PyObject *tuple =
PyList_GET_ITEM(self->added, rev - self->length);
-   return (int)PyInt_AS_LONG(PyTuple_GET_ITEM(tuple, 3));
+   long ret;
+   if (!pylong_to_long(PyTuple_GET_ITEM(tuple, 3), )) {
+   return -2;
+   }
+   return (int)ret;
} else {
data = index_deref(self, rev);
if (data == NULL) {
@@ -1384,8 +1397,13 @@
char *node;
int rev;
 
-   if (PyInt_Check(value))
-   return index_get(self, PyInt_AS_LONG(value));
+   if (PyInt_Check(value)) {
+   long idx;
+   if (!pylong_to_long(value, )) {
+   return NULL;
+   }
+   return index_get(self, idx);
+   }
 
if (node_check(value, ) == -1)
return NULL;
@@ -1516,7 +1534,10 @@
char *node;
 
if (PyInt_Check(value)) {
-   long rev = PyInt_AS_LONG(value);
+   long rev;
+   if (!pylong_to_long(value, )) {
+   return -1;
+   }
return rev >= -1 && rev < index_length(self);
}
 
@@ -2404,10 +2425,12 @@
 
 static int rustla_contains(rustlazyancestorsObject *self, PyObject *rev)
 {
-   if (!(PyInt_Check(rev))) {
+   long lrev;
+   if (!pylong_to_long(rev, )) {
+   PyErr_Clear();
return 0;
}
-   return rustlazyancestors_contains(self->iter, PyInt_AS_LONG(rev));
+   return rustlazyancestors_contains(self->iter, lrev);
 }
 
 static PySequenceMethods rustla_sequence_methods = {



To: durin42, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D5257: manifest: also reject obviously-too-short lines when parsing lines

2018-11-13 Thread yuja (Yuya Nishihara)

yuja added a comment.


  > +   if ((next - data) < 22) {
  >  +  /* We should have at least 22 bytes in a line:
  >  + 1 byte filename
  >  + 1 NUL
  >  + 20 bytes of hash
  
  Queued, but I think it should be **40** bytes of hexadecimal hash.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5257

To: durin42, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D5257: manifest: also reject obviously-too-short lines when parsing lines

2018-11-13 Thread Yuya Nishihara

> + if ((next - data) < 22) {
> + /* We should have at least 22 bytes in a line:
> +1 byte filename
> +1 NUL
> +20 bytes of hash

Queued, but I think it should be **40** bytes of hexadecimal hash.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

55 matches

Mail list logo