D8146: tests: stabilize test-rename-merge2.t on Windows

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  In D8146#121279 , @mharbison72 
wrote:
  
  > In D8146#121277 , @martinvonz 
wrote:
  >
  >> In D8146#121271 , 
@mharbison72 wrote:
  >>
  >>> In D8146#121267 , @marmoute 
wrote:
  >>>
   Since the line does not occurs on linux, this probably got affected by 
some other change without being noticed.
  >>>
  >>> Makes sense for why it was missed, but I bisected it back to the 
referenced commit.  I know @martinvonz made some other related changes prior to 
that commit that had similar test fallout on Windows, but it isn't obvious to 
me why the referenced commit would change the order of when the background 
thread spins up.
  >>
  >> Perhaps the line doesn't actually happen before that commit? You can try 
removing the `(?)` in the commit before my commit and see if that line is 
actually printed. I'm thinking that that commit pushed the number of files to 
close above some threshold that made it get printed. In that case, it was 
probably some of my earlier commits that made it no longer get printed and we 
just didn't notice since `(?)` means optional (no error if it doesn't get 
printed).
  >
  > I can confirm that it was required in e1ecfc7c84be 
, 
and reconfirmed that it is needed in the latest code by removing the line from 
both.  I'm not too worried about why it popped up- it just seemed odd that it 
didn't with the earlier followup patch I added to fix similar conditionalizing 
of this exact text elsewhere, and the commit where it started failing looked 
innocuous.
  
  Yeah, I'm not worried about it either. Thanks for checking. (I wasn't holding 
off queuing this one because of that, I was/am just busy with other stuff.)

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8146/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8146

To: mharbison72, #hg-reviewers, marmoute
Cc: martinvonz, marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8146: tests: stabilize test-rename-merge2.t on Windows

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.


  In D8146#121277 , @martinvonz 
wrote:
  
  > In D8146#121271 , @mharbison72 
wrote:
  >
  >> In D8146#121267 , @marmoute 
wrote:
  >>
  >>> Since the line does not occurs on linux, this probably got affected by 
some other change without being noticed.
  >>
  >> Makes sense for why it was missed, but I bisected it back to the 
referenced commit.  I know @martinvonz made some other related changes prior to 
that commit that had similar test fallout on Windows, but it isn't obvious to 
me why the referenced commit would change the order of when the background 
thread spins up.
  >
  > Perhaps the line doesn't actually happen before that commit? You can try 
removing the `(?)` in the commit before my commit and see if that line is 
actually printed. I'm thinking that that commit pushed the number of files to 
close above some threshold that made it get printed. In that case, it was 
probably some of my earlier commits that made it no longer get printed and we 
just didn't notice since `(?)` means optional (no error if it doesn't get 
printed).
  
  I can confirm that it was required in e1ecfc7c84be 
, 
and reconfirmed that it is needed in the latest code by removing the line from 
both.  I'm not too worried about why it popped up- it just seemed odd that it 
didn't with the earlier followup patch I added to fix similar conditionalizing 
of this exact text elsewhere, and the commit where it started failing looked 
innocuous.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8146/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8146

To: mharbison72, #hg-reviewers, marmoute
Cc: martinvonz, marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8146: tests: stabilize test-rename-merge2.t on Windows

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  In D8146#121271 , @mharbison72 
wrote:
  
  > In D8146#121267 , @marmoute 
wrote:
  >
  >> Since the line does not occurs on linux, this probably got affected by 
some other change without being noticed.
  >
  > Makes sense for why it was missed, but I bisected it back to the referenced 
commit.  I know @martinvonz made some other related changes prior to that 
commit that had similar test fallout on Windows, but it isn't obvious to me why 
the referenced commit would change the order of when the background thread 
spins up.
  
  Perhaps the line doesn't actually happen before that commit? You can try 
removing the `(?)` in the commit before my commit and see if that line is 
actually printed. I'm thinking that that commit pushed the number of files to 
close above some threshold that made it get printed. In that case, it was 
probably some of my earlier commits that made it no longer get printed and we 
just didn't notice since `(?)` means optional (no error if it doesn't get 
printed).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8146/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8146

To: mharbison72, #hg-reviewers, marmoute
Cc: martinvonz, marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8145: lfutil: provide a hint if the largefiles/lfs cache path cannot be determined

[mharbison72 (Matt Harbison)]

mharbison72 edited the summary of this revision.
mharbison72 updated this revision to Diff 20293.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D8145?vs=20290&id=20293

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8145/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8145

AFFECTED FILES
  hgext/largefiles/lfutil.py
  tests/test-lfs-bundle.t

CHANGE DETAILS

diff --git a/tests/test-lfs-bundle.t b/tests/test-lfs-bundle.t
--- a/tests/test-lfs-bundle.t
+++ b/tests/test-lfs-bundle.t
@@ -95,3 +95,27 @@
   OK
    Applying src-lfs.bundle to dst-lfs 
   OK
+
+Hint if the cache location cannot be inferred from the environment
+
+#if windows
+  $ env -u HOME -u LOCALAPPDATA -u APPDATA HGRCPATH= hg config lfs --debug
+  abort: unknown lfs usercache location
+  (define LOCALAPPDATA or APPDATA in the environment, or set lfs.usercache)
+  [255]
+#endif
+
+#if osx
+  $ env -u HOME HGRCPATH= hg config lfs --debug
+  abort: unknown lfs usercache location
+  (define HOME in the environment, or set lfs.usercache)
+  [255]
+#endif
+
+#if no-windows no-osx
+  $ env -u XDG_CACHE_HOME -u HOME HGRCPATH= hg config lfs --debug
+  abort: unknown lfs usercache location
+  (define XDG_CACHE_HOME or HOME in the environment, or set lfs.usercache)
+  [255]
+#endif
+
diff --git a/hgext/largefiles/lfutil.py b/hgext/largefiles/lfutil.py
--- a/hgext/largefiles/lfutil.py
+++ b/hgext/largefiles/lfutil.py
@@ -92,16 +92,30 @@
 path = ui.configpath(name, b'usercache')
 if path:
 return path
+
+hint = None
+
 if pycompat.iswindows:
 appdata = encoding.environ.get(
 b'LOCALAPPDATA', encoding.environ.get(b'APPDATA')
 )
 if appdata:
 return os.path.join(appdata, name)
+
+hint = _(b"define %s or %s in the environment, or set %s.usercache") % 
(
+b"LOCALAPPDATA",
+b"APPDATA",
+name,
+)
 elif pycompat.isdarwin:
 home = encoding.environ.get(b'HOME')
 if home:
 return os.path.join(home, b'Library', b'Caches', name)
+
+hint = _(b"define %s in the environment, or set %s.usercache") % (
+b"HOME",
+name,
+)
 elif pycompat.isposix:
 path = encoding.environ.get(b'XDG_CACHE_HOME')
 if path:
@@ -109,11 +123,18 @@
 home = encoding.environ.get(b'HOME')
 if home:
 return os.path.join(home, b'.cache', name)
+
+hint = _(b"define %s or %s in the environment, or set %s.usercache") % 
(
+b"XDG_CACHE_HOME",
+b"HOME",
+name,
+)
 else:
 raise error.Abort(
 _(b'unknown operating system: %s\n') % pycompat.osname
 )
-raise error.Abort(_(b'unknown %s usercache location') % name)
+
+raise error.Abort(_(b'unknown %s usercache location') % name, hint=hint)
 
 
 def inusercache(ui, hash):



To: mharbison72, #hg-reviewers
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8146: tests: stabilize test-rename-merge2.t on Windows

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  Since run-test has no idea where to put (?) line with the surounding change, 
they often end up preserved at start/end of the changeset section. This is 
probably just something like that.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8146/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8146

To: mharbison72, #hg-reviewers, marmoute
Cc: martinvonz, marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8145: lfutil: provide a hint if the largefiles/lfs cache path cannot be determined

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  In D8145#121270 , @mharbison72 
wrote:
  
  > In D8145#121269 , @marmoute 
wrote:
  >
  >> Could we get a test for this? To make sure the hint looks fine and that 
the error code does not crash ?
  >
  > The reason I didn't add a test (or see any changes to existing ones) is 
because the config property is baked into the config file generated by 
run-tests.py, so it never hits this path.  But I did test by hand on Windows.
  
  Could we alter that config in test file to trigger the error in there ?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8145/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8145

To: mharbison72, #hg-reviewers
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8146: tests: stabilize test-rename-merge2.t on Windows

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.
mharbison72 added a subscriber: martinvonz.


  In D8146#121267 , @marmoute 
wrote:
  
  > Since the line does not occurs on linux, this probably got affected by some 
other change without being noticed.
  
  Makes sense for why it was missed, but I bisected it back to the referenced 
commit.  I know @martinvonz made some other related changes prior to that 
commit that had similar test fallout on Windows, but it isn't obvious to me why 
the referenced commit would change the order of when the background thread 
spins up.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8146/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8146

To: mharbison72, #hg-reviewers, marmoute
Cc: martinvonz, marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8145: lfutil: provide a hint if the largefiles/lfs cache path cannot be determined

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.


  In D8145#121269 , @marmoute 
wrote:
  
  > Could we get a test for this? To make sure the hint looks fine and that the 
error code does not crash ?
  
  The reason I didn't add a test (or see any changes to existing ones) is 
because the config property is baked into the config file generated by 
run-tests.py, so it never hits this path.  But I did test by hand on Windows.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8145/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8145

To: mharbison72, #hg-reviewers
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8145: lfutil: provide a hint if the largefiles/lfs cache path cannot be determined

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  Could we get a test for this? To make sure the hint looks fine and that the 
error code does not crash ?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8145/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8145

To: mharbison72, #hg-reviewers
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8146: tests: stabilize test-rename-merge2.t on Windows

[marmoute (Pierre-Yves David)]

marmoute added a comment.
marmoute accepted this revision.


  Since the line does not occurs on linux, this probably got affected by some 
other change without being noticed.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8146/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8146

To: mharbison72, #hg-reviewers, marmoute
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8147: tests: replace truncate(1) with inline python

[marmoute (Pierre-Yves David)]

marmoute added a comment.
marmoute accepted this revision.


  sure

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8147/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8147

To: mharbison72, #hg-reviewers, marmoute
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8143: rust-cpython: make `NonNormalEntires` iterable to fix bug with `fsmonitor`

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  You need to add `(issue6276)` in your changeset summary so that the ticket 
get closed when you land this.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8143/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8143

To: Alphare, #hg-reviewers
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8147: tests: replace truncate(1) with inline python

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  MSYS doesn't have truncate(1) installed by default, and FreeBSD looked unhappy
  with the arguments provided.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D8147

AFFECTED FILES
  tests/test-tags.t

CHANGE DETAILS

diff --git a/tests/test-tags.t b/tests/test-tags.t
--- a/tests/test-tags.t
+++ b/tests/test-tags.t
@@ -374,7 +374,11 @@
 On junk data + missing cache entries, hg also overwrites the junk.
 
   $ rm -f .hg/cache/tags2-visible
-  $ truncate .hg/cache/hgtagsfnodes1 -s -10
+  >>> import os
+  >>> with open(".hg/cache/hgtagsfnodes1", "ab+") as fp:
+  ... fp.seek(-10, os.SEEK_END) and None
+  ... fp.truncate() and None
+
   $ hg debugtagscache | tail -2
   4 0c192d7d5e6b78a714de54a2e9627952a877e25a 
0c04f2a8af31de17fab7422878ee5a2dadbc943d
   5 8dbfe60eff306a54259cfe007db9e330e7ecf866 missing/invalid



To: mharbison72, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8146: tests: stabilize test-rename-merge2.t on Windows

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  I have no idea why, but this shifted in b4057d001760 
.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D8146

AFFECTED FILES
  tests/test-rename-merge2.t

CHANGE DETAILS

diff --git a/tests/test-rename-merge2.t b/tests/test-rename-merge2.t
--- a/tests/test-rename-merge2.t
+++ b/tests/test-rename-merge2.t
@@ -722,8 +722,8 @@
ancestor: 924404dff337, local: 0b76e65c8289+, remote: bdb19105162a
preserving b for resolve of b
preserving rev for resolve of rev
+  starting 4 threads for background file closing (?)
b: both renamed from a -> m (premerge)
-  starting 4 threads for background file closing (?)
   picked tool '* ../merge' for b (binary False symlink False changedelete 
False) (glob)
   merging b
   my b@0b76e65c8289+ other b@bdb19105162a ancestor a@924404dff337



To: mharbison72, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8145: lfutil: provide a hint if the largefiles/lfs cache path cannot be determined

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  A coworker hit this error using an LFS repo in a stripped down environment, 
and
  didn't know how to resolve it.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D8145

AFFECTED FILES
  hgext/largefiles/lfutil.py

CHANGE DETAILS

diff --git a/hgext/largefiles/lfutil.py b/hgext/largefiles/lfutil.py
--- a/hgext/largefiles/lfutil.py
+++ b/hgext/largefiles/lfutil.py
@@ -92,16 +92,30 @@
 path = ui.configpath(name, b'usercache')
 if path:
 return path
+
+hint = None
+
 if pycompat.iswindows:
 appdata = encoding.environ.get(
 b'LOCALAPPDATA', encoding.environ.get(b'APPDATA')
 )
 if appdata:
 return os.path.join(appdata, name)
+
+hint = _(b"define %s or %s in the environment, or set %s.usercache") % 
(
+b"LOCALAPPDATA",
+b"APPDATA",
+name,
+)
 elif pycompat.isdarwin:
 home = encoding.environ.get(b'HOME')
 if home:
 return os.path.join(home, b'Library', b'Caches', name)
+
+hint = _(b"define %s in the environment, or set %s.usercache") % (
+b"HOME",
+name,
+)
 elif pycompat.isposix:
 path = encoding.environ.get(b'XDG_CACHE_HOME')
 if path:
@@ -109,11 +123,18 @@
 home = encoding.environ.get(b'HOME')
 if home:
 return os.path.join(home, b'.cache', name)
+
+hint = _(b"define %s or %s in the environment, or set %s.usercache") % 
(
+b"XDG_CACHE_HOME",
+b"HOME",
+name,
+)
 else:
 raise error.Abort(
 _(b'unknown operating system: %s\n') % pycompat.osname
 )
-raise error.Abort(_(b'unknown %s usercache location') % name)
+
+raise error.Abort(_(b'unknown %s usercache location') % name, hint=hint)
 
 
 def inusercache(ui, hash):



To: mharbison72, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8144: tests: stabilize test-amend.t on Windows

[mharbison72 (Matt Harbison)]

mharbison72 added a comment.


  This is meant for stable.  The other test fixes I'll submit shortly are for 
default.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8144/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8144

To: mharbison72, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8144: tests: stabilize test-amend.t on Windows

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  If $TESTTMP isn't quoted in this context, it ends up like
  `C:Temphgtests.pikkoxchild1test-amend.t-obsstore-off`.

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D8144

AFFECTED FILES
  tests/test-amend.t

CHANGE DETAILS

diff --git a/tests/test-amend.t b/tests/test-amend.t
--- a/tests/test-amend.t
+++ b/tests/test-amend.t
@@ -487,7 +487,7 @@
   $ echo beta >> foo
   $ cat > $TESTTMP/touchy_editor.sh < sleep 1
-  > echo delta >> $TESTTMP/modify-during-amend/foo
+  > echo delta >> "$TESTTMP/modify-during-amend/foo"
   > sleep 1
   > echo hi > "\$1"
   > sleep 1



To: mharbison72, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@44373: 41 new changesets (1 on stable)

[Mercurial Commits]

41 new changesets (1 on stable) in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/50ad851efd9b
changeset:   44333:50ad851efd9b
parent:  44323:76a96e3a2bbb
user:Pierre-Yves David 
date:Wed Jan 15 15:49:54 2020 +0100
summary: nodemap: introduce append-only incremental update of the 
persistent data

https://www.mercurial-scm.org/repo/hg/rev/6614b301ea58
changeset:   44334:6614b301ea58
user:Pierre-Yves David 
date:Wed Jan 15 15:50:04 2020 +0100
summary: nodemap: add a flag to dump the details of the docket

https://www.mercurial-scm.org/repo/hg/rev/e41a164db7a9
changeset:   44335:e41a164db7a9
user:Pierre-Yves David 
date:Wed Jan 15 15:50:14 2020 +0100
summary: nodemap: track the maximum revision tracked in the nodemap

https://www.mercurial-scm.org/repo/hg/rev/8374b69aef75
changeset:   44336:8374b69aef75
user:Pierre-Yves David 
date:Wed Jan 15 15:50:24 2020 +0100
summary: nodemap: track the total and unused amount of data in the rawdata 
file

https://www.mercurial-scm.org/repo/hg/rev/1d2b37def017
changeset:   44337:1d2b37def017
user:Pierre-Yves David 
date:Wed Jan 15 15:50:33 2020 +0100
summary: nodemap: double check the source docket when doing incremental 
update

https://www.mercurial-scm.org/repo/hg/rev/2ea6a67ff502
changeset:   44338:2ea6a67ff502
user:Pierre-Yves David 
date:Wed Jan 15 15:50:43 2020 +0100
summary: nodemap: write new data from the expected current data length

https://www.mercurial-scm.org/repo/hg/rev/c7eebdb15139
changeset:   44339:c7eebdb15139
user:Pierre-Yves David 
date:Wed Jan 15 15:50:52 2020 +0100
summary: nodemap: never read more than the expected data amount

https://www.mercurial-scm.org/repo/hg/rev/6ecc34b31137
changeset:   44340:6ecc34b31137
user:Pierre-Yves David 
date:Wed Jan 15 15:51:01 2020 +0100
summary: nodemap: update the index with the newly written data (when 
appropriate)

https://www.mercurial-scm.org/repo/hg/rev/77bb38be00ea
changeset:   44341:77bb38be00ea
user:Martin von Zweigbergk 
date:Thu Jan 16 00:03:19 2020 -0800
summary: rebase: always be graft-like, not merge-like, also for merges

https://www.mercurial-scm.org/repo/hg/rev/7f7c8521e9bd
changeset:   44342:7f7c8521e9bd
user:Martin von Zweigbergk 
date:Fri Jan 10 17:46:10 2020 -0800
summary: tests: add test of rebase with conflict in merge commit

https://www.mercurial-scm.org/repo/hg/rev/8561ad49915d
changeset:   44343:8561ad49915d
user:Martin von Zweigbergk 
date:Wed Jan 29 11:30:35 2020 -0800
summary: revset: add a revset for parents in merge state

https://www.mercurial-scm.org/repo/hg/rev/ab632e27f296
changeset:   44344:ab632e27f296
user:Martin von Zweigbergk 
date:Wed Jan 29 14:42:54 2020 -0800
summary: tests: add `hg log -G` output when there are merge conflicts

https://www.mercurial-scm.org/repo/hg/rev/14d0e89520a2
changeset:   44345:14d0e89520a2
user:Martin von Zweigbergk 
date:Tue Jan 28 21:49:50 2020 -0800
summary: graphlog: use '%' for other context in merge conflict

https://www.mercurial-scm.org/repo/hg/rev/b42ce825308e
changeset:   44346:b42ce825308e
user:Martin von Zweigbergk 
date:Fri Jan 10 14:17:56 2020 -0800
summary: rebase: stop relying on having two parents to resume rebase

https://www.mercurial-scm.org/repo/hg/rev/9c9cfecd4600
changeset:   44347:9c9cfecd4600
user:Martin von Zweigbergk 
date:Fri Jan 10 14:22:20 2020 -0800
summary: rebase: don't use rebased node as dirstate p2 (BC)

https://www.mercurial-scm.org/repo/hg/rev/8082a77cc3a2
changeset:   44348:8082a77cc3a2
user:Martin von Zweigbergk 
date:Fri Jan 10 21:40:01 2020 -0800
summary: rebase: remove some redundant setting of dirstate parents

https://www.mercurial-scm.org/repo/hg/rev/cd43cae79f25
changeset:   44349:cd43cae79f25
user:Martin von Zweigbergk 
date:Fri Jan 10 17:20:12 2020 -0800
summary: rebase: remove some now-unused parent arguments

https://www.mercurial-scm.org/repo/hg/rev/e80da7a63264
changeset:   44350:e80da7a63264
user:Augie Fackler 
date:Mon Feb 10 20:47:19 2020 -0500
summary: tags: fix some type confusion exposed in python 3

https://www.mercurial-scm.org/repo/hg/rev/8ec186c1ccfe
changeset:   44351:8ec186c1ccfe
user:Augie Fackler 
date:Mon Feb 10 21:02:22 2020 -0500
summary: tags: use modern // operator for division

https://www.mercurial-scm.org/repo/hg/rev/0bf3b5e80d30
changeset:   44352:0bf3b5e80d30
user:Augie Fackler 
date:Wed Feb 05 17:13:51 2020 -0500
summary: manifest: move matches method to be outside the interface

https://www.mercurial-scm.org/repo/hg/rev/54d185eb24b5
changeset:   44353:54d185eb24b5
user:Raphaël Gomès 
date:Thu Jan 16 23:06:01 2020 +0100
summary: rust-matchers: 

D7932: debugbackupbundle: introduce command to interact with strip backups

[pulkit (Pulkit Goyal)]

pulkit retitled this revision from "debugstripbackups: introduce command to 
interact with strip backups" to "debugbackupbundle: introduce command to 
interact with strip backups".
pulkit updated this revision to Diff 20288.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7932?vs=19868&id=20288

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7932/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7932

AFFECTED FILES
  mercurial/debugcommands.py
  tests/debugbackupbundle.t

CHANGE DETAILS

diff --git a/tests/debugbackupbundle.t b/tests/debugbackupbundle.t
new file mode 100644
--- /dev/null
+++ b/tests/debugbackupbundle.t
@@ -0,0 +1,39 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > strip=
+  > EOF
+
+Setup repo
+
+  $ hg init repo
+  $ cd repo
+
+Test backups list and recover
+
+  $ hg debugbackupbundle
+  no backup changesets found
+
+  $ mkcommit() {
+  >echo "$1" > "$1"
+  >hg add "$1"
+  >hg ci -l $1
+  > }
+  $ mkcommit a
+  $ mkcommit b
+  $ hg strip .
+  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
+  saved backup bundle to 
$TESTTMP/repo/.hg/strip-backup/d2ae7f538514-2953539b-backup.hg (glob)
+  $ hg debugbackupbundle
+  Recover changesets using: hg debugbackupbundle --recover 
+  
+  Available backup changesets:
+  * (glob)
+  d2ae7f538514 b
+
+  $ hg debugbackupbundle --recover d2ae7f538514
+  Unbundling d2ae7f538514
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  new changesets d2ae7f538514 (1 drafts)
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -11,6 +11,7 @@
 import collections
 import difflib
 import errno
+import glob
 import operator
 import os
 import platform
@@ -39,6 +40,7 @@
 )
 from . import (
 bundle2,
+bundlerepo,
 changegroup,
 cmdutil,
 color,
@@ -3423,6 +3425,138 @@
 
 
 @command(
+"debugbackupbundle",
+[
+(
+"",
+"recover",
+"",
+"brings the specified changeset back into the repository",
+)
+]
++ cmdutil.logopts,
+_("hg debugbackupbundle [--recover HASH]"),
+)
+def debugbackupbundle(ui, repo, *pats, **opts):
+"""lists the changesets available in backup bundles
+
+Without any arguments, this command prints a list of the changesets in each
+backup bundle.
+
+--recover takes a changeset hash and unbundles the first bundle that
+contains that hash, which puts that changeset back in your repository.
+
+--verbose will print the entire commit message and the bundle path for that
+backup.
+"""
+backups = filter(
+os.path.isfile, glob.glob(repo.vfs.join("strip-backup") + "/*.hg")
+)
+backups.sort(key=lambda x: os.path.getmtime(x), reverse=True)
+
+opts["bundle"] = ""
+opts["force"] = None
+limit = logcmdutil.getlimit(opts)
+
+def display(other, chlist, displayer):
+if opts.get("newest_first"):
+chlist.reverse()
+count = 0
+for n in chlist:
+if limit is not None and count >= limit:
+break
+parents = [True for p in other.changelog.parents(n) if p != nullid]
+if opts.get("no_merges") and len(parents) == 2:
+continue
+count += 1
+displayer.show(other[n])
+
+recovernode = opts.get("recover")
+if recovernode:
+if scmutil.isrevsymbol(repo, recovernode):
+ui.warn(_("%s already exists in the repo\n") % recovernode)
+return
+elif backups:
+msg = _(
+"Recover changesets using: hg debugbackupbundle --recover "
+"\n\nAvailable backup changesets:"
+)
+ui.status(msg, label="status.removed")
+else:
+ui.status(_("no backup changesets found\n"))
+return
+
+for backup in backups:
+# Much of this is copied from the hg incoming logic
+source = ui.expandpath(os.path.relpath(backup, encoding.getcwd()))
+source, branches = hg.parseurl(source, opts.get("branch"))
+try:
+other = hg.peer(repo, opts, source)
+except error.LookupError as ex:
+msg = _("\nwarning: unable to open bundle %s") % source
+hint = _("\n(missing parent rev %s)\n") % short(ex.name)
+ui.warn(msg, hint=hint)
+continue
+revs, checkout = hg.addbranchrevs(
+repo, other, branches, opts.get("rev")
+)
+
+if revs:
+revs = [other.lookup(rev) for rev in revs]
+
+quiet = ui.quiet
+try:
+ui.quiet = True
+other, chlist, cleanupfn = bundlerepo.getremotechanges(
+ui, repo, other, revs, opts["bundle"], opts["force"]
+)
+except error.LookupError:
+

D8134: remotefilelog: add 'changelog' arg to shallowcg1packer.generate (issue6269)

[pulkit (Pulkit Goyal)]

pulkit updated this revision to Diff 20287.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D8134?vs=20258&id=20287

BRANCH
  stable

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8134/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8134

AFFECTED FILES
  hgext/remotefilelog/shallowbundle.py

CHANGE DETAILS

diff --git a/hgext/remotefilelog/shallowbundle.py 
b/hgext/remotefilelog/shallowbundle.py
--- a/hgext/remotefilelog/shallowbundle.py
+++ b/hgext/remotefilelog/shallowbundle.py
@@ -54,12 +54,12 @@
 
 
 class shallowcg1packer(changegroup.cgpacker):
-def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
+def generate(self, commonrevs, clnodes, fastpathlinkrev, source, **kwargs):
 if shallowutil.isenabled(self._repo):
 fastpathlinkrev = False
 
 return super(shallowcg1packer, self).generate(
-commonrevs, clnodes, fastpathlinkrev, source
+commonrevs, clnodes, fastpathlinkrev, source, **kwargs
 )
 
 def group(self, nodelist, rlog, lookup, units=None, reorder=None):



To: pulkit, #hg-reviewers, marmoute
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 6276] New: Rust panic when using fsmonitor

[mercurial-bugs]

https://bz.mercurial-scm.org/show_bug.cgi?id=6276

Bug ID: 6276
   Summary: Rust panic when using fsmonitor
   Product: Mercurial
   Version: stable branch
  Hardware: All
OS: All
Status: UNCONFIRMED
  Severity: bug
  Priority: wish
 Component: fsmonitor
  Assignee: bugzi...@mercurial-scm.org
  Reporter: rgo...@octobus.net
CC: mercurial-devel@mercurial-scm.org
Python Version: ---

`fsmonitor` iterates over the non normal entries, which the Rust code did not
anticipate. 

I have sent a bugfix for `default` to https://phab.mercurial-scm.org/D8143, and
I would like to graft it on `stable`, however the fix uses the recently
upstreamed "shared references" mechanism that Yuya pushed on `default`. 

I will be sending out emails to figure out if anyone needs this fix sooner that
the next `default` release. If you are using Rust code with `fsmonitor`, please
let me know.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH STABLE] py3: fix EOL detection in commandserver.channeledinput

[Augie Fackler]

queued for stable, thanks

> On Feb 23, 2020, at 23:45, Yuya Nishihara  wrote:
> 
> # HG changeset patch
> # User Yuya Nishihara 
> # Date 1582518529 -32400
> #  Mon Feb 24 13:28:49 2020 +0900
> # Branch stable
> # Node ID c60d7f7a59e78ce9d170a6a411b900d127ca38c0
> # Parent  69b091cdc506c8fde642da6808c18b07f8889077
> py3: fix EOL detection in commandserver.channeledinput
> 
> This breaks TortoiseHg's email preview which sends b'\n' while readline
> request is issued and the loop never ends. Spotted by Matt Harbison.
> 
> diff --git a/mercurial/commandserver.py b/mercurial/commandserver.py
> --- a/mercurial/commandserver.py
> +++ b/mercurial/commandserver.py
> @@ -160,7 +160,7 @@ class channeledinput(object):
> buf = s
> # keep asking for more until there's either no more or
> # we got a full line
> -while s and s[-1] != b'\n':
> +while s and not s.endswith(b'\n'):
> s = self._read(size, b'L')
> buf += s
> 
> diff --git a/tests/test-commandserver.t b/tests/test-commandserver.t
> --- a/tests/test-commandserver.t
> +++ b/tests/test-commandserver.t
> @@ -654,6 +654,9 @@ changelog and manifest would have invali
>   ... runcommand(server, [b'debugprompt', b'--config',
>   ... b'ui.interactive=True'],
>   ...input=stringio(b'5678\n'))
> +  ... runcommand(server, [b'debugprompt', b'--config',
> +  ... b'ui.interactive=True'],
> +  ...input=stringio(b'\nremainder\nshould\nnot\nbe\nread\n'))
>   ... runcommand(server, [b'debugreadstdin'])
>   ... runcommand(server, [b'debugwritestdout'])
>   *** runcommand debuggetpass --config ui.interactive=True
> @@ -665,6 +668,8 @@ changelog and manifest would have invali
>[255]
>   *** runcommand debugprompt --config ui.interactive=True
>   prompt: 5678
> +  *** runcommand debugprompt --config ui.interactive=True
> +  prompt: y
>   *** runcommand debugreadstdin
>   read: ''
>   *** runcommand debugwritestdout
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8143: rust-cpython: make `NonNormalEntires` an iterator to fix bug with `fsmonitor`

[Raphaël Gomès]

Alphare created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This fixes a bug when using `fsmonitor` that tries to iterate on the non 
normal
  set, by adding a shared iterator interface.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D8143

AFFECTED FILES
  rust/hg-core/src/dirstate/dirstate_map.rs
  rust/hg-cpython/src/dirstate/dirstate_map.rs
  rust/hg-cpython/src/dirstate/non_normal_entries.rs

CHANGE DETAILS

diff --git a/rust/hg-cpython/src/dirstate/non_normal_entries.rs 
b/rust/hg-cpython/src/dirstate/non_normal_entries.rs
--- a/rust/hg-cpython/src/dirstate/non_normal_entries.rs
+++ b/rust/hg-cpython/src/dirstate/non_normal_entries.rs
@@ -6,11 +6,15 @@
 // GNU General Public License version 2 or any later version.
 
 use cpython::{
-exc::NotImplementedError, CompareOp, ObjectProtocol, PyErr, PyList,
-PyObject, PyResult, PyString, Python, PythonObject, ToPyObject,
+exc::NotImplementedError, CompareOp, ObjectProtocol, PyBytes, PyClone,
+PyErr, PyList, PyObject, PyResult, PyString, Python, PythonObject,
+ToPyObject, UnsafePyLeaked,
 };
 
 use crate::dirstate::DirstateMap;
+use hg::utils::hg_path::HgPathBuf;
+use std::cell::RefCell;
+use std::collections::hash_set;
 
 py_class!(pub class NonNormalEntries |py| {
 data dmap: DirstateMap;
@@ -34,6 +38,10 @@
 def __repr__(&self) -> PyResult {
 self.dmap(py).non_normal_entries_display(py)
 }
+
+def __iter__(&self) -> PyResult {
+self.dmap(py).non_normal_entries_iter(py)
+}
 });
 
 impl NonNormalEntries {
@@ -49,4 +57,20 @@
 }
 Ok(true)
 }
+
+fn translate_key(
+py: Python,
+key: &HgPathBuf,
+) -> PyResult> {
+Ok(Some(PyBytes::new(py, key.as_ref(
+}
 }
+
+type NonNormalEntriesIter<'a> = hash_set::Iter<'a, HgPathBuf>;
+
+py_shared_iterator!(
+NonNormalEntriesIterator,
+UnsafePyLeaked>,
+NonNormalEntries::translate_key,
+Option
+);
diff --git a/rust/hg-cpython/src/dirstate/dirstate_map.rs 
b/rust/hg-cpython/src/dirstate/dirstate_map.rs
--- a/rust/hg-cpython/src/dirstate/dirstate_map.rs
+++ b/rust/hg-cpython/src/dirstate/dirstate_map.rs
@@ -20,7 +20,9 @@
 
 use crate::{
 dirstate::copymap::{CopyMap, CopyMapItemsIterator, CopyMapKeysIterator},
-dirstate::non_normal_entries::NonNormalEntries,
+dirstate::non_normal_entries::{
+NonNormalEntries, NonNormalEntriesIterator,
+},
 dirstate::{dirs_multiset::Dirs, make_dirstate_tuple},
 };
 use hg::{
@@ -244,6 +246,22 @@
 Ok(ret)
 }
 
+def non_normal_entries_iter(&self) -> PyResult {
+// Make sure the sets are defined before we no longer have a mutable
+// reference to the dmap.
+self.inner(py)
+.borrow_mut()
+.set_non_normal_other_parent_entries(false);
+
+let leaked_ref = self.inner(py).leak_immutable();
+
+NonNormalEntriesIterator::from_inner(py, unsafe {
+leaked_ref.map(py, |o| {
+o.get_non_normal_other_parent_entries_panic().0.iter()
+})
+})
+}
+
 def hastrackeddir(&self, d: PyObject) -> PyResult {
 let d = d.extract::(py)?;
 Ok(self.inner(py).borrow_mut()
diff --git a/rust/hg-core/src/dirstate/dirstate_map.rs 
b/rust/hg-core/src/dirstate/dirstate_map.rs
--- a/rust/hg-core/src/dirstate/dirstate_map.rs
+++ b/rust/hg-core/src/dirstate/dirstate_map.rs
@@ -237,6 +237,25 @@
 )
 }
 
+/// Useful to get immutable references to those sets in contexts where
+/// you only have an immutable reference to the `DirstateMap`, like when
+/// sharing references with Python.
+///
+/// TODO, get rid of this along with the other "setter/getter" stuff when
+/// a nice typestate plan is defined.
+///
+/// # Panics
+///
+/// Will panic if either set is `None`.
+pub fn get_non_normal_other_parent_entries_panic(
+&self,
+) -> (&HashSet, &HashSet) {
+(
+self.non_normal_set.as_ref().unwrap(),
+self.other_parent_set.as_ref().unwrap(),
+)
+}
+
 pub fn set_non_normal_other_parent_entries(&mut self, force: bool) {
 if !force
 && self.non_normal_set.is_some()



To: Alphare, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8117: bookmarks: prevent pushes of divergent bookmarks (foo@remote)

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  I am also +1 for this change. divergence bookmark are close to "remote" 
bookmark and should not be exchanged. Especially because it open the way to 
divergent-divergent-bookmark and other hells.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8117/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8117

To: valentin.gatienbaron, #hg-reviewers, pulkit
Cc: marmoute, durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8117: bookmarks: prevent pushes of divergent bookmarks (foo@remote)

[durin42 (Augie Fackler)]

durin42 added a comment.


  I'm enthusiastic about this change, and in general a fan of declaring 
misfeatures in bookmarks exchange bugs. I think it's pretty clear the existing 
bookmarks exchange behavior isn't right to the point of being buggy.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8117/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8117

To: valentin.gatienbaron, #hg-reviewers, pulkit
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8142: setup: exclude the __index__ module from itself when generating

[marmoute (Pierre-Yves David)]

marmoute added a comment.
marmoute accepted this revision.


  This looks good to me.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8142/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8142

To: mharbison72, #hg-reviewers, marmoute
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8134: remotefilelog: add 'changelog' arg to shallowcg1packer.generate (issue6269)

[marmoute (Pierre-Yves David)]

This revision now requires changes to proceed.
marmoute added inline comments.
marmoute requested changes to this revision.

INLINE COMMENTS

> shallowbundle.py:63
>  return super(shallowcg1packer, self).generate(
> -commonrevs, clnodes, fastpathlinkrev, source
> +commonrevs, clnodes, fastpathlinkrev, source, changelog
>  )

Since this is a keyword argument, this seems safer to pass it to the superclass 
as a keyword argument too. This will avoid sneaky issue if more argument are 
added.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8134/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8134

To: pulkit, #hg-reviewers, marmoute
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8127: graft: always allow hg graft --base . (issue6248)

[marmoute (Pierre-Yves David)]

marmoute added a comment.
marmoute accepted this revision.


  The usecase looks correct, the test change seems correct and the test is 
good. I can't think of any issue here.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8127/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8127

To: valentin.gatienbaron, #hg-reviewers, marmoute
Cc: marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7819: rust-nodemap: core implementation for shortest

[Raphaël Gomès]

Alphare updated this revision to Diff 20283.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7819?vs=20278&id=20283

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7819/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7819

AFFECTED FILES
  rust/hg-core/src/revlog/node.rs
  rust/hg-core/src/revlog/nodemap.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/revlog/nodemap.rs 
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -17,6 +17,7 @@
 RevlogIndex, NULL_REVISION,
 };
 
+use std::cmp::max;
 use std::fmt;
 use std::mem;
 use std::ops::Deref;
@@ -98,6 +99,42 @@
 ) -> Result, NodeMapError> {
 self.find_bin(idx, NodePrefix::from_hex(prefix)?.borrow())
 }
+
+/// Give the size of the shortest node prefix that determines
+/// the revision uniquely.
+///
+/// From a binary node prefix, if it is matched in the node map, this
+/// returns the number of hexadecimal digits that would had sufficed
+/// to find the revision uniquely.
+///
+/// Returns `None` if no `Revision` could be found for the prefix.
+///
+/// If several Revisions match the given prefix, a [`MultipleResults`]
+/// error is returned.
+fn unique_prefix_len_bin<'a>(
+&self,
+idx: &impl RevlogIndex,
+node_prefix: NodePrefixRef<'a>,
+) -> Result, NodeMapError>;
+
+/// Same as `unique_prefix_len_bin`, with the hexadecimal representation
+/// of the prefix as input.
+fn unique_prefix_len_hex(
+&self,
+idx: &impl RevlogIndex,
+prefix: &str,
+) -> Result, NodeMapError> {
+self.unique_prefix_len_bin(idx, NodePrefix::from_hex(prefix)?.borrow())
+}
+
+/// Same as `unique_prefix_len_bin`, with a full `Node` as input
+fn unique_prefix_len_node(
+&self,
+idx: &impl RevlogIndex,
+node: &Node,
+) -> Result, NodeMapError> {
+self.unique_prefix_len_bin(idx, node.into())
+}
 }
 
 pub trait MutableNodeMap: NodeMap {
@@ -279,20 +316,24 @@
 fn validate_candidate(
 idx: &impl RevlogIndex,
 prefix: NodePrefixRef,
-rev: Option,
-) -> Result, NodeMapError> {
-if prefix.is_prefix_of(&NULL_NODE) {
-// NULL_REVISION always matches a prefix made only of zeros
+candidate: (Option, usize),
+) -> Result<(Option, usize), NodeMapError> {
+let (rev, steps) = candidate;
+if let Some(nz_nybble) = prefix.first_different_nybble(&NULL_NODE) {
+rev.map_or(Ok((None, steps)), |r| {
+has_prefix_or_none(idx, prefix, r)
+.map(|opt| (opt, max(steps, nz_nybble + 1)))
+})
+} else {
+// the prefix is only made of zeros; NULL_REVISION always matches it
 // and any other *valid* result is an ambiguity
 match rev {
-None => Ok(Some(NULL_REVISION)),
+None => Ok((Some(NULL_REVISION), steps + 1)),
 Some(r) => match has_prefix_or_none(idx, prefix, r)? {
-None => Ok(Some(NULL_REVISION)),
+None => Ok((Some(NULL_REVISION), steps + 1)),
 _ => Err(NodeMapError::MultipleResults),
 },
 }
-} else {
-rev.map_or(Ok(None), |r| has_prefix_or_none(idx, prefix, r))
 }
 }
 
@@ -387,13 +428,26 @@
 }
 
 /// Main working method for `NodeTree` searches
-fn lookup<'p>(
+///
+/// The first returned value is the result of analysing `NodeTree` data
+/// *alone*: whereas `None` guarantees that the given prefix is absent
+/// from the `NodeTree` data (but still could match `NULL_NODE`), with
+/// `Some(rev)`, it is to be understood that `rev` is the unique `Revision`
+/// that could match the prefix. Actually, all that can be inferred from
+/// the `NodeTree` data is that `rev` is the revision with the longest
+/// common node prefix with the given prefix.
+///
+/// The second returned value is the size of the smallest subprefix
+/// of `prefix` that would give the same result, i.e. not the
+/// `MultipleResults` error variant (again, using only the data of the
+/// `NodeTree`).
+fn lookup(
 &self,
-prefix: NodePrefixRef<'p>,
-) -> Result, NodeMapError> {
-for visit_item in self.visit(prefix) {
+prefix: NodePrefixRef,
+) -> Result<(Option, usize), NodeMapError> {
+for (i, visit_item) in self.visit(prefix).enumerate() {
 if let Some(opt) = visit_item.final_revision() {
-return Ok(opt);
+return Ok((opt, i + 1));
 }
 }
 Err(NodeMapError::MultipleResults)
@@ -638,6 +692,16 @@
 prefix: NodePrefixRef<'a>,
 ) -> Result, NodeMapError> {
 validate_candidate(idx, prefix.clone(), self.lookup(prefix)?)
+.map(|(opt, _shortest)| opt)
+}
+
+fn unique_prefix

D8098: rust-nodemap: a method for full invalidation

[Raphaël Gomès]

Alphare updated this revision to Diff 20285.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D8098?vs=20280&id=20285

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8098/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8098

AFFECTED FILES
  rust/hg-core/src/revlog/nodemap.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/revlog/nodemap.rs 
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -575,11 +575,25 @@
 Ok(())
 }
 
+/// Make the whole `NodeTree` logically empty, without touching the
+/// immutable part.
+pub fn invalidate_all(&mut self) {
+self.root = Block::new();
+self.growable = Vec::new();
+self.masked_inner_blocks = self.readonly.len();
+}
+
 /// Return the number of blocks in the readonly part that are currently
 /// masked in the mutable part.
 ///
 /// The `NodeTree` structure has no efficient way to know how many blocks
 /// are already unreachable in the readonly part.
+///
+/// After a call to `invalidate_all()`, the returned number can be actually
+/// bigger than the whole readonly part, a conventional way to mean that
+/// all the readonly blocks have been masked. This is what is really
+/// useful to the caller and does not require to know how many were
+/// actually unreachable to begin with.
 pub fn masked_readonly_blocks(&self) -> usize {
 if let Some(readonly_root) = self.readonly.last() {
 if readonly_root == &self.root {
@@ -1060,6 +1074,27 @@
 }
 
 #[test]
+fn test_invalidate_all() -> Result<(), NodeMapError> {
+let mut idx = TestNtIndex::new();
+idx.insert(0, "1234")?;
+idx.insert(1, "1235")?;
+idx.insert(2, "131")?;
+idx.insert(3, "cafe")?;
+let mut idx = idx.commit();
+
+idx.nt.invalidate_all();
+
+assert_eq!(idx.find_hex("1234")?, None);
+assert_eq!(idx.find_hex("1235")?, None);
+assert_eq!(idx.find_hex("131")?, None);
+assert_eq!(idx.find_hex("cafe")?, None);
+// all the readonly blocks have been masked, this is the
+// conventional expected response
+assert_eq!(idx.nt.masked_readonly_blocks(), idx.nt.readonly.len() + 1);
+Ok(())
+}
+
+#[test]
 fn test_into_added_empty() {
 assert!(sample_nodetree().into_readonly_and_added().1.is_empty());
 assert!(sample_nodetree()



To: marmoute, #hg-reviewers, kevincox
Cc: kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8097: rust-nodemap: accounting for dead blocks

[Raphaël Gomès]

Alphare updated this revision to Diff 20284.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D8097?vs=20279&id=20284

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8097/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8097

AFFECTED FILES
  rust/hg-core/src/revlog/nodemap.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/revlog/nodemap.rs 
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -274,6 +274,7 @@
 readonly: Box + Send>,
 growable: Vec,
 root: Block,
+masked_inner_blocks: usize,
 }
 
 impl Index for NodeTree {
@@ -350,6 +351,7 @@
 readonly: readonly,
 growable: Vec::new(),
 root: root,
+masked_inner_blocks: 0,
 }
 }
 
@@ -483,6 +485,7 @@
 let ro_len = ro_blocks.len();
 let glen = self.growable.len();
 if idx < ro_len {
+self.masked_inner_blocks += 1;
 // TODO OPTIM I think this makes two copies
 self.growable.push(ro_blocks[idx].clone());
 (glen + ro_len, &mut self.growable[glen], glen + 1)
@@ -571,6 +574,22 @@
 }
 Ok(())
 }
+
+/// Return the number of blocks in the readonly part that are currently
+/// masked in the mutable part.
+///
+/// The `NodeTree` structure has no efficient way to know how many blocks
+/// are already unreachable in the readonly part.
+pub fn masked_readonly_blocks(&self) -> usize {
+if let Some(readonly_root) = self.readonly.last() {
+if readonly_root == &self.root {
+return 0;
+}
+} else {
+return 0;
+}
+self.masked_inner_blocks + 1
+}
 }
 
 pub struct NodeTreeBytes {
@@ -853,6 +872,7 @@
 readonly: sample_nodetree().readonly,
 growable: vec![block![0: Rev(1), 5: Rev(3)]],
 root: block![0: Block(1), 1:Block(3), 12: Rev(2)],
+masked_inner_blocks: 1,
 };
 assert_eq!(nt.find_hex(&idx, "10")?, Some(1));
 assert_eq!(nt.find_hex(&idx, "c")?, Some(2));
@@ -861,6 +881,7 @@
 assert_eq!(nt.find_hex(&idx, "000")?, Some(NULL_REVISION));
 assert_eq!(nt.unique_prefix_len_hex(&idx, "000")?, Some(3));
 assert_eq!(nt.find_hex(&idx, "01")?, Some(9));
+assert_eq!(nt.masked_readonly_blocks(), 2);
 Ok(())
 }
 
@@ -950,6 +971,8 @@
 assert_eq!(idx.find_hex("1a345")?, Some(3));
 assert_eq!(idx.find_hex("1a341")?, None);
 
+// there's no readonly block to mask
+assert_eq!(idx.nt.masked_readonly_blocks(), 0);
 Ok(())
 }
 
@@ -1011,6 +1034,8 @@
 assert_eq!(idx.find_hex("1235")?, Some(1));
 assert_eq!(idx.find_hex("131")?, Some(2));
 assert_eq!(idx.find_hex("cafe")?, Some(3));
+// we did not add anything since init from readonly
+assert_eq!(idx.nt.masked_readonly_blocks(), 0);
 
 idx.insert(4, "123A")?;
 assert_eq!(idx.find_hex("1234")?, Some(0));
@@ -1018,12 +1043,18 @@
 assert_eq!(idx.find_hex("131")?, Some(2));
 assert_eq!(idx.find_hex("cafe")?, Some(3));
 assert_eq!(idx.find_hex("123A")?, Some(4));
+// we masked blocks for all prefixes of "123", including the root
+assert_eq!(idx.nt.masked_readonly_blocks(), 4);
 
+eprintln!("{:?}", idx.nt);
 idx.insert(5, "c0")?;
 assert_eq!(idx.find_hex("cafe")?, Some(3));
 assert_eq!(idx.find_hex("c0")?, Some(5));
 assert_eq!(idx.find_hex("c1")?, None);
 assert_eq!(idx.find_hex("1234")?, Some(0));
+// inserting "c0" is just splitting the 'c' slot of the mutable root,
+// it doesn't mask anything
+assert_eq!(idx.nt.masked_readonly_blocks(), 4);
 
 Ok(())
 }



To: marmoute, #hg-reviewers, kevincox
Cc: kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7798: rust-nodemap: special case for prefixes of NULL_NODE

[Raphaël Gomès]

Alphare updated this revision to Diff 20282.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7798?vs=20236&id=20282

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7798/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7798

AFFECTED FILES
  rust/hg-core/src/revlog/nodemap.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/revlog/nodemap.rs 
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -13,7 +13,8 @@
 //! is used in a more abstract context.
 
 use super::{
-Node, NodeError, NodePrefix, NodePrefixRef, Revision, RevlogIndex,
+node::NULL_NODE, Node, NodeError, NodePrefix, NodePrefixRef, Revision,
+RevlogIndex, NULL_REVISION,
 };
 
 use std::fmt;
@@ -270,6 +271,31 @@
 })
 }
 
+/// validate that the candidate's node starts indeed with given prefix,
+/// and treat ambiguities related to `NULL_REVISION`.
+///
+/// From the data in the NodeTree, one can only conclude that some
+/// revision is the only one for a *subprefix* of the one being looked up.
+fn validate_candidate(
+idx: &impl RevlogIndex,
+prefix: NodePrefixRef,
+rev: Option,
+) -> Result, NodeMapError> {
+if prefix.is_prefix_of(&NULL_NODE) {
+// NULL_REVISION always matches a prefix made only of zeros
+// and any other *valid* result is an ambiguity
+match rev {
+None => Ok(Some(NULL_REVISION)),
+Some(r) => match has_prefix_or_none(idx, prefix, r)? {
+None => Ok(Some(NULL_REVISION)),
+_ => Err(NodeMapError::MultipleResults),
+},
+}
+} else {
+rev.map_or(Ok(None), |r| has_prefix_or_none(idx, prefix, r))
+}
+}
+
 impl NodeTree {
 /// Initiate a NodeTree from an immutable slice-like of `Block`
 ///
@@ -361,8 +387,6 @@
 }
 
 /// Main working method for `NodeTree` searches
-///
-/// This partial implementation lacks special cases for NULL_REVISION
 fn lookup<'p>(
 &self,
 prefix: NodePrefixRef<'p>,
@@ -613,9 +637,7 @@
 idx: &impl RevlogIndex,
 prefix: NodePrefixRef<'a>,
 ) -> Result, NodeMapError> {
-self.lookup(prefix.clone()).and_then(|opt| {
-opt.map_or(Ok(None), |rev| has_prefix_or_none(idx, prefix, rev))
-})
+validate_candidate(idx, prefix.clone(), self.lookup(prefix)?)
 }
 }
 
@@ -748,8 +770,9 @@
 
 assert_eq!(nt.find_hex(&idx, "0"), Err(MultipleResults));
 assert_eq!(nt.find_hex(&idx, "01"), Ok(Some(9)));
-assert_eq!(nt.find_hex(&idx, "00"), Ok(Some(0)));
+assert_eq!(nt.find_hex(&idx, "00"), Err(MultipleResults));
 assert_eq!(nt.find_hex(&idx, "00a"), Ok(Some(0)));
+assert_eq!(nt.find_hex(&idx, "000"), Ok(Some(NULL_REVISION)));
 }
 
 #[test]
@@ -768,7 +791,8 @@
 };
 assert_eq!(nt.find_hex(&idx, "10")?, Some(1));
 assert_eq!(nt.find_hex(&idx, "c")?, Some(2));
-assert_eq!(nt.find_hex(&idx, "00")?, Some(0));
+assert_eq!(nt.find_hex(&idx, "00"), Err(MultipleResults));
+assert_eq!(nt.find_hex(&idx, "000")?, Some(NULL_REVISION));
 assert_eq!(nt.find_hex(&idx, "01")?, Some(9));
 Ok(())
 }



To: gracinet, #hg-reviewers, kevincox, marmoute
Cc: marmoute, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7796: rust-nodemap: input/output primitives

[Raphaël Gomès]

Alphare updated this revision to Diff 20281.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7796?vs=20235&id=20281

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7796/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7796

AFFECTED FILES
  rust/hg-core/src/revlog/nodemap.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/revlog/nodemap.rs 
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -17,8 +17,10 @@
 };
 
 use std::fmt;
+use std::mem;
 use std::ops::Deref;
 use std::ops::Index;
+use std::slice;
 
 #[derive(Debug, PartialEq)]
 pub enum NodeMapError {
@@ -172,20 +174,42 @@
 /// represented at all, because we want an immutable empty nodetree
 /// to be valid.
 
-#[derive(Clone, PartialEq)]
-pub struct Block([RawElement; 16]);
+#[derive(Copy, Clone)]
+pub struct Block([u8; BLOCK_SIZE]);
+
+/// Not derivable for arrays of length >32 until const generics are stable
+impl PartialEq for Block {
+fn eq(&self, other: &Self) -> bool {
+&self.0[..] == &other.0[..]
+}
+}
+
+pub const BLOCK_SIZE: usize = 64;
 
 impl Block {
 fn new() -> Self {
-Block([-1; 16])
+// -1 in 2's complement to create an absent node
+let byte: u8 = 255;
+Block([byte; BLOCK_SIZE])
 }
 
 fn get(&self, nybble: u8) -> Element {
-Element::from(RawElement::from_be(self.0[nybble as usize]))
+let index = nybble as usize * mem::size_of::();
+Element::from(RawElement::from_be_bytes([
+self.0[index],
+self.0[index + 1],
+self.0[index + 2],
+self.0[index + 3],
+]))
 }
 
 fn set(&mut self, nybble: u8, element: Element) {
-self.0[nybble as usize] = RawElement::to_be(element.into())
+let values = RawElement::to_be_bytes(element.into());
+let index = nybble as usize * mem::size_of::();
+self.0[index] = values[0];
+self.0[index + 1] = values[1];
+self.0[index + 2] = values[2];
+self.0[index + 3] = values[3];
 }
 }
 
@@ -230,9 +254,9 @@
 }
 
 /// Return `None` unless the `Node` for `rev` has given prefix in `index`.
-fn has_prefix_or_none<'p>(
+fn has_prefix_or_none(
 idx: &impl RevlogIndex,
-prefix: NodePrefixRef<'p>,
+prefix: NodePrefixRef,
 rev: Revision,
 ) -> Result, NodeMapError> {
 idx.node(rev)
@@ -262,6 +286,67 @@
 }
 }
 
+/// Create from an opaque bunch of bytes
+///
+/// The created `NodeTreeBytes` from `buffer`,
+/// of which exactly `amount` bytes are used.
+///
+/// - `buffer` could be derived from `PyBuffer` and `Mmap` objects.
+/// - `offset` allows for the final file format to include fixed data
+///   (generation number, behavioural flags)
+/// - `amount` is expressed in bytes, and is not automatically derived from
+///   `bytes`, so that a caller that manages them atomically can perform
+///   temporary disk serializations and still rollback easily if needed.
+///   First use-case for this would be to support Mercurial shell hooks.
+///
+/// panics if `buffer` is smaller than `amount`
+pub fn load_bytes(
+bytes: Box + Send>,
+amount: usize,
+) -> Self {
+NodeTree::new(Box::new(NodeTreeBytes::new(bytes, amount)))
+}
+
+/// Retrieve added `Block` and the original immutable data
+pub fn into_readonly_and_added(
+self,
+) -> (Box + Send>, Vec) {
+let mut vec = self.growable;
+let readonly = self.readonly;
+if readonly.last() != Some(&self.root) {
+vec.push(self.root);
+}
+(readonly, vec)
+}
+
+/// Retrieve added `Blocks` as bytes, ready to be written to persistent
+/// storage
+pub fn into_readonly_and_added_bytes(
+self,
+) -> (Box + Send>, Vec) {
+let (readonly, vec) = self.into_readonly_and_added();
+// Prevent running `v`'s destructor so we are in complete control
+// of the allocation.
+let vec = mem::ManuallyDrop::new(vec);
+
+// Transmute the `Vec` to a `Vec`. Blocks are contiguous
+// bytes, so this is perfectly safe.
+let bytes = unsafe {
+// Assert that `Block` hasn't been changed and has no padding
+let _: [u8; 4 * BLOCK_SIZE] =
+std::mem::transmute([Block::new(); 4]);
+
+// /!\ Any use of `vec` after this is use-after-free.
+// TODO: use `into_raw_parts` once stabilized
+Vec::from_raw_parts(
+vec.as_ptr() as *mut u8,
+vec.len() * BLOCK_SIZE,
+vec.capacity() * BLOCK_SIZE,
+)
+};
+(readonly, bytes)
+}
+
 /// Total number of blocks
 fn len(&self) -> usize {
 self.readonly.len() + self.growable.len() + 1
@@ -410,6 +495,38 @@
 }
 }
 
+

D8098: rust-nodemap: a method for full invalidation

[kevincox (Kevin Cox)]

kevincox added inline comments.
kevincox accepted this revision.

INLINE COMMENTS

> nodemap.rs:577
> +/// immutable part.
> +pub fn invalidate_all(&mut self) -> () {
> +self.root = Block::new();

`-> ()` is unnecessary.

> nodemap.rs:580
> +self.growable = Vec::new();
> +self.masked_inner_blocks = self.readonly.len()
> +}

End the line with a `;`.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8098/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8098

To: marmoute, #hg-reviewers, kevincox
Cc: kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7796: rust-nodemap: input/output primitives

[kevincox (Kevin Cox)]

kevincox added inline comments.
kevincox accepted this revision.

INLINE COMMENTS

> nodemap.rs:191
>  fn new() -> Self {
> -Block([-1; 16])
> +Block([255u8; BLOCK_SIZE])
>  }

A comment why 255 is a good number would be nice.

> nodemap.rs:334
> +// Assert that `Block` hasn't been changed and has no padding
> +let _: [u8; 4 * mem::size_of::()] =
> +std::mem::transmute([Block::new(); 4]);

This should be `4 * BLOCK_SIZE` to match the code below.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7796/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7796

To: gracinet, #hg-reviewers, kevincox, durin42
Cc: yuja, Alphare, marmoute, durin42, kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7819: rust-nodemap: core implementation for shortest

[Raphaël Gomès]

Alphare updated this revision to Diff 20278.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7819?vs=20237&id=20278

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7819/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7819

AFFECTED FILES
  rust/hg-core/src/revlog/node.rs
  rust/hg-core/src/revlog/nodemap.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/revlog/nodemap.rs 
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -17,6 +17,7 @@
 RevlogIndex, NULL_REVISION,
 };
 
+use std::cmp::max;
 use std::fmt;
 use std::mem;
 use std::ops::Deref;
@@ -98,6 +99,42 @@
 ) -> Result, NodeMapError> {
 self.find_bin(idx, NodePrefix::from_hex(prefix)?.borrow())
 }
+
+/// Give the size of the shortest node prefix that determines
+/// the revision uniquely.
+///
+/// From a binary node prefix, if it is matched in the node map, this
+/// returns the number of hexadecimal digits that would had sufficed
+/// to find the revision uniquely.
+///
+/// Returns `None` if no `Revision` could be found for the prefix.
+///
+/// If several Revisions match the given prefix, a [`MultipleResults`]
+/// error is returned.
+fn unique_prefix_len_bin<'a>(
+&self,
+idx: &impl RevlogIndex,
+node_prefix: NodePrefixRef<'a>,
+) -> Result, NodeMapError>;
+
+/// Same as `unique_prefix_len_bin`, with the hexadecimal representation
+/// of the prefix as input.
+fn unique_prefix_len_hex(
+&self,
+idx: &impl RevlogIndex,
+prefix: &str,
+) -> Result, NodeMapError> {
+self.unique_prefix_len_bin(idx, NodePrefix::from_hex(prefix)?.borrow())
+}
+
+/// Same as `unique_prefix_len_bin`, with a full `Node` as input
+fn unique_prefix_len_node(
+&self,
+idx: &impl RevlogIndex,
+node: &Node,
+) -> Result, NodeMapError> {
+self.unique_prefix_len_bin(idx, node.into())
+}
 }
 
 pub trait MutableNodeMap: NodeMap {
@@ -277,20 +314,24 @@
 fn validate_candidate(
 idx: &impl RevlogIndex,
 prefix: NodePrefixRef,
-rev: Option,
-) -> Result, NodeMapError> {
-if prefix.is_prefix_of(&NULL_NODE) {
-// NULL_REVISION always matches a prefix made only of zeros
+candidate: (Option, usize),
+) -> Result<(Option, usize), NodeMapError> {
+let (rev, steps) = candidate;
+if let Some(nz_nybble) = prefix.first_different_nybble(&NULL_NODE) {
+rev.map_or(Ok((None, steps)), |r| {
+has_prefix_or_none(idx, prefix, r)
+.map(|opt| (opt, max(steps, nz_nybble + 1)))
+})
+} else {
+// the prefix is only made of zeros; NULL_REVISION always matches it
 // and any other *valid* result is an ambiguity
 match rev {
-None => Ok(Some(NULL_REVISION)),
+None => Ok((Some(NULL_REVISION), steps + 1)),
 Some(r) => match has_prefix_or_none(idx, prefix, r)? {
-None => Ok(Some(NULL_REVISION)),
+None => Ok((Some(NULL_REVISION), steps + 1)),
 _ => Err(NodeMapError::MultipleResults),
 },
 }
-} else {
-rev.map_or(Ok(None), |r| has_prefix_or_none(idx, prefix, r))
 }
 }
 
@@ -384,13 +425,26 @@
 }
 
 /// Main working method for `NodeTree` searches
-fn lookup<'p>(
+///
+/// The first returned value is the result of analysing `NodeTree` data
+/// *alone*: whereas `None` guarantees that the given prefix is absent
+/// from the `NodeTree` data (but still could match `NULL_NODE`), with
+/// `Some(rev)`, it is to be understood that `rev` is the unique `Revision`
+/// that could match the prefix. Actually, all that can be inferred from
+/// the `NodeTree` data is that `rev` is the revision with the longest
+/// common node prefix with the given prefix.
+///
+/// The second returned value is the size of the smallest subprefix
+/// of `prefix` that would give the same result, i.e. not the
+/// `MultipleResults` error variant (again, using only the data of the
+/// `NodeTree`).
+fn lookup(
 &self,
-prefix: NodePrefixRef<'p>,
-) -> Result, NodeMapError> {
-for visit_item in self.visit(prefix) {
+prefix: NodePrefixRef,
+) -> Result<(Option, usize), NodeMapError> {
+for (i, visit_item) in self.visit(prefix).enumerate() {
 if let Some(opt) = visit_item.final_revision() {
-return Ok(opt);
+return Ok((opt, i + 1));
 }
 }
 Err(NodeMapError::MultipleResults)
@@ -635,6 +689,16 @@
 prefix: NodePrefixRef<'a>,
 ) -> Result, NodeMapError> {
 validate_candidate(idx, prefix.clone(), self.lookup(prefix)?)
+.map(|(opt, _shortest)| opt)
+}
+
+fn unique_prefix

D8098: rust-nodemap: a method for full invalidation

[Raphaël Gomès]

Alphare updated this revision to Diff 20280.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D8098?vs=20239&id=20280

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8098/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8098

AFFECTED FILES
  rust/hg-core/src/revlog/nodemap.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/revlog/nodemap.rs 
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -572,11 +572,25 @@
 Ok(())
 }
 
+/// Make the whole `NodeTree` logically empty, without touching the
+/// immutable part.
+pub fn invalidate_all(&mut self) -> () {
+self.root = Block::new();
+self.growable = Vec::new();
+self.masked_inner_blocks = self.readonly.len()
+}
+
 /// Return the number of blocks in the readonly part that are currently
 /// masked in the mutable part.
 ///
 /// The `NodeTree` structure has no efficient way to know how many blocks
 /// are already unreachable in the readonly part.
+///
+/// After a call to `invalidate_all()`, the returned number can be actually
+/// bigger than the whole readonly part, a conventional way to mean that
+/// all the readonly blocks have been masked. This is what is really
+/// useful to the caller and does not require to know how many were
+/// actually unreachable to begin with.
 pub fn masked_readonly_blocks(&self) -> usize {
 if let Some(readonly_root) = self.readonly.last() {
 if readonly_root == &self.root {
@@ -1057,6 +1071,27 @@
 }
 
 #[test]
+fn test_invalidate_all() -> Result<(), NodeMapError> {
+let mut idx = TestNtIndex::new();
+idx.insert(0, "1234")?;
+idx.insert(1, "1235")?;
+idx.insert(2, "131")?;
+idx.insert(3, "cafe")?;
+let mut idx = idx.commit();
+
+idx.nt.invalidate_all();
+
+assert_eq!(idx.find_hex("1234")?, None);
+assert_eq!(idx.find_hex("1235")?, None);
+assert_eq!(idx.find_hex("131")?, None);
+assert_eq!(idx.find_hex("cafe")?, None);
+// all the readonly blocks have been masked, this is the
+// conventional expected response
+assert_eq!(idx.nt.masked_readonly_blocks(), idx.nt.readonly.len() + 1);
+Ok(())
+}
+
+#[test]
 fn test_into_added_empty() {
 assert!(sample_nodetree().into_readonly_and_added().1.is_empty());
 assert!(sample_nodetree()



To: marmoute, #hg-reviewers
Cc: kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D8097: rust-nodemap: accounting for dead blocks

[Raphaël Gomès]

Alphare updated this revision to Diff 20279.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D8097?vs=20238&id=20279

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D8097/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D8097

AFFECTED FILES
  rust/hg-core/src/revlog/nodemap.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/revlog/nodemap.rs 
b/rust/hg-core/src/revlog/nodemap.rs
--- a/rust/hg-core/src/revlog/nodemap.rs
+++ b/rust/hg-core/src/revlog/nodemap.rs
@@ -272,6 +272,7 @@
 readonly: Box + Send>,
 growable: Vec,
 root: Block,
+masked_inner_blocks: usize,
 }
 
 impl Index for NodeTree {
@@ -348,6 +349,7 @@
 readonly: readonly,
 growable: Vec::new(),
 root: root,
+masked_inner_blocks: 0,
 }
 }
 
@@ -480,6 +482,7 @@
 let ro_len = ro_blocks.len();
 let glen = self.growable.len();
 if idx < ro_len {
+self.masked_inner_blocks += 1;
 // TODO OPTIM I think this makes two copies
 self.growable.push(ro_blocks[idx].clone());
 (glen + ro_len, &mut self.growable[glen], glen + 1)
@@ -568,6 +571,22 @@
 }
 Ok(())
 }
+
+/// Return the number of blocks in the readonly part that are currently
+/// masked in the mutable part.
+///
+/// The `NodeTree` structure has no efficient way to know how many blocks
+/// are already unreachable in the readonly part.
+pub fn masked_readonly_blocks(&self) -> usize {
+if let Some(readonly_root) = self.readonly.last() {
+if readonly_root == &self.root {
+return 0;
+}
+} else {
+return 0;
+}
+self.masked_inner_blocks + 1
+}
 }
 
 pub struct NodeTreeBytes {
@@ -850,6 +869,7 @@
 readonly: sample_nodetree().readonly,
 growable: vec![block![0: Rev(1), 5: Rev(3)]],
 root: block![0: Block(1), 1:Block(3), 12: Rev(2)],
+masked_inner_blocks: 1,
 };
 assert_eq!(nt.find_hex(&idx, "10")?, Some(1));
 assert_eq!(nt.find_hex(&idx, "c")?, Some(2));
@@ -858,6 +878,7 @@
 assert_eq!(nt.find_hex(&idx, "000")?, Some(NULL_REVISION));
 assert_eq!(nt.unique_prefix_len_hex(&idx, "000")?, Some(3));
 assert_eq!(nt.find_hex(&idx, "01")?, Some(9));
+assert_eq!(nt.masked_readonly_blocks(), 2);
 Ok(())
 }
 
@@ -947,6 +968,8 @@
 assert_eq!(idx.find_hex("1a345")?, Some(3));
 assert_eq!(idx.find_hex("1a341")?, None);
 
+// there's no readonly block to mask
+assert_eq!(idx.nt.masked_readonly_blocks(), 0);
 Ok(())
 }
 
@@ -1008,6 +1031,8 @@
 assert_eq!(idx.find_hex("1235")?, Some(1));
 assert_eq!(idx.find_hex("131")?, Some(2));
 assert_eq!(idx.find_hex("cafe")?, Some(3));
+// we did not add anything since init from readonly
+assert_eq!(idx.nt.masked_readonly_blocks(), 0);
 
 idx.insert(4, "123A")?;
 assert_eq!(idx.find_hex("1234")?, Some(0));
@@ -1015,12 +1040,18 @@
 assert_eq!(idx.find_hex("131")?, Some(2));
 assert_eq!(idx.find_hex("cafe")?, Some(3));
 assert_eq!(idx.find_hex("123A")?, Some(4));
+// we masked blocks for all prefixes of "123", including the root
+assert_eq!(idx.nt.masked_readonly_blocks(), 4);
 
+eprintln!("{:?}", idx.nt);
 idx.insert(5, "c0")?;
 assert_eq!(idx.find_hex("cafe")?, Some(3));
 assert_eq!(idx.find_hex("c0")?, Some(5));
 assert_eq!(idx.find_hex("c1")?, None);
 assert_eq!(idx.find_hex("1234")?, Some(0));
+// inserting "c0" is just splitting the 'c' slot of the mutable root,
+// it doesn't mask anything
+assert_eq!(idx.nt.masked_readonly_blocks(), 4);
 
 Ok(())
 }



To: marmoute, #hg-reviewers
Cc: kevincox, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D7797: rust-nodemap: pure Rust example

[Raphaël Gomès]

Alphare updated this revision to Diff 20277.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7797?vs=20216&id=20277

BRANCH
  default

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7797/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7797

AFFECTED FILES
  rust/Cargo.lock
  rust/hg-core/Cargo.toml
  rust/hg-core/examples/nodemap/index.rs
  rust/hg-core/examples/nodemap/main.rs

CHANGE DETAILS

diff --git a/rust/hg-core/examples/nodemap/main.rs 
b/rust/hg-core/examples/nodemap/main.rs
new file mode 100644
--- /dev/null
+++ b/rust/hg-core/examples/nodemap/main.rs
@@ -0,0 +1,146 @@
+// Copyright 2019-2020 Georges Racinet 
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+use clap::*;
+use hg::revlog::node::*;
+use hg::revlog::nodemap::*;
+use hg::revlog::*;
+use memmap::MmapOptions;
+use rand::Rng;
+use std::fs::File;
+use std::io;
+use std::io::Write;
+use std::path::{Path, PathBuf};
+use std::str::FromStr;
+use std::time::Instant;
+
+mod index;
+use index::Index;
+
+fn mmap_index(repo_path: &Path) -> Index {
+let mut path = PathBuf::from(repo_path);
+path.extend([".hg", "store", "00changelog.i"].iter());
+Index::load_mmap(path)
+}
+
+fn mmap_nodemap(path: &Path) -> NodeTree {
+let file = File::open(path).unwrap();
+let mmap = unsafe { MmapOptions::new().map(&file).unwrap() };
+let len = mmap.len();
+NodeTree::load_bytes(Box::new(mmap), len)
+}
+
+/// Scan the whole index and create the corresponding nodemap file at `path`
+fn create(index: &Index, path: &Path) -> io::Result<()> {
+let mut file = File::create(path)?;
+let start = Instant::now();
+let mut nm = NodeTree::default();
+for rev in 0..index.len() {
+let rev = rev as Revision;
+nm.insert(index, index.node(rev).unwrap(), rev).unwrap();
+}
+eprintln!("Nodemap constructed in RAM in {:?}", start.elapsed());
+file.write(&nm.into_readonly_and_added_bytes().1)?;
+eprintln!("Nodemap written to disk");
+Ok(())
+}
+
+fn query(index: &Index, nm: &NodeTree, prefix: &str) {
+let start = Instant::now();
+let res = nm.find_hex(index, prefix);
+println!("Result found in {:?}: {:?}", start.elapsed(), res);
+}
+
+fn bench(index: &Index, nm: &NodeTree, queries: usize) {
+let len = index.len() as u32;
+let mut rng = rand::thread_rng();
+let nodes: Vec = (0..queries)
+.map(|_| {
+index
+.node((rng.gen::() % len) as Revision)
+.unwrap()
+.clone()
+})
+.collect();
+if queries < 10 {
+let nodes_hex: Vec =
+nodes.iter().map(|n| n.encode_hex()).collect();
+println!("Nodes: {:?}", nodes_hex);
+}
+let mut last: Option = None;
+let start = Instant::now();
+for node in nodes.iter() {
+last = nm.find_bin(index, node.into()).unwrap();
+}
+let elapsed = start.elapsed();
+println!(
+"Did {} queries in {:?} (mean {:?}), last was {:?} with result {:?}",
+queries,
+elapsed,
+elapsed / (queries as u32),
+nodes.last().unwrap().encode_hex(),
+last
+);
+}
+
+fn main() {
+let matches = App::new("Nodemap pure Rust example")
+.arg(
+Arg::with_name("REPOSITORY")
+.help("Path to the repository, always necessary for its index")
+.required(true),
+)
+.arg(
+Arg::with_name("NODEMAP_FILE")
+.help("Path to the nodemap file, independent of REPOSITORY")
+.required(true),
+)
+.subcommand(
+SubCommand::with_name("create")
+.about("Create NODEMAP_FILE by scanning repository index"),
+)
+.subcommand(
+SubCommand::with_name("query")
+.about("Query NODEMAP_FILE for PREFIX")
+.arg(Arg::with_name("PREFIX").required(true)),
+)
+.subcommand(
+SubCommand::with_name("bench")
+.about(
+"Perform #QUERIES random successful queries on 
NODEMAP_FILE")
+.arg(Arg::with_name("QUERIES").required(true)),
+)
+.get_matches();
+
+let repo = matches.value_of("REPOSITORY").unwrap();
+let nm_path = matches.value_of("NODEMAP_FILE").unwrap();
+
+let index = mmap_index(&Path::new(repo));
+
+if let Some(_) = matches.subcommand_matches("create") {
+println!("Creating nodemap file {} for repository {}", nm_path, repo);
+create(&index, &Path::new(nm_path)).unwrap();
+return;
+}
+
+let nm = mmap_nodemap(&Path::new(nm_path));
+if let Some(matches) = matches.subcommand_matches("query") {
+let prefix = matches.value_of("PREFIX").unwrap();
+println!(
+"Querying {} in nodemap file {} of reposito