Re: [PATCH 2 of 2] setdiscovery: stop limiting the number of local head we initially send

[Martin von Zweigbergk via Mercurial-devel]

On Tue, Apr 16, 2019 at 8:59 PM Martin von Zweigbergk 
wrote:

>
>
> On Tue, Apr 16, 2019 at 11:41 AM Pierre-Yves David <
> pierre-yves.da...@ens-lyon.org> wrote:
>
>> # HG changeset patch
>> # User Pierre-Yves David 
>> # Date 1555428398 -7200
>> #  Tue Apr 16 17:26:38 2019 +0200
>> # Node ID ffaa98def33a903f132ec4177d36823a741b6ef6
>> # Parent  017778a4463a8e6ecb4b17cacf46a3ab27bdb239
>> # EXP-Topic discovery-speedup
>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
>> ffaa98def33a
>> setdiscovery: stop limiting the number of local head we initially send
>>
>
> I think I tried something like this before. I still have a commit in my
> repo. I don't remember why I didn't send it. Perhaps the issue was that the
> heads are sent in HTTP headers and we need to limit the size of those to
> work with restrictive proxies and servers?
>
>
By the way, if I'm right about that, it would be nice to have a devel
config that makes the server reject requests longer than a configured
length. Then we could set that to 2k or something in tests, so we can catch
this in tests in the future.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] setdiscovery: stop limiting the number of local head we initially send

[Martin von Zweigbergk via Mercurial-devel]

On Tue, Apr 16, 2019 at 11:41 AM Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1555428398 -7200
> #  Tue Apr 16 17:26:38 2019 +0200
> # Node ID ffaa98def33a903f132ec4177d36823a741b6ef6
> # Parent  017778a4463a8e6ecb4b17cacf46a3ab27bdb239
> # EXP-Topic discovery-speedup
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> ffaa98def33a
> setdiscovery: stop limiting the number of local head we initially send
>

I think I tried something like this before. I still have a commit in my
repo. I don't remember why I didn't send it. Perhaps the issue was that the
heads are sent in HTTP headers and we need to limit the size of those to
work with restrictive proxies and servers?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 4 of 4] verify: also check full manifest validity during verify runs

[Martin von Zweigbergk via Mercurial-devel]

On Tue, Apr 16, 2019, 16:46 Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1551881213 -3600
> #  Wed Mar 06 15:06:53 2019 +0100
> # Node ID ed796867a06764cd78a57b2ed0249353f5809319
> # Parent  9bec7491e9b4cabdfa4d264e5213b1f416ec2607
> # EXP-Topic verify
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> ed796867a067
> verify: also check full manifest validity during verify runs
>
> Before this changes, `hg verify` only checked if a manifest revision
> existed and
> referenced the proper files. However it never checked the manifest revision
> content itself.
>
> Mercurial is expecting manifest entries to be sorted and will crash
> otherwise.
> Since `hg verify` did not attempted a full restoration of manifest entry,
> it
> could ignore this kind of corruption.
>
> This new check significantly increases the cost of a `hg verify` run. This
> especially affects large repository not using `sparse-revlog`. For now,
> this is
> hidden behind the `--full` experimental flag.
>
> diff --git a/mercurial/verify.py b/mercurial/verify.py
> --- a/mercurial/verify.py
> +++ b/mercurial/verify.py
> @@ -337,6 +337,16 @@ class verifier(object):
>  filenodes.setdefault(fullpath, {}).setdefault(fn,
> lr)
>  except Exception as inst:
>  self._exc(lr, _("reading delta %s") % short(n), inst,
> label)
> +if not dir and self._level >= VERIFY_FULL:
>

What does the "not dir" mean? I guess it's to do this check only for the
root directory when using tree manifests. Should we do it for all
directories?

+try:
> +# Manifest not in sorted order are invalid and will
> crash
> +# Mercurial. We restore each entry to make sure they
> are
>

Nit: "restore" almost makes it sound like this is fixing broken entries.
Maybe something like "We read the full manifest at each revision to..."?

+# ordered.
> +mfdelta = mfl.get(dir, n).read()
> +except Exception as inst:
> +self._exc(lr, _("reading full manifest %s") %
> short(n),
> +  inst, label)
> +
>  if not dir:
>  progress.complete()
>
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 3] rust-filepatterns: add a Rust implementation of pattern-related utils

[Raphaël Gomès]

# HG changeset patch
# User Raphaël Gomès 
# Date 1555442271 -7200
#  Tue Apr 16 21:17:51 2019 +0200
# Node ID ad489b44ef16ae39eb2c692fc5ccf3268a140a65
# Parent  a362b0b95e42c8f7d46d7e3a0eb4cc531fa5f2d6
# EXP-Topic rust-filepatterns
rust-filepatterns: add a Rust implementation of pattern-related utils

This change introduces Rust implementations of two functions related to
pattern handling, all located in `match.py`:

- `_regex`
- `readpatternfile`

These utils are useful in the long-term effort to improve `hg status`'s
performance using Rust. Experimental work done by Valentin Gatien-Baron
shows very promising improvements, but is too different from the current
Mercurial core code structure to be used "as-is".
This is the first - albeit very small - step towards the code revamp
needed down the line.

Two dependencies were added: `regex` and `lazy_static`. Both of them
will be useful for a majority of the Rust code that will be written,
are well known and maintained either by the Rust core team, or by
very frequent contributors.

diff -r a362b0b95e42 -r ad489b44ef16 rust/Cargo.lock
--- a/rust/Cargo.lock   Fri Apr 05 14:35:33 2019 +0200
+++ b/rust/Cargo.lock   Tue Apr 16 21:17:51 2019 +0200
@@ -49,8 +49,10 @@
 name = "hg-core"
 version = "0.1.0"
 dependencies = [
+ "lazy_static 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand_pcg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "regex 1.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -74,7 +76,7 @@
 
 [[package]]
 name = "lazy_static"
-version = "1.2.0"
+version = "1.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index";
 
 [[package]]
@@ -265,7 +267,7 @@
 version = "0.3.6"
 source = "registry+https://github.com/rust-lang/crates.io-index";
 dependencies = [
- "lazy_static 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "lazy_static 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
 [[package]]
@@ -310,7 +312,7 @@
 "checksum cloudabi 0.0.3 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"ddfc5b9aa5d4507acaf872de71051dfd0e309860e88966e1051e462a077aac4f"
 "checksum cpython 0.2.1 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"b489034e723e7f5109fecd19b719e664f89ef925be785885252469e9822fa940"
 "checksum fuchsia-cprng 0.1.0 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"81f7f8eb465745ea9b02e2704612a9946a59fa40572086c6fd49d6ddcf30bf31"
-"checksum lazy_static 1.2.0 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"a374c89b9db55895453a74c1e38861d9deec0b01b405a82516e9d5de4820dea1"
+"checksum lazy_static 1.3.0 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"bc5729f27f159ddd61f4df6228e827e86643d4d3e7c32183cb30a1c08f604a14"
 "checksum libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)" 
= "2d2857ec59fadc0773853c664d2d18e7198e83883e7060b63c924cb077bd5c74"
 "checksum memchr 2.1.2 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"db4c41318937f6e76648f42826b1d9ade5c09cafb5aef7e351240a70f39206e9"
 "checksum num-traits 0.2.6 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"0b3a5d7cc97d6d30d8b9bc8fa19bf45349ffe46241e8816f50f62f6d6aaabee1"
diff -r a362b0b95e42 -r ad489b44ef16 rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml   Fri Apr 05 14:35:33 2019 +0200
+++ b/rust/hg-core/Cargo.toml   Tue Apr 16 21:17:51 2019 +0200
@@ -10,3 +10,7 @@
 [dev-dependencies]
 rand = "*"
 rand_pcg = "*"
+
+[dependencies]
+lazy_static = "1.3.0"
+regex = "^1.1"
\ No newline at end of file
diff -r a362b0b95e42 -r ad489b44ef16 rust/hg-core/src/filepatterns.rs
--- /dev/null   Thu Jan 01 00:00:00 1970 +
+++ b/rust/hg-core/src/filepatterns.rs  Tue Apr 16 21:17:51 2019 +0200
@@ -0,0 +1,346 @@
+use crate::{LineNumber, PatternError, PatternFileError};
+use regex::Regex;
+use std::borrow::Cow;
+use std::collections::HashMap;
+use std::fs::File;
+use std::io::Read;
+use std::vec::Vec;
+
+lazy_static! {
+static ref reescape: Vec> = {
+let mut v: Vec> = (0..=255).map(|byte| vec![byte]).collect();
+let to_escape = b"()[]{}?*+-|^$\\.&~# \t\n\r\x0b\x0c";
+for byte in to_escape {
+v[*byte as usize].insert(0, b'\\');
+}
+v
+};
+}
+
+const GLOB_REPLACEMENTS: &[(&[u8], &[u8])] =
+&[(b"*/", b"(?:.*/)?"), (b"*", b".*"), (b"", b"[^/]*")];
+
+#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+pub enum PatternSyntax {
+Regexp,
+RootGlob,
+// Glob that matches at the front of the path
+Glob,
+// Glob that matches at any suffix of the path (still anchored at slashes)
+Path,
+RelPath,
+RelGlob,
+RelRegexp,
+RootFiles,
+}
+
+/// Transforms a glob pattern into a regex
+fn glob_to_re(pat: &[u8]) -> Vec {
+let mut input = pat;
+let mut res: Vec = vec![];
+let mut group_depth = 0;
+
+while let

[PATCH 3 of 3] rust-filepatterns: call new Rust implementations from Python

[Raphaël Gomès]

# HG changeset patch
# User Raphaël Gomès 
# Date 1555442904 -7200
#  Tue Apr 16 21:28:24 2019 +0200
# Node ID ba9a1dd508c6c167d0c920ece643854c5bc6b6e7
# Parent  8586c2273ee3f0c4dd08bf69474fa33bb19de405
# EXP-Topic rust-filepatterns
rust-filepatterns: call new Rust implementations from Python

This change adds the import to the `rust-cpython` bindings and uses
them when appropriate.

A wrapper function has been defined in the case of `_regex` to
keep this patch simple.

diff -r 8586c2273ee3 -r ba9a1dd508c6 mercurial/match.py
--- a/mercurial/match.pyTue Apr 16 21:27:47 2019 +0200
+++ b/mercurial/match.pyTue Apr 16 21:28:24 2019 +0200
@@ -24,6 +24,12 @@
 stringutil,
 )
 
+try:
+from . import rustext
+rustext.__name__  # force actual import (see hgdemandimport)
+except ImportError:
+rustext = None
+
 allpatternkinds = ('re', 'glob', 'path', 'relglob', 'relpath', 'relre',
'rootglob',
'listfile', 'listfile0', 'set', 'include', 'subinclude',
@@ -1178,8 +1184,22 @@
 return res
 
 def _regex(kind, pat, globsuffix):
-'''Convert a (normalized) pattern of any kind into a regular expression.
+'''Convert a (normalized) pattern of any kind into a
+regular expression.
 globsuffix is appended to the regexp of globs.'''
+
+if rustext is not None:
+try:
+return rustext.filepatterns.build_single_regex(
+kind,
+pat,
+globsuffix
+)
+except rustext.filepatterns.PatternError:
+raise error.ProgrammingError(
+'not a regex pattern: %s:%s' % (kind, pat)
+)
+
 if not pat:
 return ''
 if kind == 're':
@@ -1421,9 +1441,24 @@
 pattern# pattern of the current default type
 
 if sourceinfo is set, returns a list of tuples:
-(pattern, lineno, originalline). This is useful to debug ignore patterns.
+(pattern, lineno, originalline).
+This is useful to debug ignore patterns.
 '''
 
+if rustext is not None:
+result, warnings = rustext.filepatterns.read_pattern_file(
+filepath,
+bool(warn),
+sourceinfo,
+)
+
+for warning_params in warnings:
+# Can't be easily emitted from Rust, because it would require
+# a mechanism for both gettext and calling the `warn` function.
+warn(_("%s: ignoring invalid syntax '%s'\n") % warning_params)
+
+return result
+
 syntaxes = {
 're': 'relre:',
 'regexp': 'relre:',
diff -r 8586c2273ee3 -r ba9a1dd508c6 tests/common-pattern.py
--- a/tests/common-pattern.py   Tue Apr 16 21:27:47 2019 +0200
+++ b/tests/common-pattern.py   Tue Apr 16 21:28:24 2019 +0200
@@ -115,6 +115,11 @@
 # Various platform error strings, keyed on a common replacement string
 _errors = {
 br'$ENOENT$': (
+# IOError in Python does not have the same error message
+# than in Rust, and automatic conversion is not possible
+# because of module member privacy.
+br'No such file or directory \(os error 2\)',
+
 # strerror()
 br'No such file or directory',
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 3] rust-filepatterns: add `rust-cpython` bindings for `filepatterns`

[Raphaël Gomès]

# HG changeset patch
# User Raphaël Gomès 
# Date 1555442867 -7200
#  Tue Apr 16 21:27:47 2019 +0200
# Node ID 8586c2273ee3f0c4dd08bf69474fa33bb19de405
# Parent  ad489b44ef16ae39eb2c692fc5ccf3268a140a65
# EXP-Topic rust-filepatterns
rust-filepatterns: add `rust-cpython` bindings for `filepatterns`

This change adds the `rust-cpython` interface for top-level functions and
exceptions in the filepatterns module.

Contrary to the Python implementation, this tries to have finer-grained
exceptions to allow for better readability and flow control down the line.

diff -r ad489b44ef16 -r 8586c2273ee3 rust/hg-cpython/src/exceptions.rs
--- a/rust/hg-cpython/src/exceptions.rs Tue Apr 16 21:17:51 2019 +0200
+++ b/rust/hg-cpython/src/exceptions.rs Tue Apr 16 21:27:47 2019 +0200
@@ -12,8 +12,8 @@
 //! existing Python exceptions if appropriate.
 //!
 //! [`GraphError`]: struct.GraphError.html
-use cpython::exc::ValueError;
-use cpython::{PyErr, Python};
+use cpython::exc::{ValueError, RuntimeError};
+use cpython::{PyErr, Python, exc};
 use hg;
 
 py_exception!(rustext, GraphError, ValueError);
@@ -28,9 +28,43 @@
 match py
 .import("mercurial.error")
 .and_then(|m| m.get(py, "WdirUnsupported"))
-{
-Err(e) => e,
-Ok(cls) => PyErr::from_instance(py, cls),
+{
+Err(e) => e,
+Ok(cls) => PyErr::from_instance(py, cls),
+}
+}
+}
+}
+}
+
+py_exception!(rustext, PatternError, RuntimeError);
+py_exception!(rustext, PatternFileError, RuntimeError);
+
+impl PatternError {
+pub fn pynew(py: Python, inner: hg::PatternError) -> PyErr {
+match inner {
+hg::PatternError::UnsupportedSyntax(m) => {
+PatternError::new(py, ("PatternError", m))
+}
+}
+}
+}
+
+
+impl PatternFileError {
+pub fn pynew(py: Python, inner: hg::PatternFileError) -> PyErr {
+match inner {
+hg::PatternFileError::IO(e) => {
+let value = (
+e.raw_os_error().unwrap_or(2),
+e.to_string()
+);
+PyErr::new::(py, value)
+}
+hg::PatternFileError::Pattern(e, l) => {
+match e {
+hg::PatternError::UnsupportedSyntax(m) =>
+PatternFileError::new(py, ("PatternFileError", m, l))
 }
 }
 }
diff -r ad489b44ef16 -r 8586c2273ee3 rust/hg-cpython/src/filepatterns.rs
--- /dev/null   Thu Jan 01 00:00:00 1970 +
+++ b/rust/hg-cpython/src/filepatterns.rs   Tue Apr 16 21:27:47 2019 +0200
@@ -0,0 +1,115 @@
+// filepatterns.rs
+//
+// Copyright 2019, Georges Racinet ,
+// Raphaël Gomès 
+//
+// This software may be used and distributed according to the terms of the
+// GNU General Public License version 2 or any later version.
+
+//! Bindings for the `hg::filepatterns` module provided by the
+//! `hg-core` crate. From Python, this will be seen as `rustext.filepatterns`
+//! and can be used as replacement for the the pure `filepatterns` Python 
module.
+//!
+use cpython::{
+exc, PyDict, PyErr, PyModule, PyResult, PyString, PyTuple, Python,
+ToPyObject,
+};
+use hg::{build_single_regex, read_pattern_file, PatternTuple};
+use exceptions::{
+PatternError,
+PatternFileError,
+};
+
+/// Rust does not like functions with different return signatures.
+/// The 3-tuple version is always returned by the hg-core function,
+/// the (potential) conversion is handled at this level since it is not likely
+/// to have any measurable impact on performance.
+///
+/// The Python implementation passes a function reference for `warn` instead
+/// of a boolean that is used to emit warnings while parsing. The Rust
+/// implementation chooses to accumulate the warnings and propagate them to
+/// Python upon completion. See the `readpatternfile` function in `match.py`
+/// for more details.
+fn read_pattern_file_wrapper(
+py: Python,
+file_path: String,
+warn: bool,
+source_info: bool,
+) -> PyResult {
+match read_pattern_file(file_path, warn) {
+Ok((patterns, warnings)) => {
+if source_info {
+return Ok((patterns, warnings).to_py_object(py));
+}
+let itemgetter = |x: &PatternTuple| x.0.to_py_object(py);
+let results: Vec =
+patterns.iter().map(itemgetter).collect();
+Ok((results, warnings).to_py_object(py))
+}
+Err(e) => Err(PatternFileError::pynew(py, e)),
+}
+}
+
+fn build_single_regex_wrapper(
+py: Python,
+kind: String,
+pat: String,
+globsuffix: String,
+) -> PyResult {
+match build_single_regex(
+kind.as_ref(),
+pat.as_bytes(),
+globsuffix.as_bytes(),
+) {
+Ok(regex) => match String::from_u

D6258: packaging: coerce paths to strings

[durin42 (Augie Fackler)]

durin42 added a comment.


  Argh. This change appears to be wrong (!) so I'm going to drop it.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6258

To: durin42, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 4] recover: add a --[no-]verify flag

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1555454220 -7200
#  Wed Apr 17 00:37:00 2019 +0200
# Node ID f233cb63bc077267d8571378350d9563cbabcf3d
# Parent  eebf78724d94649de84f921ff5bd39cbc0f48cb6
# EXP-Topic verify
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
f233cb63bc07
recover: add a --[no-]verify flag

For trivial cases, the cost of the verify run after `hg recover` is getting in
the way. In addition for very large repositories, the cost is simply too high
to be paid, making `hg recover` an unusable commands.

We introduce a --verify flag, set by default. If is automatically associated
with a --no-verify flag that one can use to skip the verify step.

We might consider changing the default behavior in the future. However this is
out of scope for this series.

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -4655,8 +4655,11 @@ def push(ui, repo, dest=None, **opts):
 
 return result
 
-@command('recover', [], helpcategory=command.CATEGORY_MAINTENANCE)
-def recover(ui, repo):
+@command('recover',
+[('','verify', True, "run `hg verify` after succesful recover"),
+],
+helpcategory=command.CATEGORY_MAINTENANCE)
+def recover(ui, repo, **opts):
 """roll back an interrupted transaction
 
 Recover from an interrupted commit or pull.
@@ -4667,8 +4670,15 @@ def recover(ui, repo):
 
 Returns 0 if successful, 1 if nothing to recover or verify fails.
 """
-if repo.recover():
-return hg.verify(repo)
+ret = repo.recover()
+if ret:
+if opts['verify']:
+return hg.verify(repo)
+else:
+msg = _("(verify step skipped, run  `hg verify` to check your "
+"repository content)\n")
+ui.warn(msg)
+return 0
 return 1
 
 @command('remove|rm',
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -332,7 +332,7 @@ Show all commands + options
   phase: public, draft, secret, force, rev
   pull: update, force, rev, bookmark, branch, ssh, remotecmd, insecure
   push: force, rev, bookmark, branch, new-branch, pushvars, publish, ssh, 
remotecmd, insecure
-  recover: 
+  recover: verify
   remove: after, force, subrepos, include, exclude, dry-run
   rename: after, force, include, exclude, dry-run
   resolve: all, list, mark, unmark, no-status, re-merge, tool, include, 
exclude, template
diff --git a/tests/test-journal-exists.t b/tests/test-journal-exists.t
--- a/tests/test-journal-exists.t
+++ b/tests/test-journal-exists.t
@@ -21,6 +21,33 @@
   checking files
   checked 1 changesets with 1 changes to 1 files
 
+recover, explicite verify
+
+  $ touch .hg/store/journal
+  $ hg ci -Am0
+  abort: abandoned transaction found!
+  (run 'hg recover' to clean up transaction)
+  [255]
+  $ hg recover --verify
+  rolling back interrupted transaction
+  checking changesets
+  checking manifests
+  crosschecking files in changesets and manifests
+  checking files
+  checked 1 changesets with 1 changes to 1 files
+
+recover, no verify
+
+  $ touch .hg/store/journal
+  $ hg ci -Am0
+  abort: abandoned transaction found!
+  (run 'hg recover' to clean up transaction)
+  [255]
+  $ hg recover --no-verify
+  rolling back interrupted transaction
+  (verify step skipped, run  `hg verify` to check your repository content)
+
+
 Check that zero-size journals are correctly aborted:
 
 #if unix-permissions no-root
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 4] verify: introduce an experimental --full flag

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1555456269 -7200
#  Wed Apr 17 01:11:09 2019 +0200
# Node ID 9bec7491e9b4cabdfa4d264e5213b1f416ec2607
# Parent  55bd98999c25b10e220477fd4cc446a7c9c1f8ca
# EXP-Topic verify
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
9bec7491e9b4
verify: introduce an experimental --full flag

The flag currently has no effect, see next changeset for details. We introduce
the flag as experimental to keep the freedom of changing our mind on the final
UI.

Note: this patch highlight a small but in `hg help`. An option section is
generated even if no option are visible.

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -63,6 +63,7 @@ from . import (
 tags as tagsmod,
 ui as uimod,
 util,
+verify as verifymod,
 wireprotoserver,
 )
 from .utils import (
@@ -6147,8 +6148,10 @@ def update(ui, repo, node=None, **opts):
 ui.warn("(%s)\n" % obsfatemsg)
 return ret
 
-@command('verify', [], helpcategory=command.CATEGORY_MAINTENANCE)
-def verify(ui, repo):
+@command('verify',
+ [('', 'full', False, 'perform more checks (EXPERIMENTAL)')],
+ helpcategory=command.CATEGORY_MAINTENANCE)
+def verify(ui, repo, **opts):
 """verify the integrity of the repository
 
 Verify the integrity of the current repository.
@@ -6164,7 +6167,10 @@ def verify(ui, repo):
 
 Returns 0 on success, 1 if errors are encountered.
 """
-return hg.verify(repo)
+level = None
+if opts['full']:
+level = verifymod.VERIFY_FULL
+return hg.verify(repo, level)
 
 @command(
 'version', [] + formatteropts, helpcategory=command.CATEGORY_HELP,
diff --git a/mercurial/verify.py b/mercurial/verify.py
--- a/mercurial/verify.py
+++ b/mercurial/verify.py
@@ -23,6 +23,7 @@ from . import (
 )
 
 VERIFY_DEFAULT = 0
+VERIFY_FULL = 1
 
 def verify(repo, level=None):
 with repo.lock():
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -347,7 +347,7 @@ Show all commands + options
   tip: patch, git, style, template
   unbundle: update
   update: clean, check, merge, date, rev, tool
-  verify: 
+  verify: full
   version: template
 
   $ hg init a
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -615,6 +615,8 @@ Test command without options
   
   Returns 0 on success, 1 if errors are encountered.
   
+  options:
+  
   (some details hidden, use --verbose to show complete help)
 
   $ hg help diff
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 4] verify: also check full manifest validity during verify runs

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1551881213 -3600
#  Wed Mar 06 15:06:53 2019 +0100
# Node ID ed796867a06764cd78a57b2ed0249353f5809319
# Parent  9bec7491e9b4cabdfa4d264e5213b1f416ec2607
# EXP-Topic verify
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
ed796867a067
verify: also check full manifest validity during verify runs

Before this changes, `hg verify` only checked if a manifest revision existed and
referenced the proper files. However it never checked the manifest revision
content itself.

Mercurial is expecting manifest entries to be sorted and will crash otherwise.
Since `hg verify` did not attempted a full restoration of manifest entry, it
could ignore this kind of corruption.

This new check significantly increases the cost of a `hg verify` run. This
especially affects large repository not using `sparse-revlog`. For now, this is
hidden behind the `--full` experimental flag.

diff --git a/mercurial/verify.py b/mercurial/verify.py
--- a/mercurial/verify.py
+++ b/mercurial/verify.py
@@ -337,6 +337,16 @@ class verifier(object):
 filenodes.setdefault(fullpath, {}).setdefault(fn, lr)
 except Exception as inst:
 self._exc(lr, _("reading delta %s") % short(n), inst, label)
+if not dir and self._level >= VERIFY_FULL:
+try:
+# Manifest not in sorted order are invalid and will crash
+# Mercurial. We restore each entry to make sure they are
+# ordered.
+mfdelta = mfl.get(dir, n).read()
+except Exception as inst:
+self._exc(lr, _("reading full manifest %s") % short(n),
+  inst, label)
+
 if not dir:
 progress.complete()
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 4] verify: introduce a notion of "level"

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1555456341 -7200
#  Wed Apr 17 01:12:21 2019 +0200
# Node ID 55bd98999c25b10e220477fd4cc446a7c9c1f8ca
# Parent  f233cb63bc077267d8571378350d9563cbabcf3d
# EXP-Topic verify
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
55bd98999c25
verify: introduce a notion of "level"

Some checks are slower than others, to help the user to run the checks he needs,
we are about to introduce new flag to select faster vs deeper runs. This put
the scaffolding in place to do this.

diff --git a/mercurial/hg.py b/mercurial/hg.py
--- a/mercurial/hg.py
+++ b/mercurial/hg.py
@@ -1092,9 +1092,9 @@ def outgoing(ui, repo, dest, opts):
 recurse()
 return 0 # exit code is zero since we found outgoing changes
 
-def verify(repo):
+def verify(repo, level=None):
 """verify the consistency of a repository"""
-ret = verifymod.verify(repo)
+ret = verifymod.verify(repo, level=level)
 
 # Broken subrepo references in hidden csets don't seem worth worrying 
about,
 # since they can't be pushed/pulled, and --hidden can be used if they are a
diff --git a/mercurial/verify.py b/mercurial/verify.py
--- a/mercurial/verify.py
+++ b/mercurial/verify.py
@@ -22,9 +22,12 @@ from . import (
 util,
 )
 
-def verify(repo):
+VERIFY_DEFAULT = 0
+
+def verify(repo, level=None):
 with repo.lock():
-return verifier(repo).verify()
+v = verifier(repo, level)
+return v.verify()
 
 def _normpath(f):
 # under hg < 2.4, convert didn't sanitize paths properly, so a
@@ -34,10 +37,13 @@ def _normpath(f):
 return f
 
 class verifier(object):
-def __init__(self, repo):
+def __init__(self, repo, level=None):
 self.repo = repo.unfiltered()
 self.ui = repo.ui
 self.match = repo.narrowmatch()
+if level is None:
+level = VERIFY_DEFAULT
+self._level = level
 self.badrevs = set()
 self.errors = 0
 self.warnings = 0
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: Changing the wdir() {rev} and {node} output

[Yuya Nishihara]

On Tue, 16 Apr 2019 10:14:53 -0400, Jordi Gutiérrez Hermoso wrote:
> While I think it might be okay to keep the iternal values,
> 
> https://www.mercurial-scm.org/repo/hg/file/967c098eed33/mercurial/node.py#l39
> 
> I think it's far too confusing to expose them in the user interface.
> 
> A particular inconsistency I see is that we already have a way to show
> wdir(), for example. In a way, `hg id -T` is kind of an alias for `hg
> log -r 'wdir()' -T ... ` in a way that is not obvious. But trying to
> use '{node}' or '{rev}' in `hg id` results in exposing the internal
> revision and node magic numbers, which is very confusing behaviour,
> since the default behaviour of `hg id` is to show `{p1.rev}+` instead
> if there are changes in the wdir.
> 
> Another instance in which this happens is `hg annotate -r 'wdir()'`,
> which also shows plusses to indicate that the changes are only in the
> working directory.
> 
> I think we should keep this behaviour throughout, for consistency.
> Since `wdir()` is still marked as experimental, we can change its
> current behaviour. I propose consistency with `hg id` as follows:

First, I'm okay to change the output of "hg whatever -r 'wdir()'" if that
makes things more consistent. However, {rev}/{node} keywords should still
return wdirrev/wdirhex since they are valid identifiers whereas "+"
isn't.

> * If the working directory is clean, the following are equivalent:
> 
>   hg log -r 'wdir()' -T '{rev}:{node}'
>   hg log -r . -T '{rev}:{node}'

Do you mean `hg log -r 'wdir()'` should fall back to '.' if wdir is clean?
I believe it'll be way complicated.

If clean "wdir()" == ".", "hg status --change 'wdir()'" would have to
show the changes in "." as well, which makes no sense.

> * Otherwise, if there are changes in the working directory, the
>   following are equivalent:
> 
>  hg log -r 'wdir()' -T '{rev}:{node}'
>  hg log -r '.' -T'{rev}+:{node}+'
> 
> In other words, never show in the interface the internal values of
> wdir() obtained from node.py
> 
> I think this would be consistent with the existing usage of `hg id`
> and `hg log`, the main consumers of `wdir()`.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6183: copies: add config option for writing copy metadata to file and/or changset

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG0fdf45cb3dc6: copies: add config option for writing copy 
metadata to file and/or changset (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6183?vs=14766&id=14791

REVISION DETAIL
  https://phab.mercurial-scm.org/D6183

AFFECTED FILES
  mercurial/changelog.py
  mercurial/configitems.py
  mercurial/localrepo.py
  tests/test-annotate.t
  tests/test-copies-in-changeset.t
  tests/test-fastannotate-hg.t

CHANGE DETAILS

diff --git a/tests/test-fastannotate-hg.t b/tests/test-fastannotate-hg.t
--- a/tests/test-fastannotate-hg.t
+++ b/tests/test-fastannotate-hg.t
@@ -443,7 +443,7 @@
   > def reposetup(ui, repo):
   > class legacyrepo(repo.__class__):
   > def _filecommit(self, fctx, manifest1, manifest2,
-  > linkrev, tr, changelist):
+  > linkrev, tr, changelist, includecopymeta):
   > fname = fctx.path()
   > text = fctx.data()
   > flog = self.file(fname)
diff --git a/tests/test-copies-in-changeset.t b/tests/test-copies-in-changeset.t
new file mode 100644
--- /dev/null
+++ b/tests/test-copies-in-changeset.t
@@ -0,0 +1,105 @@
+
+  $ cat >> $HGRCPATH << EOF
+  > [experimental]
+  > copies.write-to=changeset-only
+  > [alias]
+  > changesetcopies = log -r . -T 'files: {files}
+  >   {extras % "{ifcontains("copies", key, "{key}: {value}\n")}"}'
+  > EOF
+
+Check that copies are recorded correctly
+
+  $ hg init repo
+  $ cd repo
+  $ echo a > a
+  $ hg add a
+  $ hg ci -m initial
+  $ hg cp a b
+  $ hg cp a c
+  $ hg cp a d
+  $ hg ci -m 'copy a to b, c, and d'
+  $ hg changesetcopies
+  files: b c d
+  p1copies: b\x00a (esc)
+  c\x00a (esc)
+  d\x00a (esc)
+
+Check that renames are recorded correctly
+
+  $ hg mv b b2
+  $ hg ci -m 'rename b to b2'
+  $ hg changesetcopies
+  files: b b2
+  p1copies: b2\x00b (esc)
+
+Rename onto existing file. This should get recorded in the changeset files 
list and in the extras,
+even though there is no filelog entry.
+
+  $ hg cp b2 c --force
+  $ hg st --copies
+  M c
+b2
+  $ hg debugindex c
+ rev linkrev nodeid   p1   p2
+   0   1 b789fdd96dc2  
+  $ hg ci -m 'move b onto d'
+  $ hg changesetcopies
+  files: c
+  p1copies: c\x00b2 (esc)
+  $ hg debugindex c
+ rev linkrev nodeid   p1   p2
+   0   1 b789fdd96dc2  
+
+Create a merge commit with copying done during merge.
+
+  $ hg co 0
+  0 files updated, 0 files merged, 3 files removed, 0 files unresolved
+  $ hg cp a e
+  $ hg cp a f
+  $ hg ci -m 'copy a to e and f'
+  created new head
+  $ hg merge 3
+  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+File 'a' exists on both sides, so 'g' could be recorded as being from p1 or 
p2, but we currently
+always record it as being from p1
+  $ hg cp a g
+File 'd' exists only in p2, so 'h' should be from p2
+  $ hg cp d h
+File 'f' exists only in p1, so 'i' should be from p1
+  $ hg cp f i
+  $ hg ci -m 'merge'
+  $ hg changesetcopies
+  files: g h i
+  p1copies: g\x00a (esc)
+  i\x00f (esc)
+  p2copies: h\x00d (esc)
+
+Test writing to both changeset and filelog
+
+  $ hg cp a j
+  $ hg ci -m 'copy a to j' --config experimental.copies.write-to=compatibility
+  $ hg changesetcopies
+  files: j
+  p1copies: j\x00a (esc)
+  $ hg debugdata j 0
+  \x01 (esc)
+  copy: a
+  copyrev: b789fdd96dc2f3bd229c1dd8eedf0fc60e2b68e3
+  \x01 (esc)
+  a
+
+Test writing only to filelog
+
+  $ hg cp a k
+  $ hg ci -m 'copy a to k' --config experimental.copies.write-to=filelog-only
+  $ hg changesetcopies
+  files: k
+  $ hg debugdata k 0
+  \x01 (esc)
+  copy: a
+  copyrev: b789fdd96dc2f3bd229c1dd8eedf0fc60e2b68e3
+  \x01 (esc)
+  a
+
+  $ cd ..
diff --git a/tests/test-annotate.t b/tests/test-annotate.t
--- a/tests/test-annotate.t
+++ b/tests/test-annotate.t
@@ -438,7 +438,7 @@
   > def reposetup(ui, repo):
   > class legacyrepo(repo.__class__):
   > def _filecommit(self, fctx, manifest1, manifest2,
-  > linkrev, tr, changelist):
+  > linkrev, tr, changelist, includecopymeta):
   > fname = fctx.path()
   > text = fctx.data()
   > flog = self.file(fname)
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -2324,7 +2324,8 @@
 """Returns the wlock if it's held, or None if it's not."""
 return self._currentlock(self._wlockref)
 
-def _filecommit(self, fctx, manifest1, manifest2, linkrev, tr, changelist):
+def _filecommit(self, fctx, manifest1, manifest2, linkrev, tr, changelist,
+includecopymeta):
 """
 commit an individual file as part of a larger transactio

D6186: changelog: parse copy metadata if available in extras

[martinvonz (Martin von Zweigbergk)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG0cfd1f9fb02b: changelog: parse copy metadata if available 
in extras (authored by martinvonz, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6186?vs=14767&id=14792

REVISION DETAIL
  https://phab.mercurial-scm.org/D6186

AFFECTED FILES
  mercurial/changelog.py
  mercurial/context.py
  mercurial/copies.py
  tests/test-copies-in-changeset.t
  tests/test-copies.t

CHANGE DETAILS

diff --git a/tests/test-copies.t b/tests/test-copies.t
--- a/tests/test-copies.t
+++ b/tests/test-copies.t
@@ -1,4 +1,4 @@
-#testcases filelog compatibility
+#testcases filelog compatibility changeset
 
   $ cat >> $HGRCPATH << EOF
   > [extensions]
@@ -14,6 +14,14 @@
   > EOF
 #endif
 
+#if changeset
+  $ cat >> $HGRCPATH << EOF
+  > [experimental]
+  > copies.read-from = changeset-only
+  > copies.write-to = changeset-only
+  > EOF
+#endif
+
   $ REPONUM=0
   $ newrepo() {
   > cd $TESTTMP
@@ -376,11 +384,13 @@
   o  0 add x on branch 1
  x
   $ hg debugp1copies -r 2
+  x -> z (changeset !)
   $ hg debugp2copies -r 2
-  x -> z
+  x -> z (no-changeset !)
   $ hg debugpathcopies 1 2
+  x -> z (changeset !)
   $ hg debugpathcopies 0 2
-  x -> z
+  x -> z (no-changeset !)
 
 Copy x->y on one side of merge and copy x->z on the other side. Pathcopies 
from one parent
 of the merge to the merge should include the copy from the other side.
@@ -539,6 +549,9 @@
 
 Grafting revision 4 on top of revision 2, showing that it respect the rename:
 
+TODO: Make this work with copy info in changesets (probably by writing a
+changeset-centric version of copies.mergecopies())
+#if no-changeset
   $ hg up 2 -q
   $ hg graft -r 4 --base 3 --hidden
   grafting 4:af28412ec03c "added d, modified b" (tip)
@@ -554,6 +567,8 @@
   b
  +baba
   
+#endif
+
 Test to make sure that fullcopytracing algorithm don't fail when both the 
merging csets are dirty
 (a dirty cset is one who is not the descendant of merge base)
 
-
diff --git a/tests/test-copies-in-changeset.t b/tests/test-copies-in-changeset.t
--- a/tests/test-copies-in-changeset.t
+++ b/tests/test-copies-in-changeset.t
@@ -2,9 +2,11 @@
   $ cat >> $HGRCPATH << EOF
   > [experimental]
   > copies.write-to=changeset-only
+  > copies.read-from=changeset-only
   > [alias]
   > changesetcopies = log -r . -T 'files: {files}
   >   {extras % "{ifcontains("copies", key, "{key}: {value}\n")}"}'
+  > showcopies = log -r . -T '{file_copies % "{source} -> {name}\n"}'
   > EOF
 
 Check that copies are recorded correctly
@@ -23,14 +25,25 @@
   p1copies: b\x00a (esc)
   c\x00a (esc)
   d\x00a (esc)
+  $ hg showcopies
+  a -> b
+  a -> c
+  a -> d
+  $ hg showcopies --config experimental.copies.read-from=compatibility
+  a -> b
+  a -> c
+  a -> d
+  $ hg showcopies --config experimental.copies.read-from=filelog-only
 
 Check that renames are recorded correctly
 
   $ hg mv b b2
   $ hg ci -m 'rename b to b2'
   $ hg changesetcopies
   files: b b2
   p1copies: b2\x00b (esc)
+  $ hg showcopies
+  b -> b2
 
 Rename onto existing file. This should get recorded in the changeset files 
list and in the extras,
 even though there is no filelog entry.
@@ -46,6 +59,8 @@
   $ hg changesetcopies
   files: c
   p1copies: c\x00b2 (esc)
+  $ hg showcopies
+  b2 -> c
   $ hg debugindex c
  rev linkrev nodeid   p1   p2
0   1 b789fdd96dc2  
@@ -74,6 +89,10 @@
   p1copies: g\x00a (esc)
   i\x00f (esc)
   p2copies: h\x00d (esc)
+  $ hg showcopies
+  a -> g
+  d -> h
+  f -> i
 
 Test writing to both changeset and filelog
 
@@ -88,6 +107,12 @@
   copyrev: b789fdd96dc2f3bd229c1dd8eedf0fc60e2b68e3
   \x01 (esc)
   a
+  $ hg showcopies
+  a -> j
+  $ hg showcopies --config experimental.copies.read-from=compatibility
+  a -> j
+  $ hg showcopies --config experimental.copies.read-from=filelog-only
+  a -> j
 
 Test writing only to filelog
 
@@ -101,5 +126,10 @@
   copyrev: b789fdd96dc2f3bd229c1dd8eedf0fc60e2b68e3
   \x01 (esc)
   a
+  $ hg showcopies
+  $ hg showcopies --config experimental.copies.read-from=compatibility
+  a -> k
+  $ hg showcopies --config experimental.copies.read-from=filelog-only
+  a -> k
 
   $ cd ..
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -162,8 +162,8 @@
 
 def usechangesetcentricalgo(repo):
 """Checks if we should use changeset-centric copy algorithms"""
-return (repo.ui.config('experimental', 'copies.read-from') ==
-'compatibility')
+return (repo.ui.config('experimental', 'copies.read-from') in
+('changeset-only', 'compatibility'))
 
 def _committedforwardcopies(a, b, match):
 """Like _forwardcopies(), but b.rev() cannot be None (working copy)"""
diff --git a/mercurial/context.py b/mercurial/context.p

D6238: branchcache: don't verify while creating a copy

[yuja (Yuya Nishihara)]

yuja added a comment.


  >   def copy(self):
  >   """return an deep copy of the branchcache object"""
  > 
  > - self._verifyall() return type(self)( self._entries, self.tipnode, 
self.tiprev, self.filteredhash, self._closednodes)
  
  Looks good, but it's probably better to copy the verification state to
  new instance, or simply make branchcache() start with "verified" state.
  IIUC, only branches loaded from file have to be validated.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6238

To: pulkit, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6207: branchcache: add functions to validate changelog nodes

[yuja (Yuya Nishihara)]

yuja added a comment.


  >   The motivation behind iteritems() change 
(https://phab.mercurial-scm.org/D6236) is optimizing 
https://www.mercurial-scm.org/repo/hg-committed/file/70b71421fd33/mercurial/commands.py#l1128.
  
  Okay, got that, thanks.
  
  >   Maybe we can iterate over verified nodes first, and then iterate over 
unverified nodes?
  
  Maybe no. I don't think it's more likely for callers to get any verified node
  and stop the iteration.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6207

To: pulkit, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D6238: branchcache: don't verify while creating a copy

[Yuya Nishihara]

>  def copy(self):
>  """return an deep copy of the branchcache object"""
> -self._verifyall()
>  return type(self)(
>  self._entries, self.tipnode, self.tiprev, self.filteredhash,
>  self._closednodes)

Looks good, but it's probably better to copy the verification state to
new instance, or simply make branchcache() start with "verified" state.
IIUC, only branches loaded from file have to be validated.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D6207: branchcache: add functions to validate changelog nodes

[Yuya Nishihara]

>   The motivation behind iteritems() change 
> (https://phab.mercurial-scm.org/D6236) is optimizing 
> https://www.mercurial-scm.org/repo/hg-committed/file/70b71421fd33/mercurial/commands.py#l1128.

Okay, got that, thanks.

>   Maybe we can iterate over verified nodes first, and then iterate over 
> unverified nodes?

Maybe no. I don't think it's more likely for callers to get any verified node
and stop the iteration.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] setdiscovery: stop limiting the number of local head we initially send

[Pierre-Yves David]

(email sent on Pulkit Goyal suggestion)

It would help us (Octobus) if this patch can make it into 5.0.

This is a small code change that remove an embarrassing slow down in 
case where they would expect Mercurial to be snappy. That kind of 
discovery performance boosts make a significant difference on our side.


Cheers,

On 4/16/19 7:24 PM, Pierre-Yves David wrote:

# HG changeset patch
# User Pierre-Yves David 
# Date 1555428398 -7200
#  Tue Apr 16 17:26:38 2019 +0200
# Node ID ffaa98def33a903f132ec4177d36823a741b6ef6
# Parent  017778a4463a8e6ecb4b17cacf46a3ab27bdb239
# EXP-Topic discovery-speedup
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
ffaa98def33a
setdiscovery: stop limiting the number of local head we initially send

In our testing this limitation provides now real gain and instead triggers
pathological discovery timing for some repository with many heads.

See inline documentation for details.

Some timing below:

Mozilla try repository, (~1M revs, ~35K heads), discovery between 2 clones with
100 head missing on each side

before:
! wall 1.492111 comb 1.49 user 1.45 sys 0.04 (best of 20)
! wall 1.813992 comb 1.82 user 1.70 sys 0.12 (max of 20)
! wall 1.574326 comb 1.573500 user 1.522000 sys 0.051500 (avg of 20)
! wall 1.572583 comb 1.57 user 1.52 sys 0.05 (median of 20)

after:
! wall 1.147834 comb 1.15 user 1.09 sys 0.06 (best of 20)
! wall 1.449144 comb 1.45 user 1.33 sys 0.12 (max of 20)
! wall 1.204618 comb 1.202500 user 1.146500 sys 0.056000 (avg of 20)
! wall 1.194407 comb 1.19 user 1.14 sys 0.05 (median of 20)


pypy (~100 heads, 317 heads) discovery between clones with only 42 common heads

before:
! wall 0.031653 comb 0.03 user 0.03 sys 0.00 (best of 25)
! wall 0.055719 comb 0.05 user 0.04 sys 0.01 (max of 25)
! wall 0.038939 comb 0.039600 user 0.038400 sys 0.001200 (avg of 25)
! wall 0.038660 comb 0.05 user 0.04 sys 0.01 (median of 25)

after:
! wall 0.018754 comb 0.02 user 0.02 sys 0.00 (best of 49)
! wall 0.034505 comb 0.04 user 0.03 sys 0.01 (max of 49)
! wall 0.019631 comb 0.019796 user 0.018367 sys 0.001429 (avg of 49)
! wall 0.019132 comb 0.02 user 0.02 sys 0.00 (median of 49)


Private repository (~1M revs, ~3K heads), discovery from a strip subset, about
100 changesets to be pulled.

before:
! wall 1.837729 comb 1.84 user 1.79 sys 0.05 (best of 20)
! wall 2.203468 comb 2.20 user 2.10 sys 0.10 (max of 20)
! wall 2.049355 comb 2.048500 user 2.002500 sys 0.046000 (avg of 20)
! wall 2.035315 comb 2.04 user 2.00 sys 0.04 (median of 20)

after:
! wall 0.136598 comb 0.13 user 0.11 sys 0.02 (best of 20)
! wall 0.330519 comb 0.33 user 0.26 sys 0.07 (max of 20)
! wall 0.157254 comb 0.155500 user 0.123000 sys 0.032500 (avg of 20)
! wall 0.149870 comb 0.14 user 0.11 sys 0.03 (median of 20)


Same private repo, discovery between two clone with 500 different heads on each
side:

before:
! wall 2.372919 comb 2.37 user 2.32 sys 0.05 (best of 20)
! wall 2.622422 comb 2.61 user 2.51 sys 0.10 (max of 20)
! wall 2.450135 comb 2.45 user 2.402000 sys 0.048000 (avg of 20)
! wall 2.443896 comb 2.45 user 2.41 sys 0.04 (median of 20)

after:
! wall 0.625497 comb 0.62 user 0.57 sys 0.05 (best of 20)
! wall 0.834723 comb 0.82 user 0.73 sys 0.09 (max of 20)
! wall 0.675725 comb 0.675500 user 0.628000 sys 0.047500 (avg of 20)
! wall 0.671614 comb 0.68 user 0.64 sys 0.04 (median of 20)

diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py
--- a/mercurial/setdiscovery.py
+++ b/mercurial/setdiscovery.py
@@ -275,9 +275,58 @@ def findcommonheads(ui, local, remote,
  # early exit if we know all the specified remote heads already
  ui.debug("query 1; heads\n")
  roundtrips += 1
-sample = _limitsample(ownheads, initialsamplesize)
-# indices between sample and externalized version must match
-sample = list(sample)
+# We also ask remote about all the local heads. That set can be arbitrarily
+# large, so we used to limit it size to `initialsamplesize`. We no longer
+# do as it proved counter productive. The skipped heads could lead to a
+# large "undecided" set, slower to be clarified than if we asked the
+# question for all heads right away.
+#
+# We are already fetching all server heads using the `heads` commands,
+# sending a equivalent number of heads the other way should not have a
+# significant impact.  In addition, it is very likely that we are going to
+# have to issue "known" request for an equivalent amount of revisions in
+# order to decide if theses heads are common or missing.
+#
+# find a detailled analysis below.
+#
+# Case 

Re: [PATCH 1 of 3 V2] repoview: move subsettable in a dedicated module

[Pierre-Yves David]

On 4/17/19 12:14 AM, Pulkit Goyal wrote:


On Tue, Apr 16, 2019 at 11:56 PM Pierre-Yves David 
mailto:pierre-yves.da...@ens-lyon.org>> 
wrote:


# HG changeset patch
# User Pierre-Yves David mailto:pierre-yves.da...@octobus.net>>
# Date 1555076492 -7200
#      Fri Apr 12 15:41:32 2019 +0200
# Node ID 56e2557a9401a3b3c60956b86860c84fb2e1330a
# Parent  2003f0bffcb197699b6665939e21a681f9c73570
# EXP-Topic repoview
# Available At https://bitbucket.org/octobus/mercurial-devel/
#              hg pull
https://bitbucket.org/octobus/mercurial-devel/ -r 56e2557a9401
repoview: move subsettable in a dedicated module

The dictionary got moved in `branchmap` to avoid import cycle.
However, we are
about to needs it in repoview too. So we introduce a now module to
define that
that mapping.


Queued 1 and 3. Many thanks!


Discussing on IRC with pulkit, he did not had time to fully grasp the 
concept around path 2. I am going to sent it as (untouched) as V3 for 
clarify. Still according to this IRC conversion, other reviewers are 
encouraged to look at it.


Having the experimental in 5.0 would help to start wider testing make a 
difference on our side.


Cheers,

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] repoview: introduce a `experimental.extra-filter-revs` config

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1554565579 -7200
#  Sat Apr 06 17:46:19 2019 +0200
# Node ID 86eac5d2d7a1386dca5970126506d07201058200
# Parent  eebf78724d94649de84f921ff5bd39cbc0f48cb6
# EXP-Topic repoview
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
86eac5d2d7a1
repoview: introduce a `experimental.extra-filter-revs` config

The option define revisions to additionally filter out of all repository "view".
The end goal is to provide and easy to way to serve multiple subset of the same
repository using multiple "shares".

The simplest use case of this feature is to have one view serving the public
changesets and one view also serving the draft. This is currently achievable
using the new `server.view` option introduced recently by Joerg Sonnenberger.
However, more advanced use cases need more advanced definitions. For example
some needs a view dedicated to some release branches, or view that hides
security fixes to be released. Joerg Sonnenberger and I discussed this topic at
the recent mini-sprint and the both of us have seen real life use cases for
this. (This series got written during the same mini-sprint).

The feature is fully functional, and use similar cache-fallback mechanism to
ensure decent performance. However,there remaining room to ensure each share
caches and hooks collaborate with each others. This will come at a later time
once users start to actually test this feature on real usecase.

diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -529,6 +529,13 @@ coreconfigitem('experimental', 'evolutio
 coreconfigitem('experimental', 'evolution.track-operation',
 default=True,
 )
+# repo-level config to exclude a revset visibility
+#
+# The target use case is to use `share` to expose different subset of the same
+# repository, especially server side. See also `server.view`.
+coreconfigitem('experimental', 'extra-filter-revs',
+default=None,
+)
 coreconfigitem('experimental', 'maxdeltachainspan',
 default=-1,
 )
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1050,6 +1050,8 @@ class localrepository(object):
 # Signature to cached matcher instance.
 self._sparsematchercache = {}
 
+self._extrafilterid = repoview.extrafilter(ui)
+
 def _getvfsward(self, origfunc):
 """build a ward for self.vfs"""
 rref = weakref.ref(self)
@@ -1197,6 +1199,9 @@ class localrepository(object):
 
 In other word, there is always only one level of `repoview` 
"filtering".
 """
+if self._extrafilterid is not None:
+name = name + '%'  + self._extrafilterid
+
 cls = repoview.newtype(self.unfiltered().__class__)
 return cls(self, name, visibilityexceptions)
 
diff --git a/mercurial/repoview.py b/mercurial/repoview.py
--- a/mercurial/repoview.py
+++ b/mercurial/repoview.py
@@ -17,6 +17,10 @@ from . import (
 phases,
 pycompat,
 tags as tagsmod,
+util,
+)
+from .utils import (
+repoviewutil,
 )
 
 def hideablerevs(repo):
@@ -154,6 +158,35 @@ filtertable = {'visible': computehidden,
'immutable':  computemutable,
'base':  computeimpactable}
 
+_basefiltername = list(filtertable)
+
+def extrafilter(ui):
+"""initialize extra filter and return its id
+
+If extra filtering is configured, we make sure the associated filtered view
+are declared and return the associated id.
+"""
+frevs = ui.config('experimental', 'extra-filter-revs')
+if frevs is None:
+return None
+
+fid = util.DIGESTS['sha512'](frevs).hexdigest()
+
+combine = lambda fname: fname + '%' + fid
+
+subsettable = repoviewutil.subsettable
+
+if combine('base') not in filtertable:
+for name in _basefiltername:
+def extrafilteredrevs(repo, *args, **kwargs):
+baserevs = filtertable[name](repo, *args, **kwargs)
+extrarevs = frozenset(repo.revs(frevs))
+return baserevs | extrarevs
+filtertable[combine(name)] = extrafilteredrevs
+if name in subsettable:
+subsettable[combine(name)] = combine(subsettable[name])
+return fid
+
 def filterrevs(repo, filtername, visibilityexceptions=None):
 """returns set of filtered revision for this filter name
 
diff --git a/mercurial/statichttprepo.py b/mercurial/statichttprepo.py
--- a/mercurial/statichttprepo.py
+++ b/mercurial/statichttprepo.py
@@ -155,6 +155,7 @@ class statichttprepository(localrepo.loc
 
 self.names = namespaces.namespaces()
 self.filtername = None
+self._extrafilterid = None
 
 try:
 requirements = set(self.vfs.read(b'requires').splitlines())
diff --git a/tests/test-server-view.t b/tests/test-server-view.

D6258: packaging: coerce paths to strings

[durin42 (Augie Fackler)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG69b4426d288b: packaging: coerce paths to strings (authored 
by durin42, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6258?vs=14788&id=14790

REVISION DETAIL
  https://phab.mercurial-scm.org/D6258

AFFECTED FILES
  contrib/packaging/hgpackaging/util.py

CHANGE DETAILS

diff --git a/contrib/packaging/hgpackaging/util.py 
b/contrib/packaging/hgpackaging/util.py
--- a/contrib/packaging/hgpackaging/util.py
+++ b/contrib/packaging/hgpackaging/util.py
@@ -17,12 +17,12 @@
 
 
 def extract_tar_to_directory(source: pathlib.Path, dest: pathlib.Path):
-with tarfile.open(source, 'r') as tf:
+with tarfile.open(str(source), 'r') as tf:
 tf.extractall(dest)
 
 
 def extract_zip_to_directory(source: pathlib.Path, dest: pathlib.Path):
-with zipfile.ZipFile(source, 'r') as zf:
+with zipfile.ZipFile(str(source), 'r') as zf:
 zf.extractall(dest)
 
 



To: durin42, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6259: revset: on-disk cache for children queries

[joerg.sonnenberger (Joerg Sonnenberger)]

joerg.sonnenberger created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is a proof of concept for further discussion.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6259

AFFECTED FILES
  mercurial/localrepo.py
  mercurial/revset.py

CHANGE DETAILS

diff --git a/mercurial/revset.py b/mercurial/revset.py
--- a/mercurial/revset.py
+++ b/mercurial/revset.py
@@ -385,13 +385,13 @@
 cs = set()
 for r in getset(repo, fullreposet(repo), x):
 for i in range(n):
-c = repo[r].children()
+c = repo._childrencache.children(r)
 if len(c) == 0:
 break
 if len(c) > 1:
 raise error.RepoLookupError(
 _("revision in set has more than one child"))
-r = c[0].rev()
+r = list(c)[0]
 else:
 cs.add(r)
 return subset & cs
@@ -586,18 +586,7 @@
 def _children(repo, subset, parentset):
 if not parentset:
 return baseset()
-cs = set()
-pr = repo.changelog.parentrevs
-minrev = parentset.min()
-nullrev = node.nullrev
-for r in subset:
-if r <= minrev:
-continue
-p1, p2 = pr(r)
-if p1 in parentset:
-cs.add(r)
-if p2 != nullrev and p2 in parentset:
-cs.add(r)
+cs = set.union(*[set(repo._childrencache.children(c)) for c in parentset])
 return baseset(cs)
 
 @predicate('children(set)', safe=True)
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -11,6 +11,7 @@
 import hashlib
 import os
 import random
+import struct
 import sys
 import time
 import weakref
@@ -994,6 +995,7 @@
 
 self._branchcaches = {}
 self._revbranchcache = None
+self.__childrencache = None
 self._filterpats = {}
 self._datafilters = {}
 self._transref = self._lockref = self._wlockref = None
@@ -1084,6 +1086,8 @@
 def _writecaches(self):
 if self._revbranchcache:
 self._revbranchcache.write()
+if self.__childrencache:
+self.__childrencache.write()
 
 def _restrictcapabilities(self, caps):
 if self.ui.configbool('experimental', 'bundle2-advertise'):
@@ -1194,6 +1198,11 @@
 return manifest.manifestlog(self.svfs, self, rootstore,
 self._storenarrowmatch)
 
+@unfilteredpropertycache
+def _childrencache(self):
+self.__childrencache = childrencache(self)
+return self.__childrencache
+
 @repofilecache('dirstate')
 def dirstate(self):
 return self._makedirstate()
@@ -2087,6 +2096,8 @@
 # ensure the working copy parents are in the manifestfulltextcache
 for ctx in self['.'].parents():
 ctx.manifest()  # accessing the manifest is enough
+self._childrencache.clear()
+self._childrencache.write()
 
 def invalidatecaches(self):
 
@@ -2097,6 +2108,8 @@
 self.unfiltered()._branchcaches.clear()
 self.invalidatevolatilesets()
 self._sparsesignaturecache.clear()
+self._childrencache.clear()
+self._childrencache.write()
 
 def invalidatevolatilesets(self):
 self.filteredrevcache.clear()
@@ -3087,3 +3100,110 @@
 # We may have a repoview, which intercepts __setattr__. So be sure
 # we operate at the lowest level possible.
 object.__setattr__(repo, r'__class__', poisonedrepository)
+
+class childrencache(object):
+"""Persistent cache, mapping from revision number to revision numbers of
+the direct children. This is a low level cache, independent of filtering.
+"""
+_filename = 'childrencache'
+def __init__(self, repo, readonly=False):
+assert repo.filtername is None
+self._data = None
+self._new_data = {}
+self._read = False
+self._repo = repo
+self._readonly = readonly
+
+def children(self, rev):
+if not self._read:
+self.read()
+children = self._new_data.get(rev, set())
+rev += 1
+if rev >= self._lastknown:
+return children
+children = children.copy()
+child_len, child_off = struct.unpack_from('>ll', self._data, 8 * rev)
+if child_len == 1:
+children.add(child_off)
+elif child_len > 1:
+start = 8 * self._lastknown + child_off * 4
+if len(self._data) < start + child_len * 4:
+raise error.Abort('Truncated children cache')
+for i in xrange(child_len):
+child, = struct.unpack_from('>l', self._data, start)
+children.add(child)
+start += 4
+return children
+
+def clear(self):
+self._read = True
+self._data = None
+self._new_data = {}
+

Re: [PATCH] revsetbenchmark: add some simpler revset for heads and roots

[Pulkit Goyal]

On Wed, Apr 17, 2019 at 3:37 AM Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1554378409 -7200
> #  Thu Apr 04 13:46:49 2019 +0200
> # Node ID 264dd70fbad7f44f9d6b131b8e40d09b554879e2
> # Parent  a362b0b95e42c8f7d46d7e3a0eb4cc531fa5f2d6
> # EXP-Topic extra-revset
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 264dd70fbad7
> revsetbenchmark: add some simpler revset for heads and roots
>
> This revset might leverage compiled code in the future so lets start to
> track
> them.
>

Queued this one, many thanks!
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6255: copies: calculate mergecopies() based on pathcopies()

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  From Pierre-Yves:
  
  > Can you give that a shot with the two revisions we use in the benchmark 
  >  suite, this is a pair expensive with the current algorithm.
  > 
  > 1daa622bbe42 76caed42cf7c
  
  Sure, it takes about 29 seconds with or without this patch. It seems the 
difference is smaller than the noise level.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6255

To: martinvonz, #hg-reviewers
Cc: marmoute, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 3 V2] repoview: move subsettable in a dedicated module

[Pulkit Goyal]

On Tue, Apr 16, 2019 at 11:56 PM Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1555076492 -7200
> #  Fri Apr 12 15:41:32 2019 +0200
> # Node ID 56e2557a9401a3b3c60956b86860c84fb2e1330a
> # Parent  2003f0bffcb197699b6665939e21a681f9c73570
> # EXP-Topic repoview
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 56e2557a9401
> repoview: move subsettable in a dedicated module
>
> The dictionary got moved in `branchmap` to avoid import cycle. However, we
> are
> about to needs it in repoview too. So we introduce a now module to define
> that
> that mapping.
>

Queued 1 and 3. Many thanks!
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH] revsetbenchmark: add some simpler revset for heads and roots

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1554378409 -7200
#  Thu Apr 04 13:46:49 2019 +0200
# Node ID 264dd70fbad7f44f9d6b131b8e40d09b554879e2
# Parent  a362b0b95e42c8f7d46d7e3a0eb4cc531fa5f2d6
# EXP-Topic extra-revset
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
264dd70fbad7
revsetbenchmark: add some simpler revset for heads and roots

This revset might leverage compiled code in the future so lets start to track
them.

diff --git a/contrib/base-revsets.txt b/contrib/base-revsets.txt
--- a/contrib/base-revsets.txt
+++ b/contrib/base-revsets.txt
@@ -47,3 +47,6 @@ parents(2)
 # The one below is used by rebase
 (children(ancestor(tip~5, tip)) and ::(tip~5))::
 heads(commonancestors(last(head(), 2)))
+heads(-1:-1)
+roots(-1:-1)
+only(max(head()), min(head()))
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6258: packaging: coerce paths to strings

[durin42 (Augie Fackler)]

durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Experimentally passing the pathlib.Path to zipfile.ZipFile fails on at
  least some Python versions. I've gotten frustrated enough I'd rather
  just force them to be strings and move on.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6258

AFFECTED FILES
  contrib/packaging/hgpackaging/util.py

CHANGE DETAILS

diff --git a/contrib/packaging/hgpackaging/util.py 
b/contrib/packaging/hgpackaging/util.py
--- a/contrib/packaging/hgpackaging/util.py
+++ b/contrib/packaging/hgpackaging/util.py
@@ -17,12 +17,12 @@
 
 
 def extract_tar_to_directory(source: pathlib.Path, dest: pathlib.Path):
-with tarfile.open(source, 'r') as tf:
+with tarfile.open(str(source), 'r') as tf:
 tf.extractall(dest)
 
 
 def extract_zip_to_directory(source: pathlib.Path, dest: pathlib.Path):
-with zipfile.ZipFile(source, 'r') as zf:
+with zipfile.ZipFile(str(source), 'r') as zf:
 zf.extractall(dest)
 
 



To: durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6257: rust-discovery: core implementation for take_quick_sample()

[gracinet (Georges Racinet)]

gracinet created this revision.
Herald added subscribers: mercurial-devel, kevincox, durin42.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This makes in particular `rand` no longer a testing dependency.
  
  We keep a seedable random generator on the `PartialDiscovery` object
  itself, to avoid lengthy initialization.
  
  In take_quick_sample() itself, we had to avoid keeping the reference
  to `self.undecided` to cope with the mutable reference introduced
  by the the call to `limit_sample`, but it's still manageable without
  resorting to inner mutability.
  
  Sampling being prone to be improved in the mid-term future, testing
  is minimal, amounting to checking which code path got executed.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6257

AFFECTED FILES
  rust/hg-core/Cargo.toml
  rust/hg-core/src/discovery.rs

CHANGE DETAILS

diff --git a/rust/hg-core/src/discovery.rs b/rust/hg-core/src/discovery.rs
--- a/rust/hg-core/src/discovery.rs
+++ b/rust/hg-core/src/discovery.rs
@@ -10,23 +10,96 @@
 //! This is a Rust counterpart to the `partialdiscovery` class of
 //! `mercurial.setdiscovery`
 
-use super::{Graph, GraphError, Revision};
+extern crate rand;
+extern crate rand_pcg;
+use self::rand::seq::SliceRandom;
+use self::rand::{thread_rng, RngCore, SeedableRng};
+use super::{Graph, GraphError, Revision, NULL_REVISION};
 use crate::ancestors::MissingAncestors;
 use crate::dagops;
-use std::collections::HashSet;
+use std::collections::{HashMap, HashSet, VecDeque};
+
+type Rng = self::rand_pcg::Pcg32;
 
 pub struct PartialDiscovery {
 target_heads: Option>,
 graph: G, // plays the role of self._repo
 common: MissingAncestors,
 undecided: Option>,
 missing: HashSet,
+rng: Rng,
 }
 
 pub struct DiscoveryStats {
 pub undecided: Option,
 }
 
+/// Update an existing sample to match the expected size
+///
+/// The sample is updated with revisions exponentially distant from each
+/// element of `heads`.
+///
+/// If a target size is specified, the sampling will stop once this size is
+/// reached. Otherwise sampling will happen until roots of the  set are
+/// reached.
+///
+/// - `revs`: set of revs we want to discover (if None, `assume` the whole dag
+///   represented by `parentfn`
+/// - `heads`: set of DAG head revs
+/// - `sample`: a sample to update
+/// - `parentfn`: a callable to resolve parents for a revision
+/// - `quicksamplesize`: optional target size of the sample
+fn update_sample(
+revs: Option<&HashSet>,
+heads: impl IntoIterator,
+sample: &mut HashSet,
+parentsfn: impl Fn(Revision) -> Result<[Revision; 2], GraphError>,
+quicksamplesize: Option,
+) -> Result<(), GraphError> {
+let mut distances: HashMap = HashMap::new();
+let mut visit: VecDeque = heads.into_iter().collect();
+let mut factor: u32 = 1;
+let mut seen: HashSet = HashSet::new();
+loop {
+let current = match visit.pop_front() {
+None => {
+break;
+}
+Some(r) => r,
+};
+if seen.contains(¤t) {
+continue;
+}
+
+let d = *distances.entry(current).or_insert(1);
+if d > factor {
+factor *= 2;
+}
+if d == factor {
+sample.insert(current);
+if let Some(sz) = quicksamplesize {
+if sample.len() >= sz {
+return Ok(());
+}
+}
+}
+seen.insert(current);
+for p in parentsfn(current)?.iter().cloned() {
+if p == NULL_REVISION {
+continue;
+}
+if let Some(revs) = revs {
+if !revs.contains(&p) {
+continue;
+}
+}
+distances.entry(p).or_insert(d + 1);
+visit.push_back(p);
+}
+}
+Ok(())
+}
+
 impl PartialDiscovery {
 /// Create a PartialDiscovery object, with the intent
 /// of comparing our `::` revset to the contents of another
@@ -39,15 +112,47 @@
 /// we'll have to make it a type argument of `PartialDiscovery` or a trait
 /// object since we'll keep it in the meanwhile
 pub fn new(graph: G, target_heads: Vec) -> Self {
+let mut seed: [u8; 16] = [0; 16];
+thread_rng().fill_bytes(&mut seed);
+Self::new_with_seed(graph, target_heads, seed)
+}
+
+pub fn new_with_seed(
+graph: G,
+target_heads: Vec,
+seed: [u8; 16],
+) -> Self {
 PartialDiscovery {
 undecided: None,
 target_heads: Some(target_heads),
 graph: graph.clone(),
 common: MissingAncestors::new(graph, vec![]),
 missing: HashSet::new(),
+rng: Rng::from_seed(seed),
 }
 }
 
+/// Extract at most `size` random elements from sample and return them
+/// as a vector
+fn limit_sample(
+&mut self,
+  

Re: D6255: copies: calculate mergecopies() based on pathcopies()

[Pierre-Yves David]

On 4/16/19 11:08 PM, martinvonz (Martin von Zweigbergk) wrote:

martinvonz added a comment.


   In https://phab.mercurial-scm.org/D6255#91019, @martinvonz wrote:
   
   > In https://phab.mercurial-scm.org/D6255#91015, @marmoute wrote:

   >
   > > I did a first path through it, the new code seems reasonable and easier
   > >  to read than the previous one. Some comments and questions below.
   >
   >
   > The rest somehow didn't make it here, so I'll copy from the email (i.e. 
the below is from Pierre-Yves, not from me):
   >
   > >     Some timings for calculating mergecopies between two revisions (all
   > >     using the common ancestor as base):
   >
   > Which revisions did you pick?
   
   
   The revisions are those before the colon, so e.g. from hg version 4.0 to 4.8. Oops, the first line there should say "4.8 4.9", not "4.8 4.8". I've fixed that now.
Can you give that a shot with the two revisions we use in the benchmark 
suite, this is a pair expensive with the current algorithm.


1daa622bbe42 76caed42cf7c

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2] rust: published the three extension crates to crates.io

[Georges Racinet]

Hi,

On 4/16/19 9:26 PM, Georges Racinet wrote:
> # HG changeset patch
> # User Georges Racinet 
> # Date 1555441784 -7200
> #  Tue Apr 16 21:09:44 2019 +0200
> # Node ID 410396d5eca793a1b8cf9ee8ba7bce466e9be3c0
> # Parent  a362b0b95e42c8f7d46d7e3a0eb4cc531fa5f2d6
> # EXP-Topic rust-crates.io
> rust: published the three extension crates to crates.io
>
> These are the contents of the first publication for
> `hg-core`, `hg-cpython` and direct `hg-direct-ffi`, done with
> the sarting point '0.0.1' version.

In order to be really explicit, yes, I've uloaded them:

    https://crates.io/crates/hg-core

    https://crates.io/crates/hg-cpython

    https://crates.io/crates/hg-direct-ffi

Of course I'm all willing to give ownership on these crates to the
project, but that requires a GitHub Organization (only for
authentication). Reference: (ref:
https://doc.rust-lang.org/cargo/reference/publishing.html#cargo-owner).
I don't feel like I should be the one to create one out of the blue, but
I could and would invite the whole steering committee if I did.

The upload is of course out of band, but I found it somewhat appropriate
to submit the commits below, because it makes the involved files really
explicit, and persists the related explanations in a strong way. Also,
this was the occasion to fill the required fields (License etc).

> This version has been chosen to be actually lower than the one
> declared up to now in the crates manifests.
>
> In the workspace (rust/Cargo.toml), there's an override so that
> if building from the whole source, the local hg-core gets selected.
>
> We will have to select a version policy for the upcoming releases,
> with the obvious option to have Rust versions equal to the general
> Mercurial version, however partial they are.

I think the time between the freeze / branch cut and actual release
would probably be the appropriate one to come up with such a scheme
(more than right now anyway). I'll be ready to help if needed.

In any case, I suppose it's better whatever the actual version number is
to have it consistent in the release tag.

Cheers,

>
> It should be noted though that the development version must stay
> be higher than the latest that be found on crates.io and that satisfies
> the rules in Cargo.lock.
> (see 
> https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#testing-a-bugfix)
>
> diff -r a362b0b95e42 -r 410396d5eca7 rust/Cargo.lock
> --- a/rust/Cargo.lock Fri Apr 05 14:35:33 2019 +0200
> +++ b/rust/Cargo.lock Tue Apr 16 21:09:44 2019 +0200
> @@ -47,7 +47,7 @@
>  
>  [[package]]
>  name = "hg-core"
> -version = "0.1.0"
> +version = "0.0.1"
>  dependencies = [
>   "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)",
>   "rand_pcg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
> @@ -55,10 +55,10 @@
>  
>  [[package]]
>  name = "hg-cpython"
> -version = "0.1.0"
> +version = "0.0.1"
>  dependencies = [
>   "cpython 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
> - "hg-core 0.1.0",
> + "hg-core 0.0.1",
>   "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)",
>   "python27-sys 0.2.1 
> (registry+https://github.com/rust-lang/crates.io-index)",
>   "python3-sys 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
> @@ -68,7 +68,7 @@
>  name = "hgdirectffi"
>  version = "0.1.0"
>  dependencies = [
> - "hg-core 0.1.0",
> + "hg-core 0.0.1",
>   "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)",
>  ]
>  
> diff -r a362b0b95e42 -r 410396d5eca7 rust/Cargo.toml
> --- a/rust/Cargo.toml Fri Apr 05 14:35:33 2019 +0200
> +++ b/rust/Cargo.toml Tue Apr 16 21:09:44 2019 +0200
> @@ -1,3 +1,6 @@
>  [workspace]
>  members = ["hg-core", "hg-direct-ffi", "hg-cpython"]
>  exclude = ["chg", "hgcli"]
> +
> +[patch.crates-io]
> +hg-core = {path = "hg-core"}
> \ No newline at end of file
> diff -r a362b0b95e42 -r 410396d5eca7 rust/hg-core/Cargo.toml
> --- a/rust/hg-core/Cargo.toml Fri Apr 05 14:35:33 2019 +0200
> +++ b/rust/hg-core/Cargo.toml Tue Apr 16 21:09:44 2019 +0200
> @@ -1,12 +1,15 @@
>  [package]
>  name = "hg-core"
> -version = "0.1.0"
> -authors = ["Georges Racinet "]
> +version = "0.0.1"
> +authors = ["Georges Racinet "]
>  description = "Mercurial pure Rust core library, with no assumption on 
> Python bindings (FFI)"
> +license = "GPL-2.0-or-later"
> +homepage = "https://mercurial-scm.org";
> +repository = "https://www.mercurial-scm.org/repo/hg";
>  
>  [lib]
>  name = "hg"
>  
>  [dev-dependencies]
> -rand = "*"
> -rand_pcg = "*"
> +rand = "~0.6"
> +rand_pcg = "~0.1"
> diff -r a362b0b95e42 -r 410396d5eca7 rust/hg-cpython/Cargo.toml
> --- a/rust/hg-cpython/Cargo.toml  Fri Apr 05 14:35:33 2019 +0200
> +++ b/rust/hg-cpython/Cargo.toml  Tue Apr 16 21:09:44 2019 +0200
> @@ -1,7 +1,11 @@
>  [package]
>  name = "hg-cpython"
> -version = "0.1.0"
> -authors = ["Georges Racinet "]
> +version = "0.0.1"
> +authors = ["Georges Racinet "]
> +description = "Mercurial Pytho

D6255: copies: calculate mergecopies() based on pathcopies()

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  In https://phab.mercurial-scm.org/D6255#91019, @martinvonz wrote:
  
  > In https://phab.mercurial-scm.org/D6255#91015, @marmoute wrote:
  >
  > > I did a first path through it, the new code seems reasonable and easier 
  > >  to read than the previous one. Some comments and questions below.
  >
  >
  > The rest somehow didn't make it here, so I'll copy from the email (i.e. the 
below is from Pierre-Yves, not from me):
  >
  > >     Some timings for calculating mergecopies between two revisions (all
  > >     using the common ancestor as base):
  >
  > Which revisions did you pick?
  
  
  The revisions are those before the colon, so e.g. from hg version 4.0 to 4.8. 
Oops, the first line there should say "4.8 4.9", not "4.8 4.8". I've fixed that 
now.
  
  >>     In the hg repo:
  >>     4.8 4.8: 0.21s -> 0.21s
  >>     4.0 4.8: 0.35s -> 0.63s
  >>     
  >>     In and old copy of the mozilla-unified repo:
  >>     FIREFOX_BETA_60_BASE^ FIREFOX_BETA_60_BASE: 0.51s -> 0.60s
  >>     FIREFOX_NIGHTLY_59_END FIREFOX_BETA_60_BASE: 2.1s -> 2.3s
  >>     FIREFOX_BETA_59_END FIREFOX_BETA_60_BASE: 3.1s -> 3.3s
  >>     FIREFOX_AURORA_50_BASE FIREFOX_BETA_60_BASE: 30s -> 35s

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6255

To: martinvonz, #hg-reviewers
Cc: marmoute, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6255: copies: calculate mergecopies() based on pathcopies()

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  Replying to just a few things now. Will reply to the rest later.
  
  In https://phab.mercurial-scm.org/D6255#91019, @martinvonz wrote:
  
  > In https://phab.mercurial-scm.org/D6255#91015, @marmoute wrote:
  >
  > > I did a first path through it, the new code seems reasonable and easier 
  > >  to read than the previous one. Some comments and questions below.
  >
  >
  > The rest somehow didn't make it here, so I'll copy from the email (i.e. the 
below is from Pierre-Yves, not from me):
  >
  > I did a first path through it, the new code seems reasonable and easier 
  >  to read than the previous one. Some comments and questions below.
  
  
  Thanks for reviewing!
  
  >>     I've run tests with hard-coded debug logging for "fullcopy" and while
  >>     I haven't looked at every difference it produces, all the ones I have
  >>     looked at seemed reasonable to me.
  > 
  > How many difference did you had? can you share some example of them with 
  >  their explanation?
  
  Without explanation :), see http://paste.debian.net/1077862/
  
  It just seemed to long to include in the commit message.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6255

To: martinvonz, #hg-reviewers
Cc: marmoute, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6255: copies: calculate mergecopies() based on pathcopies()

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  In https://phab.mercurial-scm.org/D6255#91019, @martinvonz wrote:
  
  > In https://phab.mercurial-scm.org/D6255#91015, @marmoute wrote:
  >
  > > I did a first path through it, the new code seems reasonable and easier 
  > >  to read than the previous one. Some comments and questions below.
  >
  >
  > The rest somehow didn't make it here, so I'll copy from the email (i.e. the 
below is from Pierre-Yves, not from me):
  
  
  Ah, the reason for that might be that Phabricator was down at the time of the 
email.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6255

To: martinvonz, #hg-reviewers
Cc: marmoute, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6255: copies: calculate mergecopies() based on pathcopies()

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  In https://phab.mercurial-scm.org/D6255#91015, @marmoute wrote:
  
  > I did a first path through it, the new code seems reasonable and easier 
  >  to read than the previous one. Some comments and questions below.
  
  
  The rest somehow didn't make it here, so I'll copy from the email (i.e. the 
below is from Pierre-Yves, not from me):
  
  I did a first path through it, the new code seems reasonable and easier 
  to read than the previous one. Some comments and questions below.
  
  On 4/16/19 7:19 PM, martinvonz (Martin von Zweigbergk) wrote:
  
  > martinvonz created this revision.
  >  Herald added subscribers: mercurial-devel, mjpieters.
  >  Herald added a reviewer: hg-reviewers.
  > 
  > REVISION SUMMARY
  >     When copies are stored in changesets, we need a changeset-centric
  >     version of mergecopies() just like we have a changeset-centric version
  >     of pathcopies(). I think the natural way of thinking about
  >     mergecopies() is in terms of pathcopies() from the base to each of the
  >     commits. So if we can rewrite mergecopies() based two such
  >     pathcopies() calls, we'll get the changeset-centric version for
  >     free. That's what this patch does.
  
  I like the approach, if I understand it correctly, we'll have less 
  duplicated code in the end.
  
  >     
  >     A nice bonus is that it ends up being a lot simpler. mergecopies() has
  >     accumulated a lot of technical debt over time. One good example is the
  >     code for dealing with grafts (the "partial/incomplete/dirty"
  >     stuff). Since pathcopies() already deals with backwards renames and
  >     ping-pong renames, we get that for free.
  >     
  >     I've run tests with hard-coded debug logging for "fullcopy" and while
  >     I haven't looked at every difference it produces, all the ones I have
  >     looked at seemed reasonable to me.
  
  How many difference did you had? can you share some example of them with 
  their explanation?
  
  on related topic, you seems to be fixing a case in 
  test-fastannotate-hg.t and test-annotate-hg.t that is probably worth 
  mentioning.
  
  >     One drawback of the rewritten code is that we may now make
  >     remotefilelog prefetch more files. We used to prefetch files that were
  >     unique to either side of the merge compared to the other. We now
  >     prefetch files that are unique to either sise of the merge compared to
  
  sise → side
  
  >     the base. This means that if you added the same file to each side, we
  >     would not prefetch it before, but we would now. Such cases are
  >     probably quite rare, but one likely scenario where they happen is when
  >     moving from a commit to its successor (or the other way around). The
  >     user will probably already have the files in the cache in such cases,
  >     so it's probably not a big deal.
  >     
  >     Some timings for calculating mergecopies between two revisions (all
  >     using the common ancestor as base):
  
  Which revisions did you pick?
  
  (for the record, the benchmark suite use 1daa622bbe42 and 76caed42cf7c)
  
  >     In the hg repo:
  >     4.8 4.8: 0.21s -> 0.21s
  >     4.0 4.8: 0.35s -> 0.63s
  >     
  >     In and old copy of the mozilla-unified repo:
  >     FIREFOX_BETA_60_BASE^ FIREFOX_BETA_60_BASE: 0.51s -> 0.60s
  >     FIREFOX_NIGHTLY_59_END FIREFOX_BETA_60_BASE: 2.1s -> 2.3s
  >     FIREFOX_BETA_59_END FIREFOX_BETA_60_BASE: 3.1s -> 3.3s
  >     FIREFOX_AURORA_50_BASE FIREFOX_BETA_60_BASE: 30s -> 35s
  >     
  >     So it's measurably slower in most cases. Note that merge copies are
  >     not calculated when updating with a clean working copy, which is
  >     probably the most common case. I therefore think the much simpler code
  >     is worth the slowdown.
  
  Do you know where the slowdown comes from ?
  
  > REPOSITORY
  >     https://phab.mercurial-scm.org/diffusion/HG/ Mercurial
  > 
  > REVISION DETAIL
  >     https://phab.mercurial-scm.org/D6255
  > 
  > AFFECTED FILES
  >     mercurial/copies.py
  >     tests/test-annotate.t
  >     tests/test-fastannotate-hg.t
  >     tests/test-graft.t
  >     tests/test-rename-merge2.t
  > 
  > CHANGE DETAILS
  > 
  > diff --git a/tests/test-rename-merge2.t b/tests/test-rename-merge2.t
  > 
  > - a/tests/test-rename-merge2.t +++ b/tests/test-rename-merge2.t @@ -433,6 
+433,9 @@
  > 
  >      --
  >      test L:nc a b R:up b   W:       - 12 merge b no ancestor
  >      --
  >  +    all copies found (* = to merge, ! = divergent, % = renamed and 
deleted):
  >  +     src: 'a' -> dst: 'b'
  >  +    checking for directory renames
  >      resolving manifests
  >       branchmerge: True, force: False, partial: False
  >       ancestor: 924404dff337, local: 86a2aa42fc76+, remote: af30c7647fc7
  >  @@ -469,6 +472,9 @@
  >      --
  >      test L:up b   R:nm a b W:       - 13 merge b no ancestor
  >      --
  >  +    all copies 

D6182: bundle2: handle compression in _forwardchunks

[joerg.sonnenberger (Joerg Sonnenberger)]

joerg.sonnenberger added inline comments.

INLINE COMMENTS

> durin42 wrote in bundle2.py:846
> so...what's going on with oparams here? should it be outparams?
> 
> (is there more to this series that would get this code covered by tests?)

Yes, left-over from earlier rename round. I think the code should handle 
multiple parameters, since I have no idea when we might run into them. At the 
same time, I have no idea right now how to test that case.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6182

To: joerg.sonnenberger, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6182: bundle2: handle compression in _forwardchunks

[joerg.sonnenberger (Joerg Sonnenberger)]

joerg.sonnenberger updated this revision to Diff 14786.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6182?vs=14622&id=14786

REVISION DETAIL
  https://phab.mercurial-scm.org/D6182

AFFECTED FILES
  mercurial/bundle2.py
  tests/test-pull-bundle.t

CHANGE DETAILS

diff --git a/tests/test-pull-bundle.t b/tests/test-pull-bundle.t
--- a/tests/test-pull-bundle.t
+++ b/tests/test-pull-bundle.t
@@ -120,6 +120,38 @@
   * sending pullbundle "1.hg" (glob)
   $ rm repo/.hg/blackbox.log
 
+Test pullbundle functionality for incoming
+
+  $ cd repo
+  $ hg --config blackbox.track=debug --debug serve -p $HGPORT2 -d 
--pid-file=../repo.pid
+  listening at http://*:$HGPORT2/ (bound to $LOCALIP:$HGPORT2) (glob) (?)
+  $ cat ../repo.pid >> $DAEMON_PIDS
+  $ cd ..
+  $ hg clone http://localhost:$HGPORT2/ repo.pullbundle2a -r 0
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  new changesets bbd179dfa0a7 (1 drafts)
+  updating to branch default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ cd repo.pullbundle2a
+  $ hg incoming -r ed1b79f46b9a
+  comparing with http://localhost:$HGPORT2/
+  searching for changes
+  changeset:   1:ed1b79f46b9a
+  tag: tip
+  user:test
+  date:Thu Jan 01 00:00:00 1970 +
+  summary: change foo
+  
+  $ cd ..
+  $ killdaemons.py
+  $ grep 'sending pullbundle ' repo/.hg/blackbox.log
+  * sending pullbundle "0.hg" (glob)
+  * sending pullbundle "1.hg" (glob)
+  $ rm repo/.hg/blackbox.log
+
 Test recovery from misconfigured server sending no new data
 
   $ cd repo
diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -834,12 +834,21 @@
 if paramssize < 0:
 raise error.BundleValueError('negative bundle param size: %i'
  % paramssize)
-yield _pack(_fstreamparamsize, paramssize)
 if paramssize:
 params = self._readexact(paramssize)
 self._processallparams(params)
-yield params
-assert self._compengine.bundletype()[1] == 'UN'
+# The payload itself is decompressed below, so drop
+# the compression parameter passed down to compensate.
+outparams = []
+for p in params.split(' '):
+k, v = p.split('=', 1)
+if k.lower() != 'compression':
+outparams.append(p)
+outparams = ' '.join(outparams)
+yield _pack(_fstreamparamsize, len(outparams))
+yield outparams
+else:
+yield _pack(_fstreamparamsize, paramssize)
 # From there, payload might need to be decompressed
 self._fp = self._compengine.decompressorreader(self._fp)
 emptycount = 0



To: joerg.sonnenberger, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] rust: published the three extension crates to crates.io

[Georges Racinet]

# HG changeset patch
# User Georges Racinet 
# Date 1555441784 -7200
#  Tue Apr 16 21:09:44 2019 +0200
# Node ID 410396d5eca793a1b8cf9ee8ba7bce466e9be3c0
# Parent  a362b0b95e42c8f7d46d7e3a0eb4cc531fa5f2d6
# EXP-Topic rust-crates.io
rust: published the three extension crates to crates.io

These are the contents of the first publication for
`hg-core`, `hg-cpython` and direct `hg-direct-ffi`, done with
the sarting point '0.0.1' version.
This version has been chosen to be actually lower than the one
declared up to now in the crates manifests.

In the workspace (rust/Cargo.toml), there's an override so that
if building from the whole source, the local hg-core gets selected.

We will have to select a version policy for the upcoming releases,
with the obvious option to have Rust versions equal to the general
Mercurial version, however partial they are.

It should be noted though that the development version must stay
be higher than the latest that be found on crates.io and that satisfies
the rules in Cargo.lock.
(see 
https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#testing-a-bugfix)

diff -r a362b0b95e42 -r 410396d5eca7 rust/Cargo.lock
--- a/rust/Cargo.lock   Fri Apr 05 14:35:33 2019 +0200
+++ b/rust/Cargo.lock   Tue Apr 16 21:09:44 2019 +0200
@@ -47,7 +47,7 @@
 
 [[package]]
 name = "hg-core"
-version = "0.1.0"
+version = "0.0.1"
 dependencies = [
  "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand_pcg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -55,10 +55,10 @@
 
 [[package]]
 name = "hg-cpython"
-version = "0.1.0"
+version = "0.0.1"
 dependencies = [
  "cpython 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
- "hg-core 0.1.0",
+ "hg-core 0.0.1",
  "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)",
  "python27-sys 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "python3-sys 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -68,7 +68,7 @@
 name = "hgdirectffi"
 version = "0.1.0"
 dependencies = [
- "hg-core 0.1.0",
+ "hg-core 0.0.1",
  "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
diff -r a362b0b95e42 -r 410396d5eca7 rust/Cargo.toml
--- a/rust/Cargo.toml   Fri Apr 05 14:35:33 2019 +0200
+++ b/rust/Cargo.toml   Tue Apr 16 21:09:44 2019 +0200
@@ -1,3 +1,6 @@
 [workspace]
 members = ["hg-core", "hg-direct-ffi", "hg-cpython"]
 exclude = ["chg", "hgcli"]
+
+[patch.crates-io]
+hg-core = {path = "hg-core"}
\ No newline at end of file
diff -r a362b0b95e42 -r 410396d5eca7 rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml   Fri Apr 05 14:35:33 2019 +0200
+++ b/rust/hg-core/Cargo.toml   Tue Apr 16 21:09:44 2019 +0200
@@ -1,12 +1,15 @@
 [package]
 name = "hg-core"
-version = "0.1.0"
-authors = ["Georges Racinet "]
+version = "0.0.1"
+authors = ["Georges Racinet "]
 description = "Mercurial pure Rust core library, with no assumption on Python 
bindings (FFI)"
+license = "GPL-2.0-or-later"
+homepage = "https://mercurial-scm.org";
+repository = "https://www.mercurial-scm.org/repo/hg";
 
 [lib]
 name = "hg"
 
 [dev-dependencies]
-rand = "*"
-rand_pcg = "*"
+rand = "~0.6"
+rand_pcg = "~0.1"
diff -r a362b0b95e42 -r 410396d5eca7 rust/hg-cpython/Cargo.toml
--- a/rust/hg-cpython/Cargo.tomlFri Apr 05 14:35:33 2019 +0200
+++ b/rust/hg-cpython/Cargo.tomlTue Apr 16 21:09:44 2019 +0200
@@ -1,7 +1,11 @@
 [package]
 name = "hg-cpython"
-version = "0.1.0"
-authors = ["Georges Racinet "]
+version = "0.0.1"
+authors = ["Georges Racinet "]
+description = "Mercurial Python bindings using the cpython crate"
+license = "GPL-2.0-or-later"
+homepage = "https://mercurial-scm.org";
+repository = "https://www.mercurial-scm.org/repo/hg";
 
 [lib]
 name='rusthg'
@@ -18,17 +22,17 @@
 python3 = ["python3-sys", "cpython/python3-sys", "cpython/extension-module"]
 
 [dependencies]
-hg-core = { path = "../hg-core" }
-libc = '*'
+hg-core = "0.0.1"
+libc = "^0.2"
 
 [dependencies.cpython]
-version = "*"
+version = "~0.2.1"
 default-features = false
 
 [dependencies.python27-sys]
-version = "0.2.1"
+version = "~0.2.1"
 optional = true
 
 [dependencies.python3-sys]
-version = "0.2.1"
+version = "~0.2.1"
 optional = true
diff -r a362b0b95e42 -r 410396d5eca7 rust/hg-direct-ffi/Cargo.toml
--- a/rust/hg-direct-ffi/Cargo.toml Fri Apr 05 14:35:33 2019 +0200
+++ b/rust/hg-direct-ffi/Cargo.toml Tue Apr 16 21:09:44 2019 +0200
@@ -2,11 +2,14 @@
 name = "hgdirectffi"
 version = "0.1.0"
 authors = ["Georges Racinet "]
-description = "Low level Python bindings for hg-core, going through existing C 
extensions"
+description = "Mercurial low level Python bindings, going through existing C 
extensions"
+license = "GPL-2.0-or-later"
+homepage = "https://mercurial-scm.org";
+repository = "https://www.mercurial-scm.org/repo/hg";
 
 [dependencies]
-libc = "*"
-hg-core = { path = "../hg-core" }
+libc = "^0.2"
+hg-core = "0.0.1"
 
 [lib]
 crate-type = ["staticlib"]

[PATCH 2 of 2] rust: getting back to versions 0.1.0

[Georges Racinet]

# HG changeset patch
# User Georges Racinet 
# Date 1555442327 -7200
#  Tue Apr 16 21:18:47 2019 +0200
# Node ID 9f79564faefa97d9221bd59f913691cf9df9ff64
# Parent  410396d5eca793a1b8cf9ee8ba7bce466e9be3c0
# EXP-Topic rust-crates.io
rust: getting back to versions 0.1.0

This ensures that existing development versions take
precedence over the 0.0.1s I have sent to crates.io

Note about the update process

The [patch] section of the workspace having precedence
on the various manifests, it comes in contradiction
with the precise specification of Cargo.lock.

To solve that, one simply has to do:

  cargo update -p hg-core

and commit the resulting changes in rust/Cargo.lock
I expect this to be part of the post-release routine.

As a side note, the general guidelines say that libraries
should not commit their Cargo.lock, whereas executable binaries should.

  
https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries

We'd like to argue that hg-cpython and hg-direct-ffi are to be considered
binaries, and that for them, that role is fulfilled by the Cargo.lock at the 
workspace
level, which is in version control.

diff -r 410396d5eca7 -r 9f79564faefa rust/Cargo.lock
--- a/rust/Cargo.lock   Tue Apr 16 21:09:44 2019 +0200
+++ b/rust/Cargo.lock   Tue Apr 16 21:18:47 2019 +0200
@@ -48,6 +48,11 @@
 [[package]]
 name = "hg-core"
 version = "0.0.1"
+source = "registry+https://github.com/rust-lang/crates.io-index";
+
+[[package]]
+name = "hg-core"
+version = "0.1.0"
 dependencies = [
  "rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)",
  "rand_pcg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -55,10 +60,10 @@
 
 [[package]]
 name = "hg-cpython"
-version = "0.0.1"
+version = "0.1.0"
 dependencies = [
  "cpython 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
- "hg-core 0.0.1",
+ "hg-core 0.0.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)",
  "python27-sys 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "python3-sys 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
@@ -68,7 +73,7 @@
 name = "hgdirectffi"
 version = "0.1.0"
 dependencies = [
- "hg-core 0.0.1",
+ "hg-core 0.0.1 (registry+https://github.com/rust-lang/crates.io-index)",
  "libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)",
 ]
 
@@ -310,6 +315,7 @@
 "checksum cloudabi 0.0.3 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"ddfc5b9aa5d4507acaf872de71051dfd0e309860e88966e1051e462a077aac4f"
 "checksum cpython 0.2.1 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"b489034e723e7f5109fecd19b719e664f89ef925be785885252469e9822fa940"
 "checksum fuchsia-cprng 0.1.0 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"81f7f8eb465745ea9b02e2704612a9946a59fa40572086c6fd49d6ddcf30bf31"
+"checksum hg-core 0.0.1 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"7fe168efadebadb9da6a329fdc027036e233b662285730cad27220e11e53c384"
 "checksum lazy_static 1.2.0 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"a374c89b9db55895453a74c1e38861d9deec0b01b405a82516e9d5de4820dea1"
 "checksum libc 0.2.45 (registry+https://github.com/rust-lang/crates.io-index)" 
= "2d2857ec59fadc0773853c664d2d18e7198e83883e7060b63c924cb077bd5c74"
 "checksum memchr 2.1.2 
(registry+https://github.com/rust-lang/crates.io-index)" = 
"db4c41318937f6e76648f42826b1d9ade5c09cafb5aef7e351240a70f39206e9"
diff -r 410396d5eca7 -r 9f79564faefa rust/hg-core/Cargo.toml
--- a/rust/hg-core/Cargo.toml   Tue Apr 16 21:09:44 2019 +0200
+++ b/rust/hg-core/Cargo.toml   Tue Apr 16 21:18:47 2019 +0200
@@ -1,6 +1,6 @@
 [package]
 name = "hg-core"
-version = "0.0.1"
+version = "0.1.0"
 authors = ["Georges Racinet "]
 description = "Mercurial pure Rust core library, with no assumption on Python 
bindings (FFI)"
 license = "GPL-2.0-or-later"
diff -r 410396d5eca7 -r 9f79564faefa rust/hg-cpython/Cargo.toml
--- a/rust/hg-cpython/Cargo.tomlTue Apr 16 21:09:44 2019 +0200
+++ b/rust/hg-cpython/Cargo.tomlTue Apr 16 21:18:47 2019 +0200
@@ -1,6 +1,6 @@
 [package]
 name = "hg-cpython"
-version = "0.0.1"
+version = "0.1.0"
 authors = ["Georges Racinet "]
 description = "Mercurial Python bindings using the cpython crate"
 license = "GPL-2.0-or-later"
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D6255: copies: calculate mergecopies() based on pathcopies()

[Pierre-Yves David]

On 4/16/19 9:02 PM, marmoute (Pierre-Yves David) wrote:

marmoute added a comment.


   I did a first path through it, the new code seems reasonable and easier
   to read than the previous one. Some comments and questions below.



The actual comments are visible here:


https://www.mercurial-scm.org/pipermail/mercurial-devel/2019-April/130321.html

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6255: copies: calculate mergecopies() based on pathcopies()

[marmoute (Pierre-Yves David)]

marmoute added a comment.


  I did a first path through it, the new code seems reasonable and easier 
  to read than the previous one. Some comments and questions below.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6255

To: martinvonz, #hg-reviewers
Cc: marmoute, mjpieters, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: D6255: copies: calculate mergecopies() based on pathcopies()

[Pierre-Yves David]

I did a first path through it, the new code seems reasonable and easier 
to read than the previous one. Some comments and questions below.


On 4/16/19 7:19 PM, martinvonz (Martin von Zweigbergk) wrote:

martinvonz created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
   When copies are stored in changesets, we need a changeset-centric
   version of mergecopies() just like we have a changeset-centric version
   of pathcopies(). I think the natural way of thinking about
   mergecopies() is in terms of pathcopies() from the base to each of the
   commits. So if we can rewrite mergecopies() based two such
   pathcopies() calls, we'll get the changeset-centric version for
   free. That's what this patch does.


I like the approach, if I understand it correctly, we'll have less 
duplicated code in the end.


   
   A nice bonus is that it ends up being a lot simpler. mergecopies() has

   accumulated a lot of technical debt over time. One good example is the
   code for dealing with grafts (the "partial/incomplete/dirty"
   stuff). Since pathcopies() already deals with backwards renames and
   ping-pong renames, we get that for free.
   
   I've run tests with hard-coded debug logging for "fullcopy" and while

   I haven't looked at every difference it produces, all the ones I have
   looked at seemed reasonable to me.


How many difference did you had? can you share some example of them with 
their explanation?


on related topic, you seems to be fixing a case in 
test-fastannotate-hg.t and test-annotate-hg.t that is probably worth 
mentioning.



   One drawback of the rewritten code is that we may now make
   remotefilelog prefetch more files. We used to prefetch files that were
   unique to either side of the merge compared to the other. We now
   prefetch files that are unique to either sise of the merge compared to


sise → side


   the base. This means that if you added the same file to each side, we
   would not prefetch it before, but we would now. Such cases are
   probably quite rare, but one likely scenario where they happen is when
   moving from a commit to its successor (or the other way around). The
   user will probably already have the files in the cache in such cases,
   so it's probably not a big deal.
   
   Some timings for calculating mergecopies between two revisions (all

   using the common ancestor as base):


Which revisions did you pick?

(for the record, the benchmark suite use 1daa622bbe42 and 76caed42cf7c)


   In the hg repo:
   4.8 4.8: 0.21s -> 0.21s
   4.0 4.8: 0.35s -> 0.63s
   
   In and old copy of the mozilla-unified repo:

   FIREFOX_BETA_60_BASE^ FIREFOX_BETA_60_BASE: 0.51s -> 0.60s
   FIREFOX_NIGHTLY_59_END FIREFOX_BETA_60_BASE: 2.1s -> 2.3s
   FIREFOX_BETA_59_END FIREFOX_BETA_60_BASE: 3.1s -> 3.3s
   FIREFOX_AURORA_50_BASE FIREFOX_BETA_60_BASE: 30s -> 35s
   
   So it's measurably slower in most cases. Note that merge copies are

   not calculated when updating with a clean working copy, which is
   probably the most common case. I therefore think the much simpler code
   is worth the slowdown.


Do you know where the slowdown comes from ?



REPOSITORY
   rHG Mercurial

REVISION DETAIL
   https://phab.mercurial-scm.org/D6255

AFFECTED FILES
   mercurial/copies.py
   tests/test-annotate.t
   tests/test-fastannotate-hg.t
   tests/test-graft.t
   tests/test-rename-merge2.t

CHANGE DETAILS

diff --git a/tests/test-rename-merge2.t b/tests/test-rename-merge2.t
--- a/tests/test-rename-merge2.t
+++ b/tests/test-rename-merge2.t
@@ -433,6 +433,9 @@
--
test L:nc a b R:up b   W:   - 12 merge b no ancestor
--
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b'
+checking for directory renames
resolving manifests
 branchmerge: True, force: False, partial: False
 ancestor: 924404dff337, local: 86a2aa42fc76+, remote: af30c7647fc7
@@ -469,6 +472,9 @@
--
test L:up b   R:nm a b W:   - 13 merge b no ancestor
--
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b'
+checking for directory renames
resolving manifests
 branchmerge: True, force: False, partial: False
 ancestor: 924404dff337, local: 59318016310c+, remote: bdb19105162a
@@ -506,6 +512,9 @@
--
test L:nc a b R:up a b W:   - 14 merge b no ancestor
--
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b'
+checking for directory renames
resolving manifests
 branchmerge: True, force: False, partial: False
 ancestor: 924404dff337, local: 86a2aa42fc76+, remote: 8dbce441892a
@@ -543,6 +552,9 @@
--
test L:up b   R:nm a b W:   - 15 merge b no ancestor, remove a
--
+all copies found (* = to merge, ! 

[PATCH 2 of 3 V2] repoview: introduce a `experimental.extra-filter-revs` config

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1554565579 -7200
#  Sat Apr 06 17:46:19 2019 +0200
# Node ID 898753ab5830bed2a7e16639a746376fa447577f
# Parent  56e2557a9401a3b3c60956b86860c84fb2e1330a
# EXP-Topic repoview
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
898753ab5830
repoview: introduce a `experimental.extra-filter-revs` config

The option define revisions to additionally filter out of all repository "view".
The end goal is to provide and easy to way to serve multiple subset of the same
repository using multiple "shares".

The simplest use case of this feature is to have one view serving the public
changesets and one view also serving the draft. This is currently achievable
using the new `server.view` option introduced recently by Joerg Sonnenberger.
However, more advanced use cases need more advanced definitions. For example
some needs a view dedicated to some release branches, or view that hides
security fixes to be released. Joerg Sonnenberger and I discussed this topic at
the recent mini-sprint and the both of us have seen real life use cases for
this. (This series got written during the same mini-sprint).

The feature is fully functional, and use similar cache-fallback mechanism to
ensure decent performance. However,there remaining room to ensure each share
caches and hooks collaborate with each others. This will come at a later time
once users start to actually test this feature on real usecase.

diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -529,6 +529,13 @@ coreconfigitem('experimental', 'evolutio
 coreconfigitem('experimental', 'evolution.track-operation',
 default=True,
 )
+# repo-level config to exclude a revset visibility
+#
+# The target use case is to use `share` to expose different subset of the same
+# repository, especially server side. See also `server.view`.
+coreconfigitem('experimental', 'extra-filter-revs',
+default=None,
+)
 coreconfigitem('experimental', 'maxdeltachainspan',
 default=-1,
 )
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1050,6 +1050,8 @@ class localrepository(object):
 # Signature to cached matcher instance.
 self._sparsematchercache = {}
 
+self._extrafilterid = repoview.extrafilter(ui)
+
 def _getvfsward(self, origfunc):
 """build a ward for self.vfs"""
 rref = weakref.ref(self)
@@ -1197,6 +1199,9 @@ class localrepository(object):
 
 In other word, there is always only one level of `repoview` 
"filtering".
 """
+if self._extrafilterid is not None:
+name = name + '%'  + self._extrafilterid
+
 cls = repoview.newtype(self.unfiltered().__class__)
 return cls(self, name, visibilityexceptions)
 
diff --git a/mercurial/repoview.py b/mercurial/repoview.py
--- a/mercurial/repoview.py
+++ b/mercurial/repoview.py
@@ -17,6 +17,10 @@ from . import (
 phases,
 pycompat,
 tags as tagsmod,
+util,
+)
+from .utils import (
+repoviewutil,
 )
 
 def hideablerevs(repo):
@@ -154,6 +158,35 @@ filtertable = {'visible': computehidden,
'immutable':  computemutable,
'base':  computeimpactable}
 
+_basefiltername = list(filtertable)
+
+def extrafilter(ui):
+"""initialize extra filter and return its id
+
+If extra filtering is configured, we make sure the associated filtered view
+are declared and return the associated id.
+"""
+frevs = ui.config('experimental', 'extra-filter-revs')
+if frevs is None:
+return None
+
+fid = util.DIGESTS['sha512'](frevs).hexdigest()
+
+combine = lambda fname: fname + '%' + fid
+
+subsettable = repoviewutil.subsettable
+
+if combine('base') not in filtertable:
+for name in _basefiltername:
+def extrafilteredrevs(repo, *args, **kwargs):
+baserevs = filtertable[name](repo, *args, **kwargs)
+extrarevs = frozenset(repo.revs(frevs))
+return baserevs | extrarevs
+filtertable[combine(name)] = extrafilteredrevs
+if name in subsettable:
+subsettable[combine(name)] = combine(subsettable[name])
+return fid
+
 def filterrevs(repo, filtername, visibilityexceptions=None):
 """returns set of filtered revision for this filter name
 
diff --git a/mercurial/statichttprepo.py b/mercurial/statichttprepo.py
--- a/mercurial/statichttprepo.py
+++ b/mercurial/statichttprepo.py
@@ -155,6 +155,7 @@ class statichttprepository(localrepo.loc
 
 self.names = namespaces.namespaces()
 self.filtername = None
+self._extrafilterid = None
 
 try:
 requirements = set(self.vfs.read(b'requires').splitlines())
diff --git a/tests/test-server-view.t b/tests/test-server-view.

[PATCH 1 of 3 V2] repoview: move subsettable in a dedicated module

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1555076492 -7200
#  Fri Apr 12 15:41:32 2019 +0200
# Node ID 56e2557a9401a3b3c60956b86860c84fb2e1330a
# Parent  2003f0bffcb197699b6665939e21a681f9c73570
# EXP-Topic repoview
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
56e2557a9401
repoview: move subsettable in a dedicated module

The dictionary got moved in `branchmap` to avoid import cycle. However, we are
about to needs it in repoview too. So we introduce a now module to define that
that mapping.

diff --git a/contrib/perf.py b/contrib/perf.py
--- a/contrib/perf.py
+++ b/contrib/perf.py
@@ -94,6 +94,10 @@ try:
 except ImportError:
 pass
 try:
+from mercurial.utils import repoviewutil # since 5.0
+except ImportError:
+repoviewutil = None
+try:
 from mercurial import scmutil # since 1.9 (or 8b252e826c68)
 except ImportError:
 pass
@@ -471,7 +475,8 @@ def getbranchmapsubsettable():
 # subsettable is defined in:
 # - branchmap since 2.9 (or 175c6fd8cacc)
 # - repoview since 2.5 (or 59a9f18d4587)
-for mod in (branchmap, repoview):
+# - repoviewutil since 5.0
+for mod in (branchmap, repoview, repoviewutil):
 subsettable = getattr(mod, 'subsettable', None)
 if subsettable:
 return subsettable
diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -23,28 +23,17 @@ from . import (
 util,
 )
 from .utils import (
+repoviewutil,
 stringutil,
 )
 
+subsettable = repoviewutil. subsettable
+
 calcsize = struct.calcsize
 pack_into = struct.pack_into
 unpack_from = struct.unpack_from
 
 
-### Nearest subset relation
-# Nearest subset of filter X is a filter Y so that:
-# * Y is included in X,
-# * X - Y is as small as possible.
-# This create and ordering used for branchmap purpose.
-# the ordering may be partial
-subsettable = {None: 'visible',
-   'visible-hidden': 'visible',
-   'visible': 'served',
-   'served.hidden': 'served',
-   'served': 'immutable',
-   'immutable': 'base'}
-
-
 class BranchMapCache(object):
 """mapping of filtered views of repo with their branchcache"""
 def __init__(self):
diff --git a/mercurial/repoview.py b/mercurial/repoview.py
--- a/mercurial/repoview.py
+++ b/mercurial/repoview.py
@@ -25,9 +25,9 @@ def hideablerevs(repo):
 This is a standalone function to allow extensions to wrap it.
 
 Because we use the set of immutable changesets as a fallback subset in
-branchmap (see mercurial.branchmap.subsettable), you cannot set "public"
-changesets as "hideable". Doing so would break multiple code assertions and
-lead to crashes."""
+branchmap (see mercurial.utils.repoviewutils.subsettable), you cannot set
+"public" changesets as "hideable". Doing so would break multiple code
+assertions and lead to crashes."""
 obsoletes = obsolete.getrevs(repo, 'obsolete')
 internals = repo._phasecache.getrevset(repo, phases.localhiddenphases)
 internals = frozenset(internals)
@@ -144,7 +144,7 @@ def computeimpactable(repo, visibilityex
 # function to compute filtered set
 #
 # When adding a new filter you MUST update the table at:
-# mercurial.branchmap.subsettable
+# mercurial.utils.repoviewutil.subsettable
 # Otherwise your filter will have to recompute all its branches cache
 # from scratch (very slow).
 filtertable = {'visible': computehidden,
diff --git a/mercurial/utils/repoviewutil.py b/mercurial/utils/repoviewutil.py
new file mode 100644
--- /dev/null
+++ b/mercurial/utils/repoviewutil.py
@@ -0,0 +1,22 @@
+# repoviewutil.py - constaints data relevant to repoview.py and other module
+#
+# Copyright 2012 Pierre-Yves David 
+#Logilab SA
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import
+
+### Nearest subset relation
+# Nearest subset of filter X is a filter Y so that:
+# * Y is included in X,
+# * X - Y is as small as possible.
+# This create and ordering used for branchmap purpose.
+# the ordering may be partial
+subsettable = {None: 'visible',
+   'visible-hidden': 'visible',
+   'visible': 'served',
+   'served.hidden': 'served',
+   'served': 'immutable',
+   'immutable': 'base'}
diff --git a/tests/check-perf-code.py b/tests/check-perf-code.py
--- a/tests/check-perf-code.py
+++ b/tests/check-perf-code.py
@@ -10,7 +10,7 @@ import sys
 # write static check patterns here
 perfpypats = [
   [
-(r'(branchmap|repoview)\.subsettable',
+(r'(branchmap|repoview|repoviewutil)\.subsettable',
  "use getbranchmapsubsettable() for early Mercurial"),
 (r'\.(vfs|svfs|opener|sopener)',
  "use getvfs()/getsvfs() for early M

[PATCH 3 of 3 V2] repoview: flag `server.view` as experimental

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1555079159 -7200
#  Fri Apr 12 16:25:59 2019 +0200
# Node ID 87d373ff578d10b2c2e99205add8c4c960102a94
# Parent  898753ab5830bed2a7e16639a746376fa447577f
# EXP-Topic repoview
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
87d373ff578d
repoview: flag `server.view` as experimental

Ideally, the non-experimental version of `experimental.extra-filter-revs` will
cover the use case for `server.view=immutable` well enough than having to have
this dedicated configuration. Since `server.view` is not part of any release, I
would prefer to have it marked as experimental to avoid having it to support it
for ever.

diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt
--- a/mercurial/help/config.txt
+++ b/mercurial/help/config.txt
@@ -2050,7 +2050,8 @@ Controls generic server settings.
 Repository filter used when exchanging revisions with the peer.
 
 The default view (``served``) excludes secret and hidden changesets.
-Another useful value is ``immutable`` (no draft, secret or hidden 
changesets).
+Another useful value is ``immutable`` (no draft, secret or hidden
+changesets). (EXPERIMENTAL)
 
 ``smtp``
 
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 2] debugdiscovery: include the number of heads in all sets

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1555422620 -7200
#  Tue Apr 16 15:50:20 2019 +0200
# Node ID 017778a4463a8e6ecb4b17cacf46a3ab27bdb239
# Parent  967c098eed3398d991cbd7f0d48636b575beb711
# EXP-Topic discovery-speedup
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
017778a4463a
debugdiscovery: include the number of heads in all sets

We already displayed information about heads of the common set that are either
local or remote heads. We now also do so for heads of the common set that are
both local and remote heads too. This is useful because various step in the
set discovery algorithm have head specific optimizations.

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -829,6 +829,7 @@ def debugdiscovery(ui, repo, remoteurl="
 data['nb-common'] = len(common)
 data['nb-common-local'] = len(common & lheads)
 data['nb-common-remote'] = len(common & rheads)
+data['nb-common-both'] = len(common & rheads & lheads)
 data['nb-local'] = len(lheads)
 data['nb-local-missing'] = data['nb-local'] - data['nb-common-local']
 data['nb-remote'] = len(rheads)
@@ -843,6 +844,7 @@ def debugdiscovery(ui, repo, remoteurl="
 ui.write(("  total common heads:  %(nb-common)9d\n") % data)
 ui.write(("also local heads:  %(nb-common-local)9d\n") % data)
 ui.write(("also remote heads: %(nb-common-remote)9d\n") % data)
+ui.write(("both:  %(nb-common-both)9d\n") % data)
 ui.write(("  local heads: %(nb-local)9d\n") % data)
 ui.write(("common:%(nb-common-local)9d\n") % data)
 ui.write(("missing:   %(nb-local-missing)9d\n") % data)
diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t
--- a/tests/test-setdiscovery.t
+++ b/tests/test-setdiscovery.t
@@ -48,6 +48,7 @@ Small superset:
 total common heads:  2
   also local heads:  2
   also remote heads: 1
+  both:  1
 local heads: 2
   common:2
   missing:   0
@@ -69,6 +70,7 @@ Small superset:
 total common heads:  2
   also local heads:  2
   also remote heads: 1
+  both:  1
 local heads: 2
   common:2
   missing:   0
@@ -90,6 +92,7 @@ Small superset:
 total common heads:  1
   also local heads:  1
   also remote heads: 0
+  both:  0
 local heads: 2
   common:1
   missing:   1
@@ -110,6 +113,7 @@ Small superset:
 total common heads:  2
   also local heads:  1
   also remote heads: 2
+  both:  1
 local heads: 3
   common:1
   missing:   2
@@ -131,6 +135,7 @@ Small superset:
 total common heads:  2
   also local heads:  1
   also remote heads: 2
+  both:  1
 local heads: 3
   common:1
   missing:   2
@@ -152,6 +157,7 @@ Small superset:
 total common heads:  2
   also local heads:  1
   also remote heads: 2
+  both:  1
 local heads: 3
   common:1
   missing:   2
@@ -179,6 +185,7 @@ Many new:
 total common heads:  1
   also local heads:  1
   also remote heads: 0
+  both:  0
 local heads: 2
   common:1
   missing:   1
@@ -203,6 +210,7 @@ Many new:
 total common heads:  1
   also local heads:  1
   also remote heads: 0
+  both:  0
 local heads: 2
   common:1
   missing:   1
@@ -227,6 +235,7 @@ Many new:
 total common heads:  1
   also local heads:  0
   also remote heads: 0
+  both:  0
 local heads: 2
   common:0
   missing:   2
@@ -247,6 +256,7 @@ Many new:
 total common heads:  1
   also local heads:  0
   also remote heads: 1
+  both:  0
 local heads: 1
   common:0
   missing:   1
@@ -271,6 +281,7 @@ Many new:
 total common heads:  1
   also local heads:  0
   also remote heads: 1
+  both:  0
 l

[PATCH 2 of 2] setdiscovery: stop limiting the number of local head we initially send

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1555428398 -7200
#  Tue Apr 16 17:26:38 2019 +0200
# Node ID ffaa98def33a903f132ec4177d36823a741b6ef6
# Parent  017778a4463a8e6ecb4b17cacf46a3ab27bdb239
# EXP-Topic discovery-speedup
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
ffaa98def33a
setdiscovery: stop limiting the number of local head we initially send

In our testing this limitation provides now real gain and instead triggers
pathological discovery timing for some repository with many heads.

See inline documentation for details.

Some timing below:

Mozilla try repository, (~1M revs, ~35K heads), discovery between 2 clones with
100 head missing on each side

before:
! wall 1.492111 comb 1.49 user 1.45 sys 0.04 (best of 20)
! wall 1.813992 comb 1.82 user 1.70 sys 0.12 (max of 20)
! wall 1.574326 comb 1.573500 user 1.522000 sys 0.051500 (avg of 20)
! wall 1.572583 comb 1.57 user 1.52 sys 0.05 (median of 20)

after:
! wall 1.147834 comb 1.15 user 1.09 sys 0.06 (best of 20)
! wall 1.449144 comb 1.45 user 1.33 sys 0.12 (max of 20)
! wall 1.204618 comb 1.202500 user 1.146500 sys 0.056000 (avg of 20)
! wall 1.194407 comb 1.19 user 1.14 sys 0.05 (median of 20)


pypy (~100 heads, 317 heads) discovery between clones with only 42 common heads

before:
! wall 0.031653 comb 0.03 user 0.03 sys 0.00 (best of 25)
! wall 0.055719 comb 0.05 user 0.04 sys 0.01 (max of 25)
! wall 0.038939 comb 0.039600 user 0.038400 sys 0.001200 (avg of 25)
! wall 0.038660 comb 0.05 user 0.04 sys 0.01 (median of 25)

after:
! wall 0.018754 comb 0.02 user 0.02 sys 0.00 (best of 49)
! wall 0.034505 comb 0.04 user 0.03 sys 0.01 (max of 49)
! wall 0.019631 comb 0.019796 user 0.018367 sys 0.001429 (avg of 49)
! wall 0.019132 comb 0.02 user 0.02 sys 0.00 (median of 49)


Private repository (~1M revs, ~3K heads), discovery from a strip subset, about
100 changesets to be pulled.

before:
! wall 1.837729 comb 1.84 user 1.79 sys 0.05 (best of 20)
! wall 2.203468 comb 2.20 user 2.10 sys 0.10 (max of 20)
! wall 2.049355 comb 2.048500 user 2.002500 sys 0.046000 (avg of 20)
! wall 2.035315 comb 2.04 user 2.00 sys 0.04 (median of 20)

after:
! wall 0.136598 comb 0.13 user 0.11 sys 0.02 (best of 20)
! wall 0.330519 comb 0.33 user 0.26 sys 0.07 (max of 20)
! wall 0.157254 comb 0.155500 user 0.123000 sys 0.032500 (avg of 20)
! wall 0.149870 comb 0.14 user 0.11 sys 0.03 (median of 20)


Same private repo, discovery between two clone with 500 different heads on each
side:

before:
! wall 2.372919 comb 2.37 user 2.32 sys 0.05 (best of 20)
! wall 2.622422 comb 2.61 user 2.51 sys 0.10 (max of 20)
! wall 2.450135 comb 2.45 user 2.402000 sys 0.048000 (avg of 20)
! wall 2.443896 comb 2.45 user 2.41 sys 0.04 (median of 20)

after:
! wall 0.625497 comb 0.62 user 0.57 sys 0.05 (best of 20)
! wall 0.834723 comb 0.82 user 0.73 sys 0.09 (max of 20)
! wall 0.675725 comb 0.675500 user 0.628000 sys 0.047500 (avg of 20)
! wall 0.671614 comb 0.68 user 0.64 sys 0.04 (median of 20)

diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py
--- a/mercurial/setdiscovery.py
+++ b/mercurial/setdiscovery.py
@@ -275,9 +275,58 @@ def findcommonheads(ui, local, remote,
 # early exit if we know all the specified remote heads already
 ui.debug("query 1; heads\n")
 roundtrips += 1
-sample = _limitsample(ownheads, initialsamplesize)
-# indices between sample and externalized version must match
-sample = list(sample)
+# We also ask remote about all the local heads. That set can be arbitrarily
+# large, so we used to limit it size to `initialsamplesize`. We no longer
+# do as it proved counter productive. The skipped heads could lead to a
+# large "undecided" set, slower to be clarified than if we asked the
+# question for all heads right away.
+#
+# We are already fetching all server heads using the `heads` commands,
+# sending a equivalent number of heads the other way should not have a
+# significant impact.  In addition, it is very likely that we are going to
+# have to issue "known" request for an equivalent amount of revisions in
+# order to decide if theses heads are common or missing.
+#
+# find a detailled analysis below.
+#
+# Case A: local and server both has few heads
+#
+# Ownheads is below initialsamplesize, limit would not have any effect.
+#
+# Case B: local has few heads and server has many
+#
+# Ownheads is below initialsamplesize, limit would not have any effect.
+#
+# Case C: local and server both has many heads
+#
+# We now transfert some

D6256: copies: inline _computenonoverlap() in mergecopies()

[martinvonz (Martin von Zweigbergk)]

martinvonz updated this revision to Diff 14785.
martinvonz edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6256?vs=14784&id=14785

REVISION DETAIL
  https://phab.mercurial-scm.org/D6256

AFFECTED FILES
  hgext/remotefilelog/__init__.py
  mercurial/copies.py
  tests/test-remotefilelog-sparse.t

CHANGE DETAILS

diff --git a/tests/test-remotefilelog-sparse.t 
b/tests/test-remotefilelog-sparse.t
--- a/tests/test-remotefilelog-sparse.t
+++ b/tests/test-remotefilelog-sparse.t
@@ -98,12 +98,5 @@
   $ clearcache
   $ hg prefetch -r '. + .^' -I x -I z
   4 files fetched over 1 fetches - (4 misses, 0.00% hit ratio) over * (glob)
-Originally this was testing that the rebase doesn't fetch pointless
-blobs. Right now it fails because core's sparse can't load a spec from
-the working directory. Presumably there's a fix, but I'm not sure what it is.
   $ hg rebase -d 2 --keep
   rebasing 1:876b1317060d "x2" (foo)
-  transaction abort!
-  rollback completed
-  abort: cannot parse sparse patterns from working directory
-  [255]
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -353,26 +353,6 @@
 return _chain(x, y, _backwardrenames(x, a, match=match),
   _forwardcopies(a, y, match=match))
 
-def _computenonoverlap(repo, c1, c2, addedinm1, addedinm2, debug=True):
-"""Computes, based on addedinm1 and addedinm2, the files exclusive to c1
-and c2. This is its own function so extensions can easily wrap this call
-to see what files mergecopies is about to process.
-
-Even though c1 and c2 are not used in this function, they are useful in
-other extensions for being able to read the file nodes of the changed 
files.
-"""
-u1 = sorted(addedinm1 - addedinm2)
-u2 = sorted(addedinm2 - addedinm1)
-
-if debug:
-header = "  unmatched files in %s"
-if u1:
-repo.ui.debug("%s:\n   %s\n" % (header % 'local', "\n   
".join(u1)))
-if u2:
-repo.ui.debug("%s:\n   %s\n" % (header % 'other', "\n   
".join(u2)))
-
-return u1, u2
-
 def mergecopies(repo, c1, c2, base):
 """
 Finds moves and copies between context c1 and c2 that are relevant for
@@ -558,7 +538,14 @@
 # find interesting file sets from manifests
 addedinm1 = m1.filesnotin(mb, repo.narrowmatch())
 addedinm2 = m2.filesnotin(mb, repo.narrowmatch())
-u1, u2 = _computenonoverlap(repo, c1, c2, addedinm1, addedinm2)
+u1 = sorted(addedinm1 - addedinm2)
+u2 = sorted(addedinm2 - addedinm1)
+
+header = "  unmatched files in %s"
+if u1:
+repo.ui.debug("%s:\n   %s\n" % (header % 'local', "\n   ".join(u1)))
+if u2:
+repo.ui.debug("%s:\n   %s\n" % (header % 'other', "\n   ".join(u2)))
 
 fullcopy = copies1.copy()
 fullcopy.update(copies2)
diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -487,37 +487,6 @@
 return orig(repo, matcher, added, removed, *args, **kwargs)
 extensions.wrapfunction(scmutil, '_findrenames', findrenames)
 
-# prefetch files before mergecopies check
-def computenonoverlap(orig, repo, c1, c2, *args, **kwargs):
-u1, u2 = orig(repo, c1, c2, *args, **kwargs)
-if isenabled(repo):
-m1 = c1.manifest()
-m2 = c2.manifest()
-files = []
-
-sparsematch1 = repo.maybesparsematch(c1.rev())
-if sparsematch1:
-sparseu1 = set()
-for f in u1:
-if sparsematch1(f):
-files.append((f, hex(m1[f])))
-sparseu1.add(f)
-u1 = sparseu1
-
-sparsematch2 = repo.maybesparsematch(c2.rev())
-if sparsematch2:
-sparseu2 = set()
-for f in u2:
-if sparsematch2(f):
-files.append((f, hex(m2[f])))
-sparseu2.add(f)
-u2 = sparseu2
-
-# batch fetch the needed files from the server
-repo.fileservice.prefetch(files)
-return u1, u2
-extensions.wrapfunction(copies, '_computenonoverlap', computenonoverlap)
-
 # prefetch files before pathcopies check
 def computeforwardmissing(orig, a, b, match=None):
 missing = orig(a, b, match=match)



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6249: copies: delete debug message about search limit

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  I'm about to rewrite mergecopies() and this message will no longer be
  emitted then. Let's remove the message now to remove a distraction
  from that patch.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6249

AFFECTED FILES
  mercurial/copies.py
  tests/test-copy-move-merge.t
  tests/test-double-merge.t
  tests/test-graft.t
  tests/test-issue1802.t
  tests/test-issue522.t
  tests/test-issue672.t
  tests/test-merge-commit.t
  tests/test-merge-criss-cross.t
  tests/test-merge-types.t
  tests/test-merge7.t
  tests/test-rebase-conflicts.t
  tests/test-rename-dir-merge.t
  tests/test-rename-merge1.t
  tests/test-rename-merge2.t
  tests/test-subrepo.t
  tests/test-up-local-change.t

CHANGE DETAILS

diff --git a/tests/test-up-local-change.t b/tests/test-up-local-change.t
--- a/tests/test-up-local-change.t
+++ b/tests/test-up-local-change.t
@@ -40,7 +40,6 @@
   summary: 1
   
   $ hg --debug up
-searching for copies back to rev 1
 unmatched files in other:
  b
   resolving manifests
@@ -68,7 +67,6 @@
   
   $ hg --debug up 0
   starting 4 threads for background file closing (?)
-searching for copies back to rev 0
 unmatched files in local (from topological common ancestor):
  b
   resolving manifests
@@ -95,7 +93,6 @@
   summary: 1
   
   $ hg --debug up
-searching for copies back to rev 1
 unmatched files in other:
  b
   resolving manifests
diff --git a/tests/test-subrepo.t b/tests/test-subrepo.t
--- a/tests/test-subrepo.t
+++ b/tests/test-subrepo.t
@@ -274,7 +274,6 @@
   $ hg ci -m9
   created new head
   $ hg merge 6 --debug # test change
-searching for copies back to rev 2
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 1f14a2e2d3ec, local: f0d2028bf86d+, remote: 1831e14459c4
@@ -301,7 +300,6 @@
   $ hg ci -m10
   committing subrepository t
   $ HGMERGE=internal:merge hg merge --debug 7 # test conflict
-searching for copies back to rev 2
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 1831e14459c4, local: e45c8b14af55+, remote: f94576341bcf
@@ -313,7 +311,6 @@
   starting 4 threads for background file closing (?)
   (M)erge, keep (l)ocal [working copy] or keep (r)emote [merge rev]? m
   merging subrepository "t"
-searching for copies back to rev 2
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 6747d179aa9a, local: 20a0db6fbf6c+, remote: 7af322bc1198
diff --git a/tests/test-rename-merge2.t b/tests/test-rename-merge2.t
--- a/tests/test-rename-merge2.t
+++ b/tests/test-rename-merge2.t
@@ -77,7 +77,6 @@
   --
   test L:up a   R:nc a b W:   - 1  get local a to b
   --
-searching for copies back to rev 1
 unmatched files in other:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
@@ -116,7 +115,6 @@
   --
   test L:nc a b R:up a   W:   - 2  get rem change to a and b
   --
-searching for copies back to rev 1
 unmatched files in local:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
@@ -156,7 +154,6 @@
   --
   test L:up a   R:nm a b W:   - 3  get local a change to b, remove a
   --
-searching for copies back to rev 1
 unmatched files in other:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
@@ -195,7 +192,6 @@
   --
   test L:nm a b R:up a   W:   - 4  get remote change to b
   --
-searching for copies back to rev 1
 unmatched files in local:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
@@ -233,7 +229,6 @@
   --
   test L:   R:nc a b W:   - 5  get b
   --
-searching for copies back to rev 1
 unmatched files in other:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
@@ -266,7 +261,6 @@
   --
   test L:nc a b R:   W:   - 6  nothing
   --
-searching for copies back to rev 1
 unmatched files in local:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
@@ -298,7 +292,6 @@
   --
   test L:   R:nm a b W:   - 7  get b
   --
-searching for copies back to rev 1
 unmatched files in other:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
@@ -332,7 +325,6 @@
   --
   test L:nm a b R:   W:   - 8  nothing
   --
-searching for copies back to rev 1
 unmatched files in local:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
@@ -363,7 +355,6 @@
   --
   test L:um a b R:um a b W:   - 9

[PATCH] chistedit: use context manager to set verbose ui

[Jordi Gutiérrez Hermoso]

# HG changeset patch
# User Jordi Gutiérrez Hermoso 
# Date 1555434741 14400
#  Tue Apr 16 13:12:21 2019 -0400
# Node ID f64bbdc70aaa1ca8c504c2d909f6012927a27cb8
# Parent  a362b0b95e42c8f7d46d7e3a0eb4cc531fa5f2d6
chistedit: use context manager to set verbose ui

I'm still not exactly sure why this is necessary -- perhaps setting it
unconditionally would leak this setting in chg invocations.
Regardless, this would have looked very out of place as compared to
how this setting is done everywhere else, so at least for the sake of
style, let's be consistent with the rest of the codebase.

diff --git a/hgext/histedit.py b/hgext/histedit.py
--- a/hgext/histedit.py
+++ b/hgext/histedit.py
@@ -1230,12 +1230,13 @@ def addln(win, y, x, line, color=None):
 def patchcontents(state):
 repo = state['repo']
 rule = state['rules'][state['pos']]
-repo.ui.verbose = True
 displayer = logcmdutil.changesetdisplayer(repo.ui, repo, {
 "patch": True,  "template": "status"
 }, buffered=True)
-displayer.show(rule.ctx)
-displayer.close()
+overrides = {('ui',  'verbose'): True}
+with repo.ui.configoverride(overrides, source='histedit'):
+displayer.show(rule.ctx)
+displayer.close()
 return displayer.hunk[rule.ctx.rev()].splitlines()
 
 def _chisteditmain(repo, rules, stdscr):
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6252: tests: make merge conflicts explicit in `hg annotate` tests

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We were using `true` as merge tool. I think it makes the test easier
  to understand if we make the conflicts explcit. It also papered over a
  conflict that shouldn't have been a conflict (just a bug in copy
  tracing). I've marked that "BROKEN".

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6252

AFFECTED FILES
  tests/test-annotate.t
  tests/test-fastannotate-hg.t
  tests/test-fastannotate-perfhack.t
  tests/test-fastannotate-protocol.t
  tests/test-fastannotate.t

CHANGE DETAILS

diff --git a/tests/test-fastannotate.t b/tests/test-fastannotate.t
--- a/tests/test-fastannotate.t
+++ b/tests/test-fastannotate.t
@@ -3,8 +3,6 @@
   > fastannotate=
   > EOF
 
-  $ HGMERGE=true; export HGMERGE
-
   $ hg init repo
   $ cd repo
 
diff --git a/tests/test-fastannotate-protocol.t 
b/tests/test-fastannotate-protocol.t
--- a/tests/test-fastannotate-protocol.t
+++ b/tests/test-fastannotate-protocol.t
@@ -7,8 +7,6 @@
   > mainbranch=@
   > EOF
 
-  $ HGMERGE=true; export HGMERGE
-
 setup the server repo
 
   $ hg init repo-server
diff --git a/tests/test-fastannotate-perfhack.t 
b/tests/test-fastannotate-perfhack.t
--- a/tests/test-fastannotate-perfhack.t
+++ b/tests/test-fastannotate-perfhack.t
@@ -5,8 +5,6 @@
   > perfhack=1
   > EOF
 
-  $ HGMERGE=true; export HGMERGE
-
   $ hg init repo
   $ cd repo
 
diff --git a/tests/test-fastannotate-hg.t b/tests/test-fastannotate-hg.t
--- a/tests/test-fastannotate-hg.t
+++ b/tests/test-fastannotate-hg.t
@@ -1,6 +1,8 @@
 (this file is backported from core hg tests/test-annotate.t)
 
   $ cat >> $HGRCPATH << EOF
+  > [ui]
+  > merge = :merge3
   > [diff]
   > git=1
   > [extensions]
@@ -11,8 +13,6 @@
   > mainbranch=.
   > EOF
 
-  $ HGMERGE=true; export HGMERGE
-
 init
 
   $ hg init repo
@@ -157,8 +157,34 @@
   created new head
   $ hg merge
   merging b
-  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
-  (branch merge, don't forget to commit)
+  warning: conflicts while merging b! (edit, then use 'hg resolve --mark')
+  0 files updated, 0 files merged, 0 files removed, 1 files unresolved
+  use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to 
abandon
+  [1]
+  $ cat b
+  a
+  a
+  a
+  <<< working copy: 5fbdc1152d97 - test: b2.1
+  b4
+  c
+  b5
+  ||| base
+  ===
+  b4
+  b5
+  b6
+  >>> merge rev:37ec9f5c3d1f - test: b2
+  $ cat < b
+  > a
+  > a
+  > a
+  > b4
+  > c
+  > b5
+  > EOF
+  $ hg resolve --mark -q
+  $ rm b.orig
   $ hg ci -mmergeb -d '3 0'
 
 annotate after merge
@@ -247,15 +273,37 @@
   > EOF
   $ hg ci -mc -d '3 0'
   created new head
+BROKEN: 'a' was copied to 'b' on both sides. We should not get a merge 
conflict here
   $ hg merge
   merging b
-  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
-  (branch merge, don't forget to commit)
-  $ cat <> b
+  warning: conflicts while merging b! (edit, then use 'hg resolve --mark')
+  0 files updated, 0 files merged, 0 files removed, 1 files unresolved
+  use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to 
abandon
+  [1]
+  $ cat b
+  <<< working copy: b80e3e32f75a - test: c
+  a
+  z
+  a
+  ||| base
+  ===
+  a
+  a
+  a
+  b4
+  c
+  b5
+  >>> merge rev:64afcdf8e29e - test: mergeb
+  $ cat < b
+  > a
+  > z
+  > a
   > b4
   > c
   > b5
   > EOF
+  $ hg resolve --mark -q
+  $ rm b.orig
   $ echo d >> b
   $ hg ci -mmerge2 -d '4 0'
 
@@ -745,14 +793,28 @@
   $ echo 3 >> a
   $ hg commit -m 3 -q
   $ hg merge 2 -q
+  warning: conflicts while merging a! (edit, then use 'hg resolve --mark')
+  [1]
+  $ cat a
+  <<< working copy: 0a068f0261cf - test: 3
+  1
+  2
+  3
+  ||| base
+  1
+  2
+  ===
+  a
+  >>> merge rev:9409851bc20a - test: a
   $ cat > a << EOF
   > b
   > 1
   > 2
   > 3
   > a
   > EOF
   $ hg resolve --mark -q
+  $ rm a.orig
   $ hg commit -m m
   $ hg annotate a
   4: b
diff --git a/tests/test-annotate.t b/tests/test-annotate.t
--- a/tests/test-annotate.t
+++ b/tests/test-annotate.t
@@ -1,4 +1,7 @@
-  $ HGMERGE=true; export HGMERGE
+  $ cat >> "$HGRCPATH" << EOF
+  > [ui]
+  > merge = :merge3
+  > EOF
 
 init
 
@@ -210,8 +213,34 @@
   created new head
   $ hg merge
   merging b
-  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
-  (branch merge, don't forget to commit)
+  warning: conflicts while merging b! (edit, then use 'hg resolve --mark')
+  0 files updated, 0 files merged, 0 files removed, 1 files unresolved
+  use 'hg resolve' to retry unresolved file merges or 'hg merge --abort' to 
abandon
+  [1]
+  $ cat b
+  a
+  a
+  a
+  <<< working copy: 5fbdc1152d97 - test: b2.1
+  b4
+  c
+  b5
+  ||| base
+  ===
+  b4
+  b5
+  b6
+  >>> merge rev:37ec9f5c3d1f - test: b2
+  $ cat < b
+  > a
+  > a
+  > a
+  > b4
+  > c
+  > b5
+  > EOF
+  $ hg r

D6255: copies: calculate mergecopies() based on pathcopies()

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  When copies are stored in changesets, we need a changeset-centric
  version of mergecopies() just like we have a changeset-centric version
  of pathcopies(). I think the natural way of thinking about
  mergecopies() is in terms of pathcopies() from the base to each of the
  commits. So if we can rewrite mergecopies() based two such
  pathcopies() calls, we'll get the changeset-centric version for
  free. That's what this patch does.
  
  A nice bonus is that it ends up being a lot simpler. mergecopies() has
  accumulated a lot of technical debt over time. One good example is the
  code for dealing with grafts (the "partial/incomplete/dirty"
  stuff). Since pathcopies() already deals with backwards renames and
  ping-pong renames, we get that for free.
  
  I've run tests with hard-coded debug logging for "fullcopy" and while
  I haven't looked at every difference it produces, all the ones I have
  looked at seemed reasonable to me.
  
  One drawback of the rewritten code is that we may now make
  remotefilelog prefetch more files. We used to prefetch files that were
  unique to either side of the merge compared to the other. We now
  prefetch files that are unique to either sise of the merge compared to
  the base. This means that if you added the same file to each side, we
  would not prefetch it before, but we would now. Such cases are
  probably quite rare, but one likely scenario where they happen is when
  moving from a commit to its successor (or the other way around). The
  user will probably already have the files in the cache in such cases,
  so it's probably not a big deal.
  
  Some timings for calculating mergecopies between two revisions (all
  using the common ancestor as base):
  
  In the hg repo:
  4.8 4.8: 0.21s -> 0.21s
  4.0 4.8: 0.35s -> 0.63s
  
  In and old copy of the mozilla-unified repo:
  FIREFOX_BETA_60_BASE^ FIREFOX_BETA_60_BASE: 0.51s -> 0.60s
  FIREFOX_NIGHTLY_59_END FIREFOX_BETA_60_BASE: 2.1s -> 2.3s
  FIREFOX_BETA_59_END FIREFOX_BETA_60_BASE: 3.1s -> 3.3s
  FIREFOX_AURORA_50_BASE FIREFOX_BETA_60_BASE: 30s -> 35s
  
  So it's measurably slower in most cases. Note that merge copies are
  not calculated when updating with a clean working copy, which is
  probably the most common case. I therefore think the much simpler code
  is worth the slowdown.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6255

AFFECTED FILES
  mercurial/copies.py
  tests/test-annotate.t
  tests/test-fastannotate-hg.t
  tests/test-graft.t
  tests/test-rename-merge2.t

CHANGE DETAILS

diff --git a/tests/test-rename-merge2.t b/tests/test-rename-merge2.t
--- a/tests/test-rename-merge2.t
+++ b/tests/test-rename-merge2.t
@@ -433,6 +433,9 @@
   --
   test L:nc a b R:up b   W:   - 12 merge b no ancestor
   --
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b' 
+checking for directory renames
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 86a2aa42fc76+, remote: af30c7647fc7
@@ -469,6 +472,9 @@
   --
   test L:up b   R:nm a b W:   - 13 merge b no ancestor
   --
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b' 
+checking for directory renames
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 59318016310c+, remote: bdb19105162a
@@ -506,6 +512,9 @@
   --
   test L:nc a b R:up a b W:   - 14 merge b no ancestor
   --
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b' 
+checking for directory renames
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 86a2aa42fc76+, remote: 8dbce441892a
@@ -543,6 +552,9 @@
   --
   test L:up b   R:nm a b W:   - 15 merge b no ancestor, remove a
   --
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b' 
+checking for directory renames
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 59318016310c+, remote: bdb19105162a
@@ -580,6 +592,9 @@
   --
   test L:nc a b R:up a b W:   - 16 get a, merge b no ancestor
   --
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b' 
+checking for directory renames
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 86a2aa42fc76+, remote: 8dbce441892a
@@ -617,6 +632,9 @@
   --
   test L:up a b R:nc a b W:   - 17 keep a, merge b no ancestor
   

D6256: copies: inline _computenonoverlap() in mergecopies()

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We now call pathcopies() from the base to each of the commits, and
  that calls _computeforwardmissing(), which does file prefetching (in
  the remotefilelog override). So the call to _computenonoverlap() is
  now pointless (the sets of files from _computenonoverlap() are subsets
  of the sets of files from _computeforwardmissing()).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6256

AFFECTED FILES
  hgext/remotefilelog/__init__.py
  mercurial/copies.py

CHANGE DETAILS

diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -353,26 +353,6 @@
 return _chain(x, y, _backwardrenames(x, a, match=match),
   _forwardcopies(a, y, match=match))
 
-def _computenonoverlap(repo, c1, c2, addedinm1, addedinm2, debug=True):
-"""Computes, based on addedinm1 and addedinm2, the files exclusive to c1
-and c2. This is its own function so extensions can easily wrap this call
-to see what files mergecopies is about to process.
-
-Even though c1 and c2 are not used in this function, they are useful in
-other extensions for being able to read the file nodes of the changed 
files.
-"""
-u1 = sorted(addedinm1 - addedinm2)
-u2 = sorted(addedinm2 - addedinm1)
-
-if debug:
-header = "  unmatched files in %s"
-if u1:
-repo.ui.debug("%s:\n   %s\n" % (header % 'local', "\n   
".join(u1)))
-if u2:
-repo.ui.debug("%s:\n   %s\n" % (header % 'other', "\n   
".join(u2)))
-
-return u1, u2
-
 def mergecopies(repo, c1, c2, base):
 """
 Finds moves and copies between context c1 and c2 that are relevant for
@@ -558,7 +538,14 @@
 # find interesting file sets from manifests
 addedinm1 = m1.filesnotin(mb, repo.narrowmatch())
 addedinm2 = m2.filesnotin(mb, repo.narrowmatch())
-u1, u2 = _computenonoverlap(repo, c1, c2, addedinm1, addedinm2)
+u1 = sorted(addedinm1 - addedinm2)
+u2 = sorted(addedinm2 - addedinm1)
+
+header = "  unmatched files in %s"
+if u1:
+repo.ui.debug("%s:\n   %s\n" % (header % 'local', "\n   ".join(u1)))
+if u2:
+repo.ui.debug("%s:\n   %s\n" % (header % 'other', "\n   ".join(u2)))
 
 fullcopy = copies1.copy()
 fullcopy.update(copies2)
diff --git a/hgext/remotefilelog/__init__.py b/hgext/remotefilelog/__init__.py
--- a/hgext/remotefilelog/__init__.py
+++ b/hgext/remotefilelog/__init__.py
@@ -487,37 +487,6 @@
 return orig(repo, matcher, added, removed, *args, **kwargs)
 extensions.wrapfunction(scmutil, '_findrenames', findrenames)
 
-# prefetch files before mergecopies check
-def computenonoverlap(orig, repo, c1, c2, *args, **kwargs):
-u1, u2 = orig(repo, c1, c2, *args, **kwargs)
-if isenabled(repo):
-m1 = c1.manifest()
-m2 = c2.manifest()
-files = []
-
-sparsematch1 = repo.maybesparsematch(c1.rev())
-if sparsematch1:
-sparseu1 = set()
-for f in u1:
-if sparsematch1(f):
-files.append((f, hex(m1[f])))
-sparseu1.add(f)
-u1 = sparseu1
-
-sparsematch2 = repo.maybesparsematch(c2.rev())
-if sparsematch2:
-sparseu2 = set()
-for f in u2:
-if sparsematch2(f):
-files.append((f, hex(m2[f])))
-sparseu2.add(f)
-u2 = sparseu2
-
-# batch fetch the needed files from the server
-repo.fileservice.prefetch(files)
-return u1, u2
-extensions.wrapfunction(copies, '_computenonoverlap', computenonoverlap)
-
 # prefetch files before pathcopies check
 def computeforwardmissing(orig, a, b, match=None):
 missing = orig(a, b, match=match)



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6254: tests: avoid a rename/delete conflict when updating in test-narrow-update.t

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  After the upcoming rewrite of mergecopies(), this test would otherwise
  (accurately) start warning about "inside/f1 was deleted and renamed".

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6254

AFFECTED FILES
  tests/test-narrow-update.t

CHANGE DETAILS

diff --git a/tests/test-narrow-update.t b/tests/test-narrow-update.t
--- a/tests/test-narrow-update.t
+++ b/tests/test-narrow-update.t
@@ -72,5 +72,5 @@
 
   $ hg mv inside/f1 inside/f2
   $ hg update -q 'desc("modify outside")'
-  $ hg update -q 'desc("initial")'
+  $ hg update -q 'desc("add inside and outside")'
   $ hg update -q 'desc("modify inside")'



To: martinvonz, durin42, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6253: tests: delete unused function in test-rename-merge2.t

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6253

AFFECTED FILES
  tests/test-rename-merge2.t

CHANGE DETAILS

diff --git a/tests/test-rename-merge2.t b/tests/test-rename-merge2.t
--- a/tests/test-rename-merge2.t
+++ b/tests/test-rename-merge2.t
@@ -68,7 +68,6 @@
   > hg add $2 2> /dev/null
   > fi
   > }
-  $ uc() { up $1; hg cp $1 $2; } # update + copy
   $ um() { up $1; hg mv $1 $2; }
   $ nc() { hg cp $1 $2; } # just copy
   $ nm() { hg mv $1 $2; } # just move



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6250: copies: delete debug message about changes since common ancestor

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Same reasoning as previous patch.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6250

AFFECTED FILES
  mercurial/copies.py
  tests/test-graft.t
  tests/test-rebase-conflicts.t
  tests/test-up-local-change.t

CHANGE DETAILS

diff --git a/tests/test-up-local-change.t b/tests/test-up-local-change.t
--- a/tests/test-up-local-change.t
+++ b/tests/test-up-local-change.t
@@ -67,8 +67,6 @@
   
   $ hg --debug up 0
   starting 4 threads for background file closing (?)
-unmatched files in local (from topological common ancestor):
- b
   resolving manifests
branchmerge: False, force: False, partial: False
ancestor: 1e71731e6fbb, local: 1e71731e6fbb+, remote: c19d34741b0a
diff --git a/tests/test-rebase-conflicts.t b/tests/test-rebase-conflicts.t
--- a/tests/test-rebase-conflicts.t
+++ b/tests/test-rebase-conflicts.t
@@ -248,8 +248,6 @@
   getting f1.txt
merge against 9:e31216eec445
  detach base 8:8e4e2c1a07ae
-unmatched files in other (from topological common ancestor):
- f2.txt
   resolving manifests
branchmerge: True, force: True, partial: False
ancestor: 8e4e2c1a07ae, local: 4bc80088dc6b+, remote: e31216eec445
@@ -267,8 +265,6 @@
already in destination
merge against 10:2f2496ddf49d
  detach base 9:e31216eec445
-unmatched files in other (from topological common ancestor):
- f2.txt
   resolving manifests
branchmerge: True, force: True, partial: False
ancestor: e31216eec445, local: 19c888675e13+, remote: 2f2496ddf49d
diff --git a/tests/test-graft.t b/tests/test-graft.t
--- a/tests/test-graft.t
+++ b/tests/test-graft.t
@@ -220,8 +220,6 @@
   committing changelog
   updating the branch cache
   grafting 5:97f8bfe72746 "5"
-unmatched files in other (from topological common ancestor):
- c
   resolving manifests
branchmerge: True, force: True, partial: False
ancestor: 4c60f11aa304, local: 6b9e5368ca4e+, remote: 97f8bfe72746
@@ -235,8 +233,6 @@
   $ HGEDITOR=cat hg graft 4 3 --log --debug
   scanning for duplicate grafts
   grafting 4:9c233e8e184d "4"
-unmatched files in other (from topological common ancestor):
- c
   resolving manifests
branchmerge: True, force: True, partial: False
ancestor: 4c60f11aa304, local: 1905859650ec+, remote: 9c233e8e184d
@@ -741,8 +737,6 @@
   $ hg graft -q 13 --debug
   scanning for duplicate grafts
   grafting 13:7a4785234d87 "2"
-unmatched files in other (from topological common ancestor):
- g
 unmatched files new in both:
  b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -353,27 +353,23 @@
 return _chain(x, y, _backwardrenames(x, a, match=match),
   _forwardcopies(a, y, match=match))
 
-def _computenonoverlap(repo, c1, c2, addedinm1, addedinm2, baselabel=''):
+def _computenonoverlap(repo, c1, c2, addedinm1, addedinm2, debug=True):
 """Computes, based on addedinm1 and addedinm2, the files exclusive to c1
 and c2. This is its own function so extensions can easily wrap this call
 to see what files mergecopies is about to process.
 
 Even though c1 and c2 are not used in this function, they are useful in
 other extensions for being able to read the file nodes of the changed 
files.
-
-"baselabel" can be passed to help distinguish the multiple computations
-done in the graft case.
 """
 u1 = sorted(addedinm1 - addedinm2)
 u2 = sorted(addedinm2 - addedinm1)
 
-header = "  unmatched files in %s"
-if baselabel:
-header += ' (from %s)' % baselabel
-if u1:
-repo.ui.debug("%s:\n   %s\n" % (header % 'local', "\n   ".join(u1)))
-if u2:
-repo.ui.debug("%s:\n   %s\n" % (header % 'other', "\n   ".join(u2)))
+if debug:
+header = "  unmatched files in %s"
+if u1:
+repo.ui.debug("%s:\n   %s\n" % (header % 'local', "\n   
".join(u1)))
+if u2:
+repo.ui.debug("%s:\n   %s\n" % (header % 'other', "\n   
".join(u2)))
 
 return u1, u2
 
@@ -588,15 +584,14 @@
 u1u, u2u = u1r, u2r
 else:
 # unmatched file from base (DAG rotation in the graft case)
-u1r, u2r = _computenonoverlap(repo, c1, c2, addedinm1, addedinm2,
-  baselabel='base')
+u1r, u2r = _computenonoverlap(repo, c1, c2, addedinm1, addedinm2)
 # unmatched file from topological common ancestors (no DAG rotation)
 # need to recompute this for directory move handling when grafting
 mta = tca.manifest()
 u1u, u2u = _computenonoverlap(repo, c1, c2,
   m1.filesnotin(mta, repo.narrowmatch()),
   m2.filesnotin(mta, repo.na

D6182: bundle2: handle compression in _forwardchunks

[durin42 (Augie Fackler)]

durin42 added a comment.


  Ugh I'm sorry - I evidently reviewed this and never pushed submit. :(

INLINE COMMENTS

> bundle2.py:846
> +if k.lower() != 'compression':
> +oparams.append(p)
> +outparams = ' '.join(outparams)

so...what's going on with oparams here? should it be outparams?

(is there more to this series that would get this code covered by tests?)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6182

To: joerg.sonnenberger, #hg-reviewers
Cc: durin42, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6251: copies: delete debug message about "unmatched files new in both"

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Same reasoning as previous patch.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6251

AFFECTED FILES
  mercurial/copies.py
  tests/test-graft.t
  tests/test-rename-merge2.t

CHANGE DETAILS

diff --git a/tests/test-rename-merge2.t b/tests/test-rename-merge2.t
--- a/tests/test-rename-merge2.t
+++ b/tests/test-rename-merge2.t
@@ -355,8 +355,6 @@
   --
   test L:um a b R:um a b W:   - 9  do merge with ancestor in a
   --
-unmatched files new in both:
- b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
  src: 'a' -> dst: 'b' *
 checking for directory renames
@@ -436,8 +434,6 @@
   --
   test L:nc a b R:up b   W:   - 12 merge b no ancestor
   --
-unmatched files new in both:
- b
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 86a2aa42fc76+, remote: af30c7647fc7
@@ -474,8 +470,6 @@
   --
   test L:up b   R:nm a b W:   - 13 merge b no ancestor
   --
-unmatched files new in both:
- b
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 59318016310c+, remote: bdb19105162a
@@ -513,8 +507,6 @@
   --
   test L:nc a b R:up a b W:   - 14 merge b no ancestor
   --
-unmatched files new in both:
- b
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 86a2aa42fc76+, remote: 8dbce441892a
@@ -552,8 +544,6 @@
   --
   test L:up b   R:nm a b W:   - 15 merge b no ancestor, remove a
   --
-unmatched files new in both:
- b
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 59318016310c+, remote: bdb19105162a
@@ -591,8 +581,6 @@
   --
   test L:nc a b R:up a b W:   - 16 get a, merge b no ancestor
   --
-unmatched files new in both:
- b
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 86a2aa42fc76+, remote: 8dbce441892a
@@ -630,8 +618,6 @@
   --
   test L:up a b R:nc a b W:   - 17 keep a, merge b no ancestor
   --
-unmatched files new in both:
- b
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 0b76e65c8289+, remote: 4ce40f5aca24
@@ -668,8 +654,6 @@
   --
   test L:nm a b R:up a b W:   - 18 merge b no ancestor
   --
-unmatched files new in both:
- b
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 02963e448370+, remote: 8dbce441892a
@@ -712,8 +696,6 @@
   --
   test L:up a b R:nm a b W:   - 19 merge b no ancestor, prompt remove a
   --
-unmatched files new in both:
- b
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 0b76e65c8289+, remote: bdb19105162a
@@ -930,9 +912,6 @@
  3/g
  4/g
  7/f
-unmatched files new in both:
- 0/f
- 1/g
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
  src: '1/f' -> dst: '1/g' *
  src: '3/f' -> dst: '3/g' *
diff --git a/tests/test-graft.t b/tests/test-graft.t
--- a/tests/test-graft.t
+++ b/tests/test-graft.t
@@ -737,8 +737,6 @@
   $ hg graft -q 13 --debug
   scanning for duplicate grafts
   grafting 13:7a4785234d87 "2"
-unmatched files new in both:
- b
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
  src: 'a' -> dst: 'b' *
 checking for directory renames
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -625,9 +625,6 @@
 else:
 divergeset.update(fl) # reverse map for below
 
-if bothnew:
-repo.ui.debug("  unmatched files new in both:\n   %s\n"
-  % "\n   ".join(bothnew))
 bothdiverge = {}
 bothincompletediverge = {}
 remainder = {}



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6246: copies: fix up "fullcopy" with missing entries from "copy"

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is just a workaround similar to the previous one. It will make it
  easier to follow later patches.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6246

AFFECTED FILES
  mercurial/copies.py
  tests/test-graft.t
  tests/test-rename-merge2.t

CHANGE DETAILS

diff --git a/tests/test-rename-merge2.t b/tests/test-rename-merge2.t
--- a/tests/test-rename-merge2.t
+++ b/tests/test-rename-merge2.t
@@ -366,6 +366,9 @@
 searching for copies back to rev 1
 unmatched files new in both:
  b
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b' *
+checking for directory renames
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 924404dff337, local: 62e7bf090eba+, remote: 49b6d8032493
@@ -953,6 +956,7 @@
  0/f
  1/g
 all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: '1/f' -> dst: '1/g' *
  src: '3/f' -> dst: '3/g' *
  src: '4/f' -> dst: '4/g' *
  src: '5/f' -> dst: '5/g' *
diff --git a/tests/test-graft.t b/tests/test-graft.t
--- a/tests/test-graft.t
+++ b/tests/test-graft.t
@@ -749,6 +749,9 @@
  g
 unmatched files new in both:
  b
+all copies found (* = to merge, ! = divergent, % = renamed and deleted):
+ src: 'a' -> dst: 'b' *
+checking for directory renames
   resolving manifests
branchmerge: True, force: True, partial: False
ancestor: b592ea63bb0c, local: 7e61b508e709+, remote: 7a4785234d87
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -687,6 +687,10 @@
 for dst, src in fullcopy.copy().items():
 if src not in mb:
 del fullcopy[dst]
+# Sometimes we forget to add entries from "copy" to "fullcopy", so fix
+# that up here
+for dst, src in copy.items():
+fullcopy[dst] = src
 if fullcopy and repo.ui.debugflag:
 repo.ui.debug("  all copies found (* = to merge, ! = divergent, "
   "% = renamed and deleted):\n")



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6248: copies: move early return for "no copies" case a little earlier

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We can return before the block that prints debug messages. That block
  will not be run anyway when there are no copies.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6248

AFFECTED FILES
  mercurial/copies.py

CHANGE DETAILS

diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -696,7 +696,11 @@
 for src, dsts in diverge.items():
 for dst in dsts:
 fullcopy[dst] = src
-if fullcopy and repo.ui.debugflag:
+
+if not fullcopy:
+return copy, {}, diverge, renamedelete, {}
+
+if repo.ui.debugflag:
 repo.ui.debug("  all copies found (* = to merge, ! = divergent, "
   "% = renamed and deleted):\n")
 for f in sorted(fullcopy):
@@ -711,9 +715,6 @@
   note))
 del divergeset
 
-if not fullcopy:
-return copy, {}, diverge, renamedelete, {}
-
 repo.ui.debug("  checking for directory renames\n")
 
 # generate a directory move map



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6245: merge: remove workaround for issue5020

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As I explained in the previous commit, I think the filtering added
  there is a better fix for the issue, so the workaround from
  https://phab.mercurial-scm.org/rHG41f6af50c0d8824600b1112da932dcdf399e8d5d 
(merge: fix crash on criss cross merge with dir move and
  delete (issue5020), 2017-01-31) should no longer be needed.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6245

AFFECTED FILES
  mercurial/merge.py

CHANGE DETAILS

diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -1380,17 +1380,14 @@
 # Pick the best bid for each file
 repo.ui.note(_('\nauction for merging merge bids\n'))
 actions = {}
-dms = [] # filenames that have dm actions
 for f, bids in sorted(fbids.items()):
 # bids is a mapping from action method to list af actions
 # Consensus?
 if len(bids) == 1: # all bids are the same kind of method
 m, l = list(bids.items())[0]
 if all(a == l[0] for a in l[1:]): # len(bids) is > 1
 repo.ui.note(_(" %s: consensus for %s\n") % (f, m))
 actions[f] = l[0]
-if m == ACTION_DIR_RENAME_MOVE_LOCAL:
-dms.append(f)
 continue
 # If keep is an option, just do it.
 if ACTION_KEEP in bids:
@@ -1415,18 +1412,7 @@
 repo.ui.warn(_(' %s: ambiguous merge - picked %s action\n') %
  (f, m))
 actions[f] = l[0]
-if m == ACTION_DIR_RENAME_MOVE_LOCAL:
-dms.append(f)
 continue
-# Work around 'dm' that can cause multiple actions for the same file
-for f in dms:
-dm, (f0, flags), msg = actions[f]
-assert dm == ACTION_DIR_RENAME_MOVE_LOCAL, dm
-if f0 in actions and actions[f0][0] == ACTION_REMOVE:
-# We have one bid for removing a file and another for moving 
it.
-# These two could be merged as first move and then delete ...
-# but instead drop moving and just delete.
-del actions[f]
 repo.ui.note(_('end of auction\n\n'))
 
 if wctx.rev() is None:



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6247: copies: fix up "fullcopy" with missing entries from "diverge"

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Similar to the previous patch, but this doesn't even affect tests. It
  does affect tests if you change them to turn on debug logging. I'm
  fixing it here so reviewers of the later rewrite patch can hard-code
  debug logging to be on and more easily compare test results.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6247

AFFECTED FILES
  mercurial/copies.py

CHANGE DETAILS

diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -691,6 +691,11 @@
 # that up here
 for dst, src in copy.items():
 fullcopy[dst] = src
+# Sometimes we forget to add entries from "diverge" to "fullcopy", so fix
+# that up here
+for src, dsts in diverge.items():
+for dst in dsts:
+fullcopy[dst] = src
 if fullcopy and repo.ui.debugflag:
 repo.ui.debug("  all copies found (* = to merge, ! = divergent, "
   "% = renamed and deleted):\n")



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6244: copies: don't include copies that are not in source in directory move

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  I've been working on a rewrite of mergecopies(). I compared the output
  of the rewritten version with the current version. I noticed that
  between FIREFOX_NIGHTLY_59_END and FIREFOX_BETA_60_BASE in the
  mozilla-unified repo, there were many copies that the current version
  detected that the rewritten version did not. One example was
  js/src/gc/Iteration.h -> js/src/gc/PublicIterators.h. Then I realized
  that js/src/gc/Iteration.h doesn't even exist in
  FIREFOX_NIGHTLY_59_END.
  
  This patch adds a filtering step for the "fullcopy" dict. It turns out
  that that change also affects the test for issue5020 in
  test-merge-criss-cross.t. The 'dm' action no longer happens there. At
  first I thought that the test case change meant that this patch was
  broken, but I think it's actually correct tha the 'dm' action should
  not happen there. The result of the bid merge is still the same.
  
  I suspect this filtering is a better solution for the issue than
  https://phab.mercurial-scm.org/rHG41f6af50c0d8824600b1112da932dcdf399e8d5d 
(merge: fix crash on criss cross merge with dir move and
  delete (issue5020), 2017-01-31). I also suspect that it was broken
  just a few months earlier by 
https://phab.mercurial-scm.org/rHGa005c33d0bd785745a905e87f34e90885d0f8cc4 
(mergecopies: add logic to
  process incomplete data, 2016-10-04). Note that bid merge had been
  enabled for a few years at that point, since 
https://phab.mercurial-scm.org/rHG19903277f0355e3e4b1a991e7fc4b51f71cfd553 
(merge: use
  bid merge by default (BC), 2014-10-01).
  
  This patch is still just a workaround. It will be cleaned up soon
  (with the rewrite of mergecopies()). But doing this in a separate
  patch makes later patches easier to understand and gives a place to
  explain why this is changing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6244

AFFECTED FILES
  mercurial/copies.py
  tests/test-merge-criss-cross.t

CHANGE DETAILS

diff --git a/tests/test-merge-criss-cross.t b/tests/test-merge-criss-cross.t
--- a/tests/test-merge-criss-cross.t
+++ b/tests/test-merge-criss-cross.t
@@ -423,17 +423,10 @@
  d1/b
 unmatched files in other:
  d2/b
-all copies found (* = to merge, ! = divergent, % = renamed and deleted):
- src: 'd1/b' -> dst: 'd2/b' 
-checking for directory renames
- discovered dir src: 'd1/' -> dst: 'd2/'
- pending file src: 'd1/a' -> dst: 'd2/a'
- pending file src: 'd1/b' -> dst: 'd2/b'
   resolving manifests
branchmerge: True, force: False, partial: False
ancestor: 11b5b303e36c, local: c0ef19750a22+, remote: 6ca01f7342b9
-   d2/a: remote directory rename - move from d1/a -> dm
-   d2/b: remote directory rename, both created -> m
+   d2/b: remote created -> g
   
   calculating bids for ancestor 154e6000f54e
 searching for copies back to rev 3
@@ -453,8 +446,7 @@
   auction for merging merge bids
d1/a: consensus for r
d1/b: consensus for r
-   d2/a: consensus for dm
-   d2/b: picking 'get' action
+   d2/b: consensus for g
   end of auction
   
d1/a: other deleted -> r
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -682,6 +682,11 @@
 if len(fl) == 2 and fl[0] == fl[1]:
 copy[fl[0]] = of # not actually divergent, just matching renames
 
+# Sometimes we get invalid copies here (the "and not remotebase" in
+# _checkcopies() seems suspicious). Filter them out.
+for dst, src in fullcopy.copy().items():
+if src not in mb:
+del fullcopy[dst]
 if fullcopy and repo.ui.debugflag:
 repo.ui.debug("  all copies found (* = to merge, ! = divergent, "
   "% = renamed and deleted):\n")



To: martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6243: tests: split out separate test for issue5020

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The test was added to the existing setup in 
https://phab.mercurial-scm.org/rHG41f6af50c0d8824600b1112da932dcdf399e8d5d 
(merge: fix
  crash on criss cross merge with dir move and delete (issue5020),
  2017-01-31). I'm about to make a change that affects that test and
  it's much easier to follow if the test case for issue5020 is a
  separate test case. The separate test case is based on what mpm
  provided in comment 12 on the issue.
  
  `hg diff -r 41f6af50c0d8^ tests/test-merge-criss-cross.t` after this
  patch is pretty small (besides the added test). It's probably easier
  for reviewers to look at that than to try to understand the diff
  itself (I don't understand it).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6243

AFFECTED FILES
  tests/test-merge-criss-cross.t

CHANGE DETAILS

diff --git a/tests/test-merge-criss-cross.t b/tests/test-merge-criss-cross.t
--- a/tests/test-merge-criss-cross.t
+++ b/tests/test-merge-criss-cross.t
@@ -11,15 +11,9 @@
 
   $ hg up -qr0
   $ echo '2 first change' > f2
-  $ mkdir d1
-  $ echo '0 base' > d1/f3
-  $ echo '0 base' > d1/f4
-  $ hg add -q d1
   $ hg ci -qm '2 first change f2'
 
   $ hg merge -qr 1
-  $ hg rm d1/f3
-  $ hg mv -q d1 d2
   $ hg ci -m '3 merge'
 
   $ hg up -qr2
@@ -30,38 +24,38 @@
   $ hg ci -m '5 second change f1'
 
   $ hg up -r3
-  2 files updated, 0 files merged, 2 files removed, 0 files unresolved
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ echo '6 second change' > f2
   $ hg ci -m '6 second change f2'
 
   $ hg log -G
-  @  changeset:   6:6373bbfdae1d
+  @  changeset:   6:3b08d01b0ab5
   |  tag: tip
-  |  parent:  3:c202c8af058d
+  |  parent:  3:cf89f02107e5
   |  user:test
   |  date:Thu Jan 01 00:00:00 1970 +
   |  summary: 6 second change f2
   |
-  | o  changeset:   5:e673248094b1
+  | o  changeset:   5:adfe50279922
   | |  user:test
   | |  date:Thu Jan 01 00:00:00 1970 +
   | |  summary: 5 second change f1
   | |
-  | ochangeset:   4:177f58377c06
-  | |\   parent:  2:d1d156401c1b
+  | ochangeset:   4:7d3e55501ae6
+  | |\   parent:  2:40663881a6dd
   | | |  parent:  1:0f6b37dbe527
   | | |  user:test
   | | |  date:Thu Jan 01 00:00:00 1970 +
   | | |  summary: 4 merge
   | | |
-  o---+  changeset:   3:c202c8af058d
-  | | |  parent:  2:d1d156401c1b
+  o---+  changeset:   3:cf89f02107e5
+  | | |  parent:  2:40663881a6dd
   |/ /   parent:  1:0f6b37dbe527
   | |user:test
   | |date:Thu Jan 01 00:00:00 1970 +
   | |summary: 3 merge
   | |
-  | o  changeset:   2:d1d156401c1b
+  | o  changeset:   2:40663881a6dd
   | |  parent:  0:40494bf2444c
   | |  user:test
   | |  date:Thu Jan 01 00:00:00 1970 +
@@ -79,51 +73,27 @@
   
 
   $ hg merge -v --debug --tool internal:dump 5 --config 
merge.preferancestor='!'
-  note: using 0f6b37dbe527 as ancestor of 6373bbfdae1d and e673248094b1
-alternatively, use --config merge.preferancestor=d1d156401c1b
+  note: using 0f6b37dbe527 as ancestor of 3b08d01b0ab5 and adfe50279922
+alternatively, use --config merge.preferancestor=40663881a6dd
 searching for copies back to rev 3
-unmatched files in local:
- d2/f4
-unmatched files in other:
- d1/f3
- d1/f4
-all copies found (* = to merge, ! = divergent, % = renamed and deleted):
- src: 'd1/f4' -> dst: 'd2/f4' 
-checking for directory renames
- discovered dir src: 'd1/' -> dst: 'd2/'
- pending file src: 'd1/f3' -> dst: 'd2/f3'
- pending file src: 'd1/f4' -> dst: 'd2/f4'
   resolving manifests
branchmerge: True, force: False, partial: False
-   ancestor: 0f6b37dbe527, local: 6373bbfdae1d+, remote: e673248094b1
-   preserving d2/f4 for resolve of d2/f4
+   ancestor: 0f6b37dbe527, local: 3b08d01b0ab5+, remote: adfe50279922
preserving f2 for resolve of f2
f1: remote is newer -> g
   getting f1
-   d2/f3: local directory rename - get from d1/f3 -> dg
-  getting d1/f3 to d2/f3
-   d2/f4: local directory rename, both created -> m (premerge)
f2: versions differ -> m (premerge)
   picked tool ':dump' for f2 (binary False symlink False changedelete False)
   merging f2
-  my f2@6373bbfdae1d+ other f2@e673248094b1 ancestor f2@0f6b37dbe527
+  my f2@3b08d01b0ab5+ other f2@adfe50279922 ancestor f2@0f6b37dbe527
f2: versions differ -> m (merge)
   picked tool ':dump' for f2 (binary False symlink False changedelete False)
-  my f2@6373bbfdae1d+ other f2@e673248094b1 ancestor f2@0f6b37dbe527
-  3 files updated, 0 files merged, 0 files removed, 1 files unresolved
+  my f2@3b08d01b0ab5+ other f2@adfe50279922 ancestor f2@0f6b37dbe527
+  1 files updated, 0 files merged, 0 files removed, 1 files unresolved
   use 'hg resolve' 

D6242: tests: add test for issue5343 (grafting with copies)

[martinvonz (Martin von Zweigbergk)]

martinvonz created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  It seems that issue5353 resulted in a lot of tests in test-graft.t,
  but the bug actually reported in that issue didn't get a test
  case. This patch adds one for the "move" and one for the "copy"
  version of it. I also added a "copy+modify" case, to show what should
  be a merge conflict. I didn't add one for the "backwards" version of
  it since the comment says that that was already covered by previous
  work.
  
  The tests added by this patch show the broken behavior (the bug is
  still open). I suspect the results returned from mergecopies() are not
  expressive enough to fix this issue: it has a dict for copies to merge
  with, but that can only give one more filename, but here we need two
  (one for the path on the remote side and one for the path in the merge
  base). I want to have it tested anyway since I'm about to refactor
  mergecopies().

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6242

AFFECTED FILES
  tests/test-graft.t

CHANGE DETAILS

diff --git a/tests/test-graft.t b/tests/test-graft.t
--- a/tests/test-graft.t
+++ b/tests/test-graft.t
@@ -969,7 +969,6 @@
 A.4 has a degenerate case a<-b<-a->a, where checkcopies isn't needed at all.
 A.5 has a special case a<-b<-b->a, which is treated like a<-b->a in a merge.
 A.5 has issue5343 as a special case.
-TODO: add test coverage for A.5
 A.6 has a special case a<-a<-b->a. Here, checkcopies will find a spurious
 incomplete divergence, which is in fact complete. This is handled later in
 mergecopies.
@@ -1072,13 +1071,32 @@
   $ hg mv f4a f4e
   $ hg mv f5a f5b
   $ hg ci -qAm "E0"
+  $ hg up -q "min(desc("A0"))"
+  $ hg cp f1a f1f
+  $ hg ci -qAm "F0"
+  $ hg up -q "min(desc("A0"))"
+  $ hg cp f1a f1g
+  $ echo c1g > f1g
+  $ hg ci -qAm "G0"
   $ hg log -G
-  @  changeset:   6:6bd1736cab86
+  @  changeset:   8:ba67f08fb15a
   |  tag: tip
   |  parent:  0:11f7a1b56675
   |  user:test
   |  date:Thu Jan 01 00:00:00 1970 +
-  |  summary: E0
+  |  summary: G0
+  |
+  | o  changeset:   7:d376ab0d7fda
+  |/   parent:  0:11f7a1b56675
+  |user:test
+  |date:Thu Jan 01 00:00:00 1970 +
+  |summary: F0
+  |
+  | o  changeset:   6:6bd1736cab86
+  |/   parent:  0:11f7a1b56675
+  |user:test
+  |date:Thu Jan 01 00:00:00 1970 +
+  |summary: E0
   |
   | o  changeset:   5:560daee679da
   | |  user:test
@@ -1115,6 +1133,7 @@
 Test the cases A.4 (f1x), the "ping-pong" special case of A.7 (f5x),
 and A.3 with a local content change to be preserved (f2x).
 
+  $ hg up -q "desc("E0")"
   $ HGEDITOR="echo C2 >" hg graft -r 'desc("C0")' --edit
   grafting 2:f58c7e2b28fa "C0"
   merging f1e and f1b to f1e
@@ -1131,154 +1150,301 @@
   merging f4e and f4a to f4e
   warning: can't find ancestor for 'f3d' copied from 'f3b'!
 
+  $ hg cat f2c
+  c2e
+
+Test the case A.5 (move case, f1x).
+
+  $ hg up -q "desc("C0")"
+BROKEN: Shouldn't get the warning about missing ancestor
+  $ HGEDITOR="echo E1 >" hg graft -r 'desc("E0")' --edit
+  grafting 6:6bd1736cab86 "E0"
+  note: possible conflict - f1a was renamed multiple times to:
+   f1b
+   f1e
+  note: possible conflict - f3a was renamed multiple times to:
+   f3b
+   f3e
+  merging f2c and f2a to f2c
+  merging f5a and f5b to f5b
+  warning: can't find ancestor for 'f1e' copied from 'f1a'!
+  warning: can't find ancestor for 'f3e' copied from 'f3a'!
+  $ cat f1e
+  c1a
+
+Test the case A.5 (copy case, f1x).
+
+  $ hg up -q "desc("C0")"
+BROKEN: Shouldn't get the warning about missing ancestor
+  $ HGEDITOR="echo F1 >" hg graft -r 'desc("F0")' --edit
+  grafting 7:d376ab0d7fda "F0"
+  warning: can't find ancestor for 'f1f' copied from 'f1a'!
+BROKEN: f1f should be marked a copy from f1b
+  $ hg st --copies --change .
+  A f1f
+BROKEN: f1f should have the new content from f1b (i.e. "c1c")
+  $ cat f1f
+  c1a
+
+Test the case A.5 (copy+modify case, f1x).
+
+  $ hg up -q "desc("C0")"
+BROKEN: We should get a merge conflict from the 3-way merge between f1b in C0
+(content "c1c") and f1g in G0 (content "c1g") with f1a in A0 as base (content
+"c1a")
+  $ HGEDITOR="echo G1 >" hg graft -r 'desc("G0")' --edit
+  grafting 8:ba67f08fb15a "G0"
+  warning: can't find ancestor for 'f1g' copied from 'f1a'!
+
 Check the results of the grafts tested
 
   $ hg log -CGv --patch --git
-  @  changeset:   8:93ee502e8b0a
+  @  changeset:   13:ef3adf6c20a4
   |  tag: tip
+  |  parent:  2:f58c7e2b28fa
   |  user:test
   |  date:Thu Jan 01 00:00:00 1970 +
-  |  files:   f3d f4e
+  |  files:   f1g
   |  description:
-  |  D2
+  |  G1
   |
   |
-  |  diff --git a/f3d b/f3d
+  |  diff --git a/f1g b/f1g
   |  new file mode 100644
   |  --- /dev/null
-  |  +++ b/f3d
+  |  +++ b/f1g
   |  @@ -0,0 +1,1 @@
-  |  +c3a
-  |  diff 

D6241: narrow: drop unrequired if conditionals

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG2003f0bffcb1: narrow: drop unrequired if conditionals 
(authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6241?vs=14768&id=14769

REVISION DETAIL
  https://phab.mercurial-scm.org/D6241

AFFECTED FILES
  hgext/narrow/narrowbundle2.py
  mercurial/exchange.py

CHANGE DETAILS

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -2214,14 +2214,8 @@
 if (kwargs.get(r'narrow', False) and kwargs.get(r'narrow_acl', False)
 and (include or exclude)):
 narrowspecpart = bundler.newpart('narrow:spec')
-data = ''
-if include:
-data += '\n'.join(include)
-data += '\0'
-if exclude:
-data += '\n'.join(exclude)
-
-narrowspecpart.data = data
+narrowspecpart.data = '%s\0%s' % ('\n'.join(include),
+   '\n'.join(exclude))
 
 @getbundle2partsgenerator('bookmarks')
 def _getbundlebookmarkpart(bundler, repo, source, bundlecaps=None,
diff --git a/hgext/narrow/narrowbundle2.py b/hgext/narrow/narrowbundle2.py
--- a/hgext/narrow/narrowbundle2.py
+++ b/hgext/narrow/narrowbundle2.py
@@ -149,10 +149,8 @@
 # bundle2 parameters instead.
 if data:
 inc, exc = data.split('\0')
-if inc:
-includepats |= set(inc.splitlines())
-if exc:
-excludepats |= set(exc.splitlines())
+includepats |= set(inc.splitlines())
+excludepats |= set(exc.splitlines())
 narrowspec.validatepatterns(includepats)
 narrowspec.validatepatterns(excludepats)
 



To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6218: narrow: send specs as bundle2 data instead of param (issue5952) (issue6019)

[pulkit (Pulkit Goyal)]

pulkit added inline comments.

INLINE COMMENTS

> martinvonz wrote in narrowbundle2.py:152
> I'd drop these checks

Sent https://phab.mercurial-scm.org/D6241.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6218

To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: idlsoft, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 3 "V4] upgrade: support upgrade to/from zstd storage (issue6088)

[Pulkit Goyal]

I see that Augie has already queued this series. :)

On Tue, Apr 16, 2019 at 7:53 PM Gregory Szorc 
wrote:

> I would also like to get this in the RC. Makes it more accessible sooner,
> etc.
>
> > On Apr 16, 2019, at 18:03, Pierre-Yves David <
> pierre-yves.da...@ens-lyon.org> wrote:
> >
> >
> >
> >> On 4/16/19 4:31 PM, Augie Fackler wrote:
> >>> On Tue, Apr 16, 2019 at 03:26:07PM +0200, Pierre-Yves David wrote:
> >>> # HG changeset patch
> >>> # User Pierre-Yves David 
> >>> # Date 1549032662 -3600
> >>> #  Fri Feb 01 15:51:02 2019 +0100
> >>> # Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
> >>> # Parent  cbc090e88191176d0d7708e069a98596df78c2c5
> >>> # EXP-Topic zstd-revlog
> >>> # Available At https://bitbucket.org/octobus/mercurial-devel/
> >>> #  hg pull https://bitbucket.org/octobus/mercurial-devel/
> -r 821f57c6dda8
> >>> upgrade: support upgrade to/from zstd storage (issue6088)
> >> These look fine, but I'm unclear if we should ship this in the RC or
> >> hold for 5.1. Do you have a preference?
> >
> > (summary of IRC discussion: Yes, my goal is to get this in 5.0 (disabled
> by default) and I prefer if it gets in).
> >
> >
> > --
> > Pierre-Yves David
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6241: narrow: drop unrequired if conditionals

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a reviewer: martinvonz.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6241

AFFECTED FILES
  hgext/narrow/narrowbundle2.py
  mercurial/exchange.py

CHANGE DETAILS

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -2214,14 +2214,8 @@
 if (kwargs.get(r'narrow', False) and kwargs.get(r'narrow_acl', False)
 and (include or exclude)):
 narrowspecpart = bundler.newpart('narrow:spec')
-data = ''
-if include:
-data += '\n'.join(include)
-data += '\0'
-if exclude:
-data += '\n'.join(exclude)
-
-narrowspecpart.data = data
+narrowspecpart.data = '%s\0%s' % ('\n'.join(include),
+   '\n'.join(exclude))
 
 @getbundle2partsgenerator('bookmarks')
 def _getbundlebookmarkpart(bundler, repo, source, bundlecaps=None,
diff --git a/hgext/narrow/narrowbundle2.py b/hgext/narrow/narrowbundle2.py
--- a/hgext/narrow/narrowbundle2.py
+++ b/hgext/narrow/narrowbundle2.py
@@ -149,10 +149,8 @@
 # bundle2 parameters instead.
 if data:
 inc, exc = data.split('\0')
-if inc:
-includepats |= set(inc.splitlines())
-if exc:
-excludepats |= set(exc.splitlines())
+includepats |= set(inc.splitlines())
+excludepats |= set(exc.splitlines())
 narrowspec.validatepatterns(includepats)
 narrowspec.validatepatterns(excludepats)
 



To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 3 "V4] upgrade: support upgrade to/from zstd storage (issue6088)

[Gregory Szorc]

I would also like to get this in the RC. Makes it more accessible sooner, etc.

> On Apr 16, 2019, at 18:03, Pierre-Yves David  
> wrote:
> 
> 
> 
>> On 4/16/19 4:31 PM, Augie Fackler wrote:
>>> On Tue, Apr 16, 2019 at 03:26:07PM +0200, Pierre-Yves David wrote:
>>> # HG changeset patch
>>> # User Pierre-Yves David 
>>> # Date 1549032662 -3600
>>> #  Fri Feb 01 15:51:02 2019 +0100
>>> # Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
>>> # Parent  cbc090e88191176d0d7708e069a98596df78c2c5
>>> # EXP-Topic zstd-revlog
>>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>>> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
>>> 821f57c6dda8
>>> upgrade: support upgrade to/from zstd storage (issue6088)
>> These look fine, but I'm unclear if we should ship this in the RC or
>> hold for 5.1. Do you have a preference?
> 
> (summary of IRC discussion: Yes, my goal is to get this in 5.0 (disabled by 
> default) and I prefer if it gets in).
> 
> 
> -- 
> Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 3 "V4] upgrade: support upgrade to/from zstd storage (issue6088)

[Pierre-Yves David]

On 4/16/19 4:31 PM, Augie Fackler wrote:

On Tue, Apr 16, 2019 at 03:26:07PM +0200, Pierre-Yves David wrote:

# HG changeset patch
# User Pierre-Yves David 
# Date 1549032662 -3600
#  Fri Feb 01 15:51:02 2019 +0100
# Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
# Parent  cbc090e88191176d0d7708e069a98596df78c2c5
# EXP-Topic zstd-revlog
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
821f57c6dda8
upgrade: support upgrade to/from zstd storage (issue6088)


These look fine, but I'm unclear if we should ship this in the RC or
hold for 5.1. Do you have a preference?


(summary of IRC discussion: Yes, my goal is to get this in 5.0 (disabled 
by default) and I prefer if it gets in).



--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6183: copies: add config option for writing copy metadata to file and/or changset

[martinvonz (Martin von Zweigbergk)]

martinvonz updated this revision to Diff 14766.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6183?vs=14738&id=14766

REVISION DETAIL
  https://phab.mercurial-scm.org/D6183

AFFECTED FILES
  mercurial/changelog.py
  mercurial/configitems.py
  mercurial/localrepo.py
  tests/test-annotate.t
  tests/test-copies-in-changeset.t
  tests/test-fastannotate-hg.t

CHANGE DETAILS

diff --git a/tests/test-fastannotate-hg.t b/tests/test-fastannotate-hg.t
--- a/tests/test-fastannotate-hg.t
+++ b/tests/test-fastannotate-hg.t
@@ -443,7 +443,7 @@
   > def reposetup(ui, repo):
   > class legacyrepo(repo.__class__):
   > def _filecommit(self, fctx, manifest1, manifest2,
-  > linkrev, tr, changelist):
+  > linkrev, tr, changelist, includecopymeta):
   > fname = fctx.path()
   > text = fctx.data()
   > flog = self.file(fname)
diff --git a/tests/test-copies-in-changeset.t b/tests/test-copies-in-changeset.t
new file mode 100644
--- /dev/null
+++ b/tests/test-copies-in-changeset.t
@@ -0,0 +1,105 @@
+
+  $ cat >> $HGRCPATH << EOF
+  > [experimental]
+  > copies.write-to=changeset-only
+  > [alias]
+  > changesetcopies = log -r . -T 'files: {files}
+  >   {extras % "{ifcontains("copies", key, "{key}: {value}\n")}"}'
+  > EOF
+
+Check that copies are recorded correctly
+
+  $ hg init repo
+  $ cd repo
+  $ echo a > a
+  $ hg add a
+  $ hg ci -m initial
+  $ hg cp a b
+  $ hg cp a c
+  $ hg cp a d
+  $ hg ci -m 'copy a to b, c, and d'
+  $ hg changesetcopies
+  files: b c d
+  p1copies: b\x00a (esc)
+  c\x00a (esc)
+  d\x00a (esc)
+
+Check that renames are recorded correctly
+
+  $ hg mv b b2
+  $ hg ci -m 'rename b to b2'
+  $ hg changesetcopies
+  files: b b2
+  p1copies: b2\x00b (esc)
+
+Rename onto existing file. This should get recorded in the changeset files 
list and in the extras,
+even though there is no filelog entry.
+
+  $ hg cp b2 c --force
+  $ hg st --copies
+  M c
+b2
+  $ hg debugindex c
+ rev linkrev nodeid   p1   p2
+   0   1 b789fdd96dc2  
+  $ hg ci -m 'move b onto d'
+  $ hg changesetcopies
+  files: c
+  p1copies: c\x00b2 (esc)
+  $ hg debugindex c
+ rev linkrev nodeid   p1   p2
+   0   1 b789fdd96dc2  
+
+Create a merge commit with copying done during merge.
+
+  $ hg co 0
+  0 files updated, 0 files merged, 3 files removed, 0 files unresolved
+  $ hg cp a e
+  $ hg cp a f
+  $ hg ci -m 'copy a to e and f'
+  created new head
+  $ hg merge 3
+  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+File 'a' exists on both sides, so 'g' could be recorded as being from p1 or 
p2, but we currently
+always record it as being from p1
+  $ hg cp a g
+File 'd' exists only in p2, so 'h' should be from p2
+  $ hg cp d h
+File 'f' exists only in p1, so 'i' should be from p1
+  $ hg cp f i
+  $ hg ci -m 'merge'
+  $ hg changesetcopies
+  files: g h i
+  p1copies: g\x00a (esc)
+  i\x00f (esc)
+  p2copies: h\x00d (esc)
+
+Test writing to both changeset and filelog
+
+  $ hg cp a j
+  $ hg ci -m 'copy a to j' --config experimental.copies.write-to=compatibility
+  $ hg changesetcopies
+  files: j
+  p1copies: j\x00a (esc)
+  $ hg debugdata j 0
+  \x01 (esc)
+  copy: a
+  copyrev: b789fdd96dc2f3bd229c1dd8eedf0fc60e2b68e3
+  \x01 (esc)
+  a
+
+Test writing only to filelog
+
+  $ hg cp a k
+  $ hg ci -m 'copy a to k' --config experimental.copies.write-to=filelog-only
+  $ hg changesetcopies
+  files: k
+  $ hg debugdata k 0
+  \x01 (esc)
+  copy: a
+  copyrev: b789fdd96dc2f3bd229c1dd8eedf0fc60e2b68e3
+  \x01 (esc)
+  a
+
+  $ cd ..
diff --git a/tests/test-annotate.t b/tests/test-annotate.t
--- a/tests/test-annotate.t
+++ b/tests/test-annotate.t
@@ -438,7 +438,7 @@
   > def reposetup(ui, repo):
   > class legacyrepo(repo.__class__):
   > def _filecommit(self, fctx, manifest1, manifest2,
-  > linkrev, tr, changelist):
+  > linkrev, tr, changelist, includecopymeta):
   > fname = fctx.path()
   > text = fctx.data()
   > flog = self.file(fname)
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -2324,7 +2324,8 @@
 """Returns the wlock if it's held, or None if it's not."""
 return self._currentlock(self._wlockref)
 
-def _filecommit(self, fctx, manifest1, manifest2, linkrev, tr, changelist):
+def _filecommit(self, fctx, manifest1, manifest2, linkrev, tr, changelist,
+includecopymeta):
 """
 commit an individual file as part of a larger transaction
 """
@@ -2383,8 +2384,9 @@
 
 if cnode:
 self.ui.debug(" %s: copy %s:%s\n" % (fname, cfname, 
hex(cnode)))
-meta["copy"] = c

D6186: changelog: parse copy metadata if available in extras

[martinvonz (Martin von Zweigbergk)]

martinvonz updated this revision to Diff 14767.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6186?vs=14739&id=14767

REVISION DETAIL
  https://phab.mercurial-scm.org/D6186

AFFECTED FILES
  mercurial/changelog.py
  mercurial/context.py
  mercurial/copies.py
  tests/test-copies-in-changeset.t
  tests/test-copies.t

CHANGE DETAILS

diff --git a/tests/test-copies.t b/tests/test-copies.t
--- a/tests/test-copies.t
+++ b/tests/test-copies.t
@@ -1,4 +1,4 @@
-#testcases filelog compatibility
+#testcases filelog compatibility changeset
 
   $ cat >> $HGRCPATH << EOF
   > [extensions]
@@ -14,6 +14,14 @@
   > EOF
 #endif
 
+#if changeset
+  $ cat >> $HGRCPATH << EOF
+  > [experimental]
+  > copies.read-from = changeset-only
+  > copies.write-to = changeset-only
+  > EOF
+#endif
+
   $ REPONUM=0
   $ newrepo() {
   > cd $TESTTMP
@@ -376,11 +384,13 @@
   o  0 add x on branch 1
  x
   $ hg debugp1copies -r 2
+  x -> z (changeset !)
   $ hg debugp2copies -r 2
-  x -> z
+  x -> z (no-changeset !)
   $ hg debugpathcopies 1 2
+  x -> z (changeset !)
   $ hg debugpathcopies 0 2
-  x -> z
+  x -> z (no-changeset !)
 
 Copy x->y on one side of merge and copy x->z on the other side. Pathcopies 
from one parent
 of the merge to the merge should include the copy from the other side.
@@ -539,6 +549,9 @@
 
 Grafting revision 4 on top of revision 2, showing that it respect the rename:
 
+TODO: Make this work with copy info in changesets (probably by writing a
+changeset-centric version of copies.mergecopies())
+#if no-changeset
   $ hg up 2 -q
   $ hg graft -r 4 --base 3 --hidden
   grafting 4:af28412ec03c "added d, modified b" (tip)
@@ -554,6 +567,8 @@
   b
  +baba
   
+#endif
+
 Test to make sure that fullcopytracing algorithm don't fail when both the 
merging csets are dirty
 (a dirty cset is one who is not the descendant of merge base)
 
-
diff --git a/tests/test-copies-in-changeset.t b/tests/test-copies-in-changeset.t
--- a/tests/test-copies-in-changeset.t
+++ b/tests/test-copies-in-changeset.t
@@ -2,9 +2,11 @@
   $ cat >> $HGRCPATH << EOF
   > [experimental]
   > copies.write-to=changeset-only
+  > copies.read-from=changeset-only
   > [alias]
   > changesetcopies = log -r . -T 'files: {files}
   >   {extras % "{ifcontains("copies", key, "{key}: {value}\n")}"}'
+  > showcopies = log -r . -T '{file_copies % "{source} -> {name}\n"}'
   > EOF
 
 Check that copies are recorded correctly
@@ -23,14 +25,25 @@
   p1copies: b\x00a (esc)
   c\x00a (esc)
   d\x00a (esc)
+  $ hg showcopies
+  a -> b
+  a -> c
+  a -> d
+  $ hg showcopies --config experimental.copies.read-from=compatibility
+  a -> b
+  a -> c
+  a -> d
+  $ hg showcopies --config experimental.copies.read-from=filelog-only
 
 Check that renames are recorded correctly
 
   $ hg mv b b2
   $ hg ci -m 'rename b to b2'
   $ hg changesetcopies
   files: b b2
   p1copies: b2\x00b (esc)
+  $ hg showcopies
+  b -> b2
 
 Rename onto existing file. This should get recorded in the changeset files 
list and in the extras,
 even though there is no filelog entry.
@@ -46,6 +59,8 @@
   $ hg changesetcopies
   files: c
   p1copies: c\x00b2 (esc)
+  $ hg showcopies
+  b2 -> c
   $ hg debugindex c
  rev linkrev nodeid   p1   p2
0   1 b789fdd96dc2  
@@ -74,6 +89,10 @@
   p1copies: g\x00a (esc)
   i\x00f (esc)
   p2copies: h\x00d (esc)
+  $ hg showcopies
+  a -> g
+  d -> h
+  f -> i
 
 Test writing to both changeset and filelog
 
@@ -88,6 +107,12 @@
   copyrev: b789fdd96dc2f3bd229c1dd8eedf0fc60e2b68e3
   \x01 (esc)
   a
+  $ hg showcopies
+  a -> j
+  $ hg showcopies --config experimental.copies.read-from=compatibility
+  a -> j
+  $ hg showcopies --config experimental.copies.read-from=filelog-only
+  a -> j
 
 Test writing only to filelog
 
@@ -101,5 +126,10 @@
   copyrev: b789fdd96dc2f3bd229c1dd8eedf0fc60e2b68e3
   \x01 (esc)
   a
+  $ hg showcopies
+  $ hg showcopies --config experimental.copies.read-from=compatibility
+  a -> k
+  $ hg showcopies --config experimental.copies.read-from=filelog-only
+  a -> k
 
   $ cd ..
diff --git a/mercurial/copies.py b/mercurial/copies.py
--- a/mercurial/copies.py
+++ b/mercurial/copies.py
@@ -162,8 +162,8 @@
 
 def usechangesetcentricalgo(repo):
 """Checks if we should use changeset-centric copy algorithms"""
-return (repo.ui.config('experimental', 'copies.read-from') ==
-'compatibility')
+return (repo.ui.config('experimental', 'copies.read-from') in
+('changeset-only', 'compatibility'))
 
 def _committedforwardcopies(a, b, match):
 """Like _forwardcopies(), but b.rev() cannot be None (working copy)"""
diff --git a/mercurial/context.py b/mercurial/context.py
--- a/mercurial/context.py
+++ b/mercurial/context.py
@@ -441,6 +441,21 @@
 return self._changeset.files
 @propertycache
 def _copies(self

D6218: narrow: send specs as bundle2 data instead of param (issue5952) (issue6019)

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG8a37c9b6cf0a: narrow: send specs as bundle2 data instead of 
param (issue5952) (issue6019) (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6218?vs=14711&id=14765

REVISION DETAIL
  https://phab.mercurial-scm.org/D6218

AFFECTED FILES
  hgext/narrow/narrowbundle2.py
  mercurial/exchange.py

CHANGE DETAILS

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -2214,12 +2214,14 @@
 if (kwargs.get(r'narrow', False) and kwargs.get(r'narrow_acl', False)
 and (include or exclude)):
 narrowspecpart = bundler.newpart('narrow:spec')
+data = ''
 if include:
-narrowspecpart.addparam(
-'include', '\n'.join(include), mandatory=True)
+data += '\n'.join(include)
+data += '\0'
 if exclude:
-narrowspecpart.addparam(
-'exclude', '\n'.join(exclude), mandatory=True)
+data += '\n'.join(exclude)
+
+narrowspecpart.data = data
 
 @getbundle2partsgenerator('bookmarks')
 def _getbundlebookmarkpart(bundler, repo, source, bundlecaps=None,
diff --git a/hgext/narrow/narrowbundle2.py b/hgext/narrow/narrowbundle2.py
--- a/hgext/narrow/narrowbundle2.py
+++ b/hgext/narrow/narrowbundle2.py
@@ -144,6 +144,15 @@
 def _handlechangespec_2(op, inpart):
 includepats = set(inpart.params.get(_SPECPART_INCLUDE, '').splitlines())
 excludepats = set(inpart.params.get(_SPECPART_EXCLUDE, '').splitlines())
+data = inpart.read()
+# old servers don't send includes and excludes using bundle2 data, they use
+# bundle2 parameters instead.
+if data:
+inc, exc = data.split('\0')
+if inc:
+includepats |= set(inc.splitlines())
+if exc:
+excludepats |= set(exc.splitlines())
 narrowspec.validatepatterns(includepats)
 narrowspec.validatepatterns(excludepats)
 



To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: idlsoft, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6183: copies: add config option for writing copy metadata to file and/or changset

[martinvonz (Martin von Zweigbergk)]

martinvonz added a comment.


  In https://phab.mercurial-scm.org/D6183#90738, @yuja wrote:
  
  > >   > >   >  extras field. Almost all extras data are texts, and IIRC we 
regret that
  > >   > >   >  transplant sources are stored in binary form.
  > >   > >   
  > >   > >   Why not? I picked \0 and \n because they won't appear in 
filenames, so it's convenient in that way.
  > >   >
  > >   > I don't remember, but we do store even boolean value as text, not in 
binary
  > >   >  `\0`/`\1` form. `transplant_source` is the solo exception.
  > >   
  > >   
  > >   Perhaps it's just so `{extras}` doesn't print ANSI escape codes and 
such? (I assume that can still happen if put escape characters in your 
filenames, for example.)
  >
  > Ok. That might be the reason, and I'm fine with the `\0` separator.
  >
  > >   > https://www.mercurial-scm.org/wiki/ChangesetExtra
  > >   > 
  > >   > And if we pick \0/\n separators, _string_escape() wouldn't be needed
  > >   >  at the encodecopies() layer.
  > >   
  > >   Oh, now I see what you're saying! That's embarrassing. So maybe we 
should `_string_escape()` the whole thing? I'll do that.
  >
  > Not really. I meant `_string_escape()` could be removed entirely if we store
  >  copies in binary (valid_filename + invalid_filename_separator) form. The 
extra
  >  dict will be encoded later.
  
  
  Sure, if we're okay with the `\0` and `\n` separators being printed to the 
terminal when the user uses the `{extras}` template, then we can just drop the 
encoding. Sounds like you're okay with that, and I also don't care too much, so 
I'll drop the encoding.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6183

To: martinvonz, #hg-reviewers
Cc: yuja, pulkit, gracinet, marmoute, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6218: narrow: send specs as bundle2 data instead of param (issue5952) (issue6019)

[martinvonz (Martin von Zweigbergk)]

martinvonz added inline comments.

INLINE COMMENTS

> narrowbundle2.py:152
> +inc, exc = data.split('\0')
> +if inc:
> +includepats |= set(inc.splitlines())

I'd drop these checks

> exchange.py:2218
> +data = ''
>  if include:
> +data += '\n'.join(include)

And these

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6218

To: pulkit, durin42, martinvonz, #hg-reviewers
Cc: idlsoft, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@42130: 5 new changesets

[Mercurial Commits]

5 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/ef0e3cc684b3
changeset:   42126:ef0e3cc684b3
user:Pierre-Yves David 
date:Mon May 21 17:28:35 2018 +0200
summary: repoview: introduce a filter for serving hidden changesets

https://www.mercurial-scm.org/repo/hg/rev/70b71421fd33
changeset:   42127:70b71421fd33
user:Pierre-Yves David 
date:Thu Apr 04 13:58:49 2019 +0200
summary: repoview: keep the branchmap cache for the `served.hidden` view 
warm

https://www.mercurial-scm.org/repo/hg/rev/537a8aeb9977
changeset:   42128:537a8aeb9977
user:Rodrigo Damazio Bovendorp 
date:Mon Apr 15 22:13:11 2019 -0700
summary: absorb: aborting if another operation is in progress

https://www.mercurial-scm.org/repo/hg/rev/232a33a11ce0
changeset:   42129:232a33a11ce0
user:Pierre-Yves David 
date:Fri Apr 05 15:56:05 2019 +0200
summary: pull: deal with locally filtered changeset passed into --rev

https://www.mercurial-scm.org/repo/hg/rev/a362b0b95e42
changeset:   42130:a362b0b95e42
bookmark:@
tag: tip
user:Pierre-Yves David 
date:Fri Apr 05 14:35:33 2019 +0200
summary: pull: improved message issued in case of failed update

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6240: branchcache: only iterate over branches which needs to be verified

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG75e8e16ca107: branchcache: only iterate over branches which 
needs to be verified (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6240?vs=14758&id=14764

REVISION DETAIL
  https://phab.mercurial-scm.org/D6240

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -200,7 +200,8 @@
 
 def _verifyall(self):
 """ verifies nodes of all the branches """
-for b in self._entries:
+needverification = set(self._entries.keys()) - self._verifiedbranches
+for b in needverification:
 self._verifybranch(b)
 
 def __iter__(self):



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6236: branchcache: lazily validate nodes in iteritems()

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGf0203c3406e7: branchcache: lazily validate nodes in 
iteritems() (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6236?vs=14759&id=14763

REVISION DETAIL
  https://phab.mercurial-scm.org/D6236

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -219,8 +219,9 @@
 return key in self._entries
 
 def iteritems(self):
-self._verifyall()
-return self._entries.iteritems()
+for k, v in self._entries.iteritems():
+self._verifybranch(k)
+yield k, v
 
 def hasbranch(self, label):
 """ checks whether a branch of this name exists or not """



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6239: branchcache: fix the docstring of _verifybranch()

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGd9dc0896e1d3: branchcache: fix the docstring of 
_verifybranch() (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6239?vs=14757&id=14762

REVISION DETAIL
  https://phab.mercurial-scm.org/D6239

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -189,8 +189,7 @@
 self._closedverified = True
 
 def _verifybranch(self, branch):
-""" verify head nodes for the given branch. If branch is None, verify
-for all the branches """
+""" verify head nodes for the given branch. """
 if branch not in self._entries or branch in self._verifiedbranches:
 return
 for n in self._entries[branch]:



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6238: branchcache: don't verify while creating a copy

[pulkit (Pulkit Goyal)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHGb26455ffa7fa: branchcache: don't verify while creating 
a copy (authored by pulkit, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6238?vs=14756&id=14761

REVISION DETAIL
  https://phab.mercurial-scm.org/D6238

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -339,7 +339,6 @@
 
 def copy(self):
 """return an deep copy of the branchcache object"""
-self._verifyall()
 return type(self)(
 self._entries, self.tipnode, self.tiprev, self.filteredhash,
 self._closednodes)



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 3 "V4] upgrade: support upgrade to/from zstd storage (issue6088)

[Augie Fackler]

On Tue, Apr 16, 2019 at 03:26:07PM +0200, Pierre-Yves David wrote:
> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1549032662 -3600
> #  Fri Feb 01 15:51:02 2019 +0100
> # Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
> # Parent  cbc090e88191176d0d7708e069a98596df78c2c5
> # EXP-Topic zstd-revlog
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> 821f57c6dda8
> upgrade: support upgrade to/from zstd storage (issue6088)

These look fine, but I'm unclear if we should ship this in the RC or
hold for 5.1. Do you have a preference?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

mercurial@42125: 7 new changesets

[Mercurial Commits]

7 new changesets in mercurial:

https://www.mercurial-scm.org/repo/hg/rev/2d428b859282
changeset:   42119:2d428b859282
user:Kyle Lippincott 
date:Mon Apr 15 14:32:47 2019 -0700
summary: readline: provide styled prompt to readline (issue6070)

https://www.mercurial-scm.org/repo/hg/rev/2f8147521e59
changeset:   42120:2f8147521e59
user:Pulkit Goyal 
date:Tue Mar 19 16:52:15 2019 +0300
summary: branchcache: add functions to validate changelog nodes

https://www.mercurial-scm.org/repo/hg/rev/6578654916ae
changeset:   42121:6578654916ae
user:Pulkit Goyal 
date:Mon Apr 01 13:56:47 2019 +0300
summary: branchcache: lazily validate nodes from the branchmap

https://www.mercurial-scm.org/repo/hg/rev/7c9d4cf23adf
changeset:   42122:7c9d4cf23adf
user:Pulkit Goyal 
date:Fri Apr 05 15:56:33 2019 +0300
summary: branchcache: don't verify closed nodes in iteropen()

https://www.mercurial-scm.org/repo/hg/rev/be5eeaf5c24a
changeset:   42123:be5eeaf5c24a
user:Pulkit Goyal 
date:Fri Apr 05 15:57:09 2019 +0300
summary: branchcache: don't verify closed nodes in _branchtip()

https://www.mercurial-scm.org/repo/hg/rev/d6437f414437
changeset:   42124:d6437f414437
user:Pulkit Goyal 
date:Fri Apr 05 16:05:20 2019 +0300
summary: branch: return early if we find an open named branch apart from 
default

https://www.mercurial-scm.org/repo/hg/rev/bc15e37ecc16
changeset:   42125:bc15e37ecc16
bookmark:@
tag: tip
user:Pierre-Yves David 
date:Sat Apr 13 20:57:50 2019 +0200
summary: repoview: fix conditional around unserved changesets

-- 
Repository URL: https://www.mercurial-scm.org/repo/hg
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Changing the wdir() {rev} and {node} output

[Jordi Gutiérrez Hermoso]

While I think it might be okay to keep the iternal values,

https://www.mercurial-scm.org/repo/hg/file/967c098eed33/mercurial/node.py#l39

I think it's far too confusing to expose them in the user interface.

A particular inconsistency I see is that we already have a way to show
wdir(), for example. In a way, `hg id -T` is kind of an alias for `hg
log -r 'wdir()' -T ... ` in a way that is not obvious. But trying to
use '{node}' or '{rev}' in `hg id` results in exposing the internal
revision and node magic numbers, which is very confusing behaviour,
since the default behaviour of `hg id` is to show `{p1.rev}+` instead
if there are changes in the wdir.

Another instance in which this happens is `hg annotate -r 'wdir()'`,
which also shows plusses to indicate that the changes are only in the
working directory.

I think we should keep this behaviour throughout, for consistency.
Since `wdir()` is still marked as experimental, we can change its
current behaviour. I propose consistency with `hg id` as follows:

* If the working directory is clean, the following are equivalent:

  hg log -r 'wdir()' -T '{rev}:{node}'
  hg log -r . -T '{rev}:{node}'

* Otherwise, if there are changes in the working directory, the
  following are equivalent:

 hg log -r 'wdir()' -T '{rev}:{node}'
 hg log -r '.' -T'{rev}+:{node}+'

In other words, never show in the interface the internal values of
wdir() obtained from node.py

I think this would be consistent with the existing usage of `hg id`
and `hg log`, the main consumers of `wdir()`.

Thoughts?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 3 of 3 "V4] upgrade: support upgrade to/from zstd storage (issue6088)

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1549032662 -3600
#  Fri Feb 01 15:51:02 2019 +0100
# Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
# Parent  cbc090e88191176d0d7708e069a98596df78c2c5
# EXP-Topic zstd-revlog
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
821f57c6dda8
upgrade: support upgrade to/from zstd storage (issue6088)

Now that we have an official config option for a shiny format improvement, we
better make it simple to migrate to/from it.

diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -24,6 +24,10 @@ from . import (
 vfs as vfsmod,
 )
 
+from .utils import (
+compression,
+)
+
 def requiredsourcerequirements(repo):
 """Obtain requirements required to be present to upgrade a repo.
 
@@ -61,9 +65,16 @@ def supportremovedrequirements(repo):
 the dropped requirement must appear in the returned set for the upgrade
 to be allowed.
 """
-return {
+supported = {
 localrepo.SPARSEREVLOG_REQUIREMENT,
 }
+for name in compression.compengines:
+engine = compression.compengines[name]
+if engine.available() and engine.revlogheader():
+supported.add(b'exp-compression-%s' % name)
+if engine.name() == 'zstd':
+supported.add(b'revlog-compression-zstd')
+return supported
 
 def supporteddestrequirements(repo):
 """Obtain requirements that upgrade supports in the destination.
@@ -73,7 +84,7 @@ def supporteddestrequirements(repo):
 
 Extensions should monkeypatch this to add their custom requirements.
 """
-return {
+supported = {
 'dotencode',
 'fncache',
 'generaldelta',
@@ -81,6 +92,13 @@ def supporteddestrequirements(repo):
 'store',
 localrepo.SPARSEREVLOG_REQUIREMENT,
 }
+for name in compression.compengines:
+engine = compression.compengines[name]
+if engine.available() and engine.revlogheader():
+supported.add(b'exp-compression-%s' % name)
+if engine.name() == 'zstd':
+supported.add(b'revlog-compression-zstd')
+return supported
 
 def allowednewrequirements(repo):
 """Obtain requirements that can be added to a repository during upgrade.
@@ -92,12 +110,19 @@ def allowednewrequirements(repo):
 bad additions because the whitelist approach is safer and will prevent
 future, unknown requirements from accidentally being added.
 """
-return {
+supported = {
 'dotencode',
 'fncache',
 'generaldelta',
 localrepo.SPARSEREVLOG_REQUIREMENT,
 }
+for name in compression.compengines:
+engine = compression.compengines[name]
+if engine.available() and engine.revlogheader():
+supported.add(b'exp-compression-%s' % name)
+if engine.name() == 'zstd':
+supported.add(b'revlog-compression-zstd')
+return supported
 
 def preservedrequirements(repo):
 return set()
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
--- a/tests/test-upgrade-repo.t
+++ b/tests/test-upgrade-repo.t
@@ -854,4 +854,78 @@ Check that we can remove the sparse-revl
   generaldelta
   revlogv1
   store
+
+#if zstd
+
+Check upgrading to a zstd revlog
+
+
+upgrade
+
+  $ hg --config format.revlog-compression=zstd debugupgraderepo --run  
--no-backup >/dev/null
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  plain-cl-delta:yesyes yes
+  compression:   zstd   zlibzlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlog-compression-zstd
+  revlogv1
+  sparserevlog
+  store
+
+downgrade
+
+  $ hg debugupgraderepo --run --no-backup > /dev/null
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  plain-cl-delta:yesyes yes
+  compression:   zlib   zlibzlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlogv1
+  sparserevlog
+  store
+
+upgrade from hgrc
+
+  $ cat >> .hg/hgrc << EOF
+  > [format]
+  > revlog-compression=zstd
+  > EOF
+  $ hg debugupgraderepo --run --no-backup > /dev/null
+  $ hg debugformat -v
+  format-variantrepo config default
+  fncache:   yesyes yes
+  dotencode: yesyes yes
+  generaldelta:  yesyes yes
+  sparserevlog:  yesyes yes
+  plain-cl-delta:yesyes yes
+  compression:  

[PATCH 2 of 3 "V4] compression: introduce an official `zstd-revlog` requirement

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1553707623 -3600
#  Wed Mar 27 18:27:03 2019 +0100
# Node ID cbc090e88191176d0d7708e069a98596df78c2c5
# Parent  32be9488a4f86228802c523512b29a17b420b3ea
# EXP-Topic zstd-revlog
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
cbc090e88191
compression: introduce an official `zstd-revlog` requirement

This requirement supersedes `exp-compression-zstd`. However, we keep support for
the old requirement.

Strictly speaking, we do not need to add a new requirement, there are no logic
change making "new" repo incompatible with mercurial client using a version
that support `exp-compression-zstd`.

The choice to introduce a new requirement is motivated by the following:

* The previous requirement was explicitly "experimental". Using it by default
  could confuse users.

* adding support for a hypothetical third compression engine will requires new
  code, and will comes with its own requirement tool.

* We won't use it as the default for a while since I do not think we support
  zstd on all platform. I can imagine we'll gain another (unrelated but on my
  default) requirement by the time we turn this zstd by default.

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -645,6 +645,8 @@ def gathersupportedrequirements(ui):
 engine = util.compengines[name]
 if engine.available() and engine.revlogheader():
 supported.add(b'exp-compression-%s' % name)
+if engine.name() == 'zstd':
+supported.add(b'revlog-compression-zstd')
 
 return supported
 
@@ -794,8 +796,13 @@ def resolverevlogstorevfsoptions(ui, req
 options[b'maxchainlen'] = maxchainlen
 
 for r in requirements:
-if r.startswith(b'exp-compression-'):
-options[b'compengine'] = r[len(b'exp-compression-'):]
+# we allow multiple compression engine requirement to co-exist because
+# strickly speaking, revlog seems to support mixed compression style.
+#
+# The compression used for new entries will be "the last one"
+prefix = r.startswith
+if prefix('revlog-compression-') or prefix('exp-compression-'):
+options[b'compengine'] = r.split('-', 2)[2]
 
 options[b'zlib.level'] = ui.configint(b'storage', b'revlog.zlib.level')
 if options[b'zlib.level'] is not None:
@@ -2942,7 +2949,9 @@ def newreporequirements(ui, createopts):
  'compression engines'))
 
 # zlib is the historical default and doesn't need an explicit requirement.
-if compengine != 'zlib':
+elif compengine == 'zstd':
+requirements.add('revlog-compression-zstd')
+elif compengine != 'zlib':
 requirements.add('exp-compression-%s' % compengine)
 
 if scmutil.gdinitconfig(ui):
diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -325,10 +325,16 @@ class compressionengine(formatvariant):
 
 @classmethod
 def fromrepo(cls, repo):
+# we allow multiple compression engine requirement to co-exist because
+# strickly speaking, revlog seems to support mixed compression style.
+#
+# The compression used for new entries will be "the last one"
+compression = 'zlib'
 for req in repo.requirements:
-if req.startswith('exp-compression-'):
-return req.split('-', 2)[2]
-return 'zlib'
+prefix = req.startswith
+if prefix('revlog-compression-') or prefix('exp-compression-'):
+compression = req.split('-', 2)[2]
+return compression
 
 @classmethod
 def fromconfig(cls, repo):
diff --git a/tests/test-repo-compengines.t b/tests/test-repo-compengines.t
--- a/tests/test-repo-compengines.t
+++ b/tests/test-repo-compengines.t
@@ -44,9 +44,9 @@ A requirement specifying an unknown comp
   $ cd zstd
   $ cat .hg/requires
   dotencode
-  exp-compression-zstd
   fncache
   generaldelta
+  revlog-compression-zstd
   revlogv1
   sparserevlog
   store
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 3 "V4] compression: only declare revlog support for available engine

[Pierre-Yves David]

# HG changeset patch
# User Pierre-Yves David 
# Date 1555420216 -7200
#  Tue Apr 16 15:10:16 2019 +0200
# Node ID 32be9488a4f86228802c523512b29a17b420b3ea
# Parent  967c098eed3398d991cbd7f0d48636b575beb711
# EXP-Topic zstd-revlog
# Available At https://bitbucket.org/octobus/mercurial-devel/
#  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
32be9488a4f8
compression: only declare revlog support for available engine

Even if we know that an engine support revlog compression, the Mercurial process
still won't support it if the compression engine is not available.

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -643,7 +643,7 @@ def gathersupportedrequirements(ui):
 # Add derived requirements from registered compression engines.
 for name in util.compengines:
 engine = util.compengines[name]
-if engine.revlogheader():
+if engine.available() and engine.revlogheader():
 supported.add(b'exp-compression-%s' % name)
 
 return supported
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2 "V3] compression: introduce an official `zstd-revlog` requirement

[Pierre-Yves David]

On 4/16/19 12:28 PM, Gregory Szorc wrote:
On Mon, Apr 15, 2019 at 8:34 AM Augie Fackler > wrote:


Greg: ping on these? I'm aware you had some unease on this topic, and
I'd like to figure out if we should ship this. Thanks!


I've been enjoying my holiday away from the computer. I met with 
Pierre-Yves and Georges yesterday and agreed to review these before the 
freeze...



On Tue, Apr 09, 2019 at 12:06:00PM +0200, Pierre-Yves David wrote:
 > # HG changeset patch
 > # User Pierre-Yves David mailto:pierre-yves.da...@octobus.net>>
 > # Date 1553707623 -3600
 > #      Wed Mar 27 18:27:03 2019 +0100
 > # Node ID 8ce788576265eb8b84cb697ea2e09b2246fb7b81
 > # Parent  456c37433c433ee59d321315da24eb944e495f08
 > # EXP-Topic zstd-revlog
 > # Available At https://bitbucket.org/octobus/mercurial-devel/
 > #              hg pull
https://bitbucket.org/octobus/mercurial-devel/ -r 8ce788576265
 > compression: introduce an official `zstd-revlog` requirement
 >
 > This requirement supersedes `exp-compression-zstd`. However, we
keep support for
 > the old requirement.
 >
 > Strictly speaking, we do not need to add a new requirement, there
are no logic
 > change making "new" repo incompatible with mercurial client using
a version
 > that support `exp-compression-zstd`.
 >
 > The choice to introduce a new requirement is motivated by the
following:
 >
 > * The previous requirement was explicitly "experimental". Using
it by default
 >   could confuse users.
 >
 > * adding support for a hypothetical third compression engine will
requires new
 >   code, and will comes with its own requirement tool.
 >
 > * We won't use it as the default for a while since I do not think
we support
 >   zstd on all platform. I can imagine we'll gain another
(unrelated but on my
 >   default) requirement by the time we turn this zstd by default.


So, the reason why I named things "compression-" was so we could 
have a per-compressor requirement that could potentially allow 
*anything* to use that compressor. For example, "compression-zstd" could 
allow random on-disk files to be compressed with zstd.


Of course, *any* time you make a BC change to how the repository is read 
from the filesystem, you need a new requirement, otherwise old clients 
may attempt to open the repo and get confused or cause corruption. So 
even if we have an e.g. "compression-zstd" requirement, we would still 
need additional requirements to denote when zstd support is added to 
individual components. e.g. we'd need a dedicated requirement for "zstd 
in revlogs."


I think I'm OK with the approach in this patch.

 >
 > diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
 > --- a/mercurial/localrepo.py
 > +++ b/mercurial/localrepo.py
 > @@ -645,6 +645,8 @@ def gathersupportedrequirements(ui):
 >          engine = util.compengines[name]
 >          if engine.revlogheader():
 >              supported.add(b'exp-compression-%s' % name)
 > +            if engine.name () == 'zstd':
 > +                supported.add(b'revlog-compression-zstd')


I'm still not a fan of special casing zstd: there should be something on 
the compression engine API doing this. But I'm willing to look the other 
way for now because doing it as part of the compression engine API means 
stabilizing the "exp-compression-" bit and I don't want to do 
that just yet.


There is also an existing bug where we fail to check engine.available(). 
This will result in Mercurial thinking it can open zstd repositories and 
then failing when it tries to use zstd when it isn't available. 
(Compression engines are always present/registered but they may not be 
available at run-time due to missing dependencies.)


Could you please send a patch (either standalone or a v4 of this one) 
that adds a check for engine.available() and skips engines that aren't 
usable?


I am on it. It looks like only one line needs to be updated. I am 
sending a V4 with this extra patch + update to the last patch.


--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 04 of 14 "] pull: deal with locally filtered changeset passed into --rev

[Pulkit Goyal]

On Tue, Apr 16, 2019 at 3:37 PM Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

>
>
> On 4/16/19 12:56 PM, Gregory Szorc wrote:
> > On Tue, Apr 16, 2019 at 4:54 AM Gregory Szorc  > > wrote:
> >
> > On Sat, Apr 13, 2019 at 5:56 PM Pierre-Yves David
> >  > > wrote:
> >
> > # HG changeset patch
> > # User Pierre-Yves David  > >
> > # Date 1554472565 -7200
> > #  Fri Apr 05 15:56:05 2019 +0200
> > # Node ID 0adcfded9b03fff84190594ef29e37110967419f
> > # Parent  d5f42ea7b06825ee86620cdc18aaa3a53504bff5
> > # EXP-Topic hgweb-obsolete
> > # Available At https://bitbucket.org/octobus/mercurial-devel/
> > #  hg pull
> > https://bitbucket.org/octobus/mercurial-devel/ -r 0adcfded9b03
> > pull: deal with locally filtered changeset passed into --rev
> >
> > Nowadays, it is possible to explicitly pull a remote revision
> > that end up being
> > hidden locally (eg: obsoleted locally). However before this
> > patch, some
> > internal processing where crashing trying to resolve a filtered
> > revision.
> >
> > Without this patches, the pull output result a confusing output:
> >
> >$ hg pull ../repo-Bob --rev 956063ac4557
> >pulling from ../repo-Bob
> >searching for changes
> >adding changesets
> >adding manifests
> >adding file changes
> >added 2 changesets with 0 changes to 2 files (+1 heads)
> >(2 other changesets obsolete on arrival)
> >abort:
> > 00changelog.i@956063ac4557828781733b2d5677a351ce856f59: filtered
> > node!
> >
> >
> > The existing abort message is bad and should be improved because
> > typical users won't have a clue what it means.
> >
> > But I have reservations about this patch because it isn't clear what
> > will happen with `pull -u -r `. If the working directory
> > will be updated to a hidden revision without --hidden specified,
> > this feels wrong to me.
> >
> >
> > Oh - maybe part 5 (and later?) address my concerns?
>
> It does. Regarding the abort message. I know we have a better message
> available for this kind of error. I am not sure why that better message
> is not used here and I intend to dig into that next cycle (there are a
> couple of usual suspect: wrong exception types, bits still in evolve
> extension, etc…)
>
> > Could we get test coverage showing what happens in this case? Please
> > also check for behavior with `hg clone -r ` and `hg clone -u
> > ` as well.
>
> (as you already noticed, this is addressed in the next patch).
>

Queued 4 and 5. Many thanks!

>
> > Also, I'm a little confused about "checkout" and "brev" both doing
> > similar things. It seems that "checkout" is used internally and
> > "brev" is used for user-facing output. I wish this code were better
> > documented. But that is scope bloat...
>
> +1
>

I also tried to understand and was not successful. Maybe hg.addbranchrevs()
needs docs.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6237: absorb: aborting if another operation is in progress

[rdamazio (Rodrigo Damazio Bovendorp)]

This revision was automatically updated to reflect the committed changes.
Closed by commit rHG537a8aeb9977: absorb: aborting if another operation is in 
progress (authored by rdamazio, committed by ).

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6237?vs=14755&id=14760

REVISION DETAIL
  https://phab.mercurial-scm.org/D6237

AFFECTED FILES
  hgext/absorb.py
  tests/test-absorb-unfinished.t

CHANGE DETAILS

diff --git a/tests/test-absorb-unfinished.t b/tests/test-absorb-unfinished.t
new file mode 100644
--- /dev/null
+++ b/tests/test-absorb-unfinished.t
@@ -0,0 +1,30 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > absorb=
+  > EOF
+
+Abort absorb if there is an unfinished operation.
+
+  $ hg init abortunresolved
+  $ cd abortunresolved
+
+  $ echo "foo1" > foo.whole
+  $ hg commit -Aqm "foo 1"
+
+  $ hg update null
+  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
+  $ echo "foo2" > foo.whole
+  $ hg commit -Aqm "foo 2"
+
+  $ hg --config extensions.rebase= rebase -r 1 -d 0
+  rebasing 1:c3b6dc0e177a "foo 2" (tip)
+  merging foo.whole
+  warning: conflicts while merging foo.whole! (edit, then use 'hg resolve 
--mark')
+  unresolved conflicts (see hg resolve, then hg rebase --continue)
+  [1]
+
+  $ hg --config extensions.rebase= absorb
+  abort: rebase in progress
+  (use 'hg rebase --continue' or 'hg rebase --abort')
+  [255]
+
diff --git a/hgext/absorb.py b/hgext/absorb.py
--- a/hgext/absorb.py
+++ b/hgext/absorb.py
@@ -682,13 +682,12 @@
 
 def commit(self):
 """commit changes. update self.finalnode, self.replacemap"""
-with self.repo.wlock(), self.repo.lock():
-with self.repo.transaction('absorb') as tr:
-self._commitstack()
-self._movebookmarks(tr)
-if self.repo['.'].node() in self.replacemap:
-self._moveworkingdirectoryparent()
-self._cleanupoldcommits()
+with self.repo.transaction('absorb') as tr:
+self._commitstack()
+self._movebookmarks(tr)
+if self.repo['.'].node() in self.replacemap:
+self._moveworkingdirectoryparent()
+self._cleanupoldcommits()
 return self.finalnode
 
 def printchunkstats(self):
@@ -1006,6 +1005,11 @@
 Returns 0 on success, 1 if all chunks were ignored and nothing amended.
 """
 opts = pycompat.byteskwargs(opts)
-state = absorb(ui, repo, pats=pats, opts=opts)
-if sum(s[0] for s in state.chunkstats.values()) == 0:
-return 1
+
+with repo.wlock(), repo.lock():
+if not opts['dry_run']:
+cmdutil.checkunfinished(repo)
+
+state = absorb(ui, repo, pats=pats, opts=opts)
+if sum(s[0] for s in state.chunkstats.values()) == 0:
+return 1



To: rdamazio, #hg-reviewers, pulkit
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6207: branchcache: add functions to validate changelog nodes

[pulkit (Pulkit Goyal)]

pulkit added a comment.


  In https://phab.mercurial-scm.org/D6207#90778, @yuja wrote:
  
  > Queued, thanks.
  >
  > > +def _verifybranch(self, branch):
  > >  +""" verify head nodes for the given branch. If branch is None, 
verify
  > >  +for all the branches """
  >
  > "If branch is None, ..." appears wrong.
  >
  > > +if branch not in self._entries or branch in 
self._verifiedbranches:
  > >  +return
  > >  +for n in self._entries[branch]:
  > >  +if not self._hasnode(n):
  > >  +_unknownnode(n)
  > >  +
  > >  +self._verifiedbranches.add(branch)
  >
  > Regarding https://phab.mercurial-scm.org/D6236, `_verifiedbranches` could 
be inverted (i.e. a set of branches
  >  to be verified) so that `_verifyall()` can return early. I don't know which
  >  will be faster, but in principle, fewer loops and Python gives a better 
result.
  
  
  In _verifyall(), I changed code to calculate a needverification set and then 
iterate over it.
  
  The motivation behind iteritems() change 
(https://phab.mercurial-scm.org/D6236) is optimizing 
https://www.mercurial-scm.org/repo/hg-committed/file/70b71421fd33/mercurial/commands.py#l1128.
  
  Maybe we can iterate over verified nodes first, and then iterate over 
unverified nodes?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6207

To: pulkit, #hg-reviewers
Cc: yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6236: branchcache: lazily validate nodes in iteritems()

[pulkit (Pulkit Goyal)]

pulkit updated this revision to Diff 14759.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6236?vs=14743&id=14759

REVISION DETAIL
  https://phab.mercurial-scm.org/D6236

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -219,8 +219,9 @@
 return key in self._entries
 
 def iteritems(self):
-self._verifyall()
-return self._entries.iteritems()
+for k, v in self._entries.iteritems():
+self._verifybranch(k)
+yield k, v
 
 def hasbranch(self, label):
 """ checks whether a branch of this name exists or not """



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 04 of 14 "] pull: deal with locally filtered changeset passed into --rev

[Pierre-Yves David]

On 4/16/19 12:56 PM, Gregory Szorc wrote:
On Tue, Apr 16, 2019 at 4:54 AM Gregory Szorc > wrote:


On Sat, Apr 13, 2019 at 5:56 PM Pierre-Yves David
mailto:pierre-yves.da...@ens-lyon.org>> wrote:

# HG changeset patch
# User Pierre-Yves David mailto:pierre-yves.da...@octobus.net>>
# Date 1554472565 -7200
#      Fri Apr 05 15:56:05 2019 +0200
# Node ID 0adcfded9b03fff84190594ef29e37110967419f
# Parent  d5f42ea7b06825ee86620cdc18aaa3a53504bff5
# EXP-Topic hgweb-obsolete
# Available At https://bitbucket.org/octobus/mercurial-devel/
#              hg pull
https://bitbucket.org/octobus/mercurial-devel/ -r 0adcfded9b03
pull: deal with locally filtered changeset passed into --rev

Nowadays, it is possible to explicitly pull a remote revision
that end up being
hidden locally (eg: obsoleted locally). However before this
patch, some
internal processing where crashing trying to resolve a filtered
revision.

Without this patches, the pull output result a confusing output:

   $ hg pull ../repo-Bob --rev 956063ac4557
   pulling from ../repo-Bob
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 2 changesets with 0 changes to 2 files (+1 heads)
   (2 other changesets obsolete on arrival)
   abort:
00changelog.i@956063ac4557828781733b2d5677a351ce856f59: filtered
node!


The existing abort message is bad and should be improved because
typical users won't have a clue what it means.

But I have reservations about this patch because it isn't clear what
will happen with `pull -u -r `. If the working directory
will be updated to a hidden revision without --hidden specified,
this feels wrong to me.


Oh - maybe part 5 (and later?) address my concerns?


It does. Regarding the abort message. I know we have a better message 
available for this kind of error. I am not sure why that better message 
is not used here and I intend to dig into that next cycle (there are a 
couple of usual suspect: wrong exception types, bits still in evolve 
extension, etc…)



Could we get test coverage showing what happens in this case? Please
also check for behavior with `hg clone -r ` and `hg clone -u
` as well.


(as you already noticed, this is addressed in the next patch).


Also, I'm a little confused about "checkout" and "brev" both doing
similar things. It seems that "checkout" is used internally and
"brev" is used for user-facing output. I wish this code were better
documented. But that is scope bloat...


+1

Cheers,

--
Pierre-Yves David
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6240: branchcache: only iterate over branches which needs to be verified

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Otherwise we loop over all the branches and call _verifybranch() even if not
  required.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6240

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -200,7 +200,8 @@
 
 def _verifyall(self):
 """ verifies nodes of all the branches """
-for b in self._entries:
+needverification = set(self._entries.keys()) - self._verifiedbranches
+for b in needverification:
 self._verifybranch(b)
 
 def __iter__(self):



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6239: branchcache: fix the docstring of _verifybranch()

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Initially the function was designed to support verifying all branches but 
later
  I decided to have a separate function. I forget to remove the doc related to
  that. Thanks to Yuya for spotting this in review.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6239

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -189,8 +189,7 @@
 self._closedverified = True
 
 def _verifybranch(self, branch):
-""" verify head nodes for the given branch. If branch is None, verify
-for all the branches """
+""" verify head nodes for the given branch. """
 if branch not in self._entries or branch in self._verifiedbranches:
 return
 for n in self._entries[branch]:



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D6238: branchcache: don't verify while creating a copy

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  The copy will not be mark as verified, so there is no need to verify nodes.
  Thanks to Yuya who spotted this while reviewing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6238

AFFECTED FILES
  mercurial/branchmap.py

CHANGE DETAILS

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -339,7 +339,6 @@
 
 def copy(self):
 """return an deep copy of the branchcache object"""
-self._verifyall()
 return type(self)(
 self._entries, self.tipnode, self.tiprev, self.filteredhash,
 self._closednodes)



To: pulkit, #hg-reviewers
Cc: mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 04 of 14 "] pull: deal with locally filtered changeset passed into --rev

[Gregory Szorc]

On Tue, Apr 16, 2019 at 4:54 AM Gregory Szorc 
wrote:

> On Sat, Apr 13, 2019 at 5:56 PM Pierre-Yves David <
> pierre-yves.da...@ens-lyon.org> wrote:
>
>> # HG changeset patch
>> # User Pierre-Yves David 
>> # Date 1554472565 -7200
>> #  Fri Apr 05 15:56:05 2019 +0200
>> # Node ID 0adcfded9b03fff84190594ef29e37110967419f
>> # Parent  d5f42ea7b06825ee86620cdc18aaa3a53504bff5
>> # EXP-Topic hgweb-obsolete
>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
>> 0adcfded9b03
>> pull: deal with locally filtered changeset passed into --rev
>>
>> Nowadays, it is possible to explicitly pull a remote revision that end up
>> being
>> hidden locally (eg: obsoleted locally). However before this patch, some
>> internal processing where crashing trying to resolve a filtered revision.
>>
>> Without this patches, the pull output result a confusing output:
>>
>>   $ hg pull ../repo-Bob --rev 956063ac4557
>>   pulling from ../repo-Bob
>>   searching for changes
>>   adding changesets
>>   adding manifests
>>   adding file changes
>>   added 2 changesets with 0 changes to 2 files (+1 heads)
>>   (2 other changesets obsolete on arrival)
>>   abort: 00changelog.i@956063ac4557828781733b2d5677a351ce856f59:
>> filtered node!
>>
>
> The existing abort message is bad and should be improved because typical
> users won't have a clue what it means.
>
> But I have reservations about this patch because it isn't clear what will
> happen with `pull -u -r `. If the working directory will be updated
> to a hidden revision without --hidden specified, this feels wrong to me.
>

Oh - maybe part 5 (and later?) address my concerns?


>
> Could we get test coverage showing what happens in this case? Please also
> check for behavior with `hg clone -r ` and `hg clone -u ` as
> well.
>
> Also, I'm a little confused about "checkout" and "brev" both doing similar
> things. It seems that "checkout" is used internally and "brev" is used for
> user-facing output. I wish this code were better documented. But that is
> scope bloat...
>
>
>>
>> diff --git a/mercurial/commands.py b/mercurial/commands.py
>> --- a/mercurial/commands.py
>> +++ b/mercurial/commands.py
>> @@ -4488,7 +4488,7 @@ def pull(ui, repo, source="default", **o
>>  brev = None
>>
>>  if checkout:
>> -checkout = repo.changelog.rev(checkout)
>> +checkout = repo.unfiltered().changelog.rev(checkout)
>>
>>  # order below depends on implementation of
>>  # hg.addbranchrevs(). opts['bookmark'] is ignored,
>> diff --git a/tests/test-obsolete-distributed.t
>> b/tests/test-obsolete-distributed.t
>> --- a/tests/test-obsolete-distributed.t
>> +++ b/tests/test-obsolete-distributed.t
>> @@ -488,6 +488,22 @@ decision is made in that case, so receiv
>>d33b0a3a64647d79583526be8107802b1f9fedfa
>> 5b5708a437f27665db42c5a261a539a1bcb2a8c2 0 (Thu Jan 01 00:00:00 1970 +)
>> {'ef1': '1', 'operation': 'amend', 'user': 'bob'}
>>ef908e42ce65ef57f970d799acaddde26f58a4cc
>> 5ffb9e311b35f6ab6f76f667ca5d6e595645481b 0 (Thu Jan 01 00:00:00 1970 +)
>> {'ef1': '4', 'operation': 'rebase', 'user': 'bob'}
>>
>> +
>> +Same tests, but with --rev, this prevent regressing case where `hg pull
>> --rev
>> +X` has to process a X that is filtered locally.
>> +
>> +  $ hg rollback
>> +  repository tip rolled back to revision 4 (undo unbundle)
>> +  $ hg pull ../repo-Bob --rev 956063ac4557
>> +  pulling from ../repo-Bob
>> +  searching for changes
>> +  adding changesets
>> +  adding manifests
>> +  adding file changes
>> +  added 2 changesets with 0 changes to 2 files (+1 heads)
>> +  (2 other changesets obsolete on arrival)
>> +  (run 'hg heads' to see heads)
>> +
>>$ cd ..
>>
>>  Test pull report consistency
>> ___
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>>
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 04 of 14 "] pull: deal with locally filtered changeset passed into --rev

[Gregory Szorc]

On Sat, Apr 13, 2019 at 5:56 PM Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1554472565 -7200
> #  Fri Apr 05 15:56:05 2019 +0200
> # Node ID 0adcfded9b03fff84190594ef29e37110967419f
> # Parent  d5f42ea7b06825ee86620cdc18aaa3a53504bff5
> # EXP-Topic hgweb-obsolete
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 0adcfded9b03
> pull: deal with locally filtered changeset passed into --rev
>
> Nowadays, it is possible to explicitly pull a remote revision that end up
> being
> hidden locally (eg: obsoleted locally). However before this patch, some
> internal processing where crashing trying to resolve a filtered revision.
>
> Without this patches, the pull output result a confusing output:
>
>   $ hg pull ../repo-Bob --rev 956063ac4557
>   pulling from ../repo-Bob
>   searching for changes
>   adding changesets
>   adding manifests
>   adding file changes
>   added 2 changesets with 0 changes to 2 files (+1 heads)
>   (2 other changesets obsolete on arrival)
>   abort: 00changelog.i@956063ac4557828781733b2d5677a351ce856f59: filtered
> node!
>

The existing abort message is bad and should be improved because typical
users won't have a clue what it means.

But I have reservations about this patch because it isn't clear what will
happen with `pull -u -r `. If the working directory will be updated
to a hidden revision without --hidden specified, this feels wrong to me.

Could we get test coverage showing what happens in this case? Please also
check for behavior with `hg clone -r ` and `hg clone -u ` as
well.

Also, I'm a little confused about "checkout" and "brev" both doing similar
things. It seems that "checkout" is used internally and "brev" is used for
user-facing output. I wish this code were better documented. But that is
scope bloat...


>
> diff --git a/mercurial/commands.py b/mercurial/commands.py
> --- a/mercurial/commands.py
> +++ b/mercurial/commands.py
> @@ -4488,7 +4488,7 @@ def pull(ui, repo, source="default", **o
>  brev = None
>
>  if checkout:
> -checkout = repo.changelog.rev(checkout)
> +checkout = repo.unfiltered().changelog.rev(checkout)
>
>  # order below depends on implementation of
>  # hg.addbranchrevs(). opts['bookmark'] is ignored,
> diff --git a/tests/test-obsolete-distributed.t
> b/tests/test-obsolete-distributed.t
> --- a/tests/test-obsolete-distributed.t
> +++ b/tests/test-obsolete-distributed.t
> @@ -488,6 +488,22 @@ decision is made in that case, so receiv
>d33b0a3a64647d79583526be8107802b1f9fedfa
> 5b5708a437f27665db42c5a261a539a1bcb2a8c2 0 (Thu Jan 01 00:00:00 1970 +)
> {'ef1': '1', 'operation': 'amend', 'user': 'bob'}
>ef908e42ce65ef57f970d799acaddde26f58a4cc
> 5ffb9e311b35f6ab6f76f667ca5d6e595645481b 0 (Thu Jan 01 00:00:00 1970 +)
> {'ef1': '4', 'operation': 'rebase', 'user': 'bob'}
>
> +
> +Same tests, but with --rev, this prevent regressing case where `hg pull
> --rev
> +X` has to process a X that is filtered locally.
> +
> +  $ hg rollback
> +  repository tip rolled back to revision 4 (undo unbundle)
> +  $ hg pull ../repo-Bob --rev 956063ac4557
> +  pulling from ../repo-Bob
> +  searching for changes
> +  adding changesets
> +  adding manifests
> +  adding file changes
> +  added 2 changesets with 0 changes to 2 files (+1 heads)
> +  (2 other changesets obsolete on arrival)
> +  (run 'hg heads' to see heads)
> +
>$ cd ..
>
>  Test pull report consistency
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 01 of 14 "] repoview: fix conditional around unserved changesets

[Gregory Szorc]

On Sat, Apr 13, 2019 at 5:49 PM Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

> # HG changeset patch
> # User Pierre-Yves David 
> # Date 1555181870 -7200
> #  Sat Apr 13 20:57:50 2019 +0200
> # Node ID 5ce5915a48297de3f7464e8f149629799da5ca6f
> # Parent  2a3c0106ded9584938116ec2b1cbc7e21f13e1e1
> # EXP-Topic hgweb-obsolete
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 5ce5915a4829
> repoview: fix conditional around unserved changesets
>

Queued parts 1-3. I dropped the internal-phase requirement from part 2.


>
> The conditional could lead to wrong computation since we have more unserved
> changesets than just the "secret" phase.
>
> In addition, now that we have efficient caching of phased changesets, we
> don't
> need the conditional anymore.
>
> diff --git a/mercurial/repoview.py b/mercurial/repoview.py
> --- a/mercurial/repoview.py
> +++ b/mercurial/repoview.py
> @@ -93,8 +93,8 @@ def computeunserved(repo, visibilityexce
>  assert not repo.changelog.filteredrevs
>  # fast path in simple case to avoid impact of non optimised code
>  hiddens = filterrevs(repo, 'visible')
> -if phases.hassecret(repo):
> -secrets = repo._phasecache.getrevset(repo,
> phases.remotehiddenphases)
> +secrets = repo._phasecache.getrevset(repo, phases.remotehiddenphases)
> +if secrets:
>  return frozenset(hiddens | frozenset(secrets))
>  else:
>  return hiddens
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2 "V3] compression: introduce an official `zstd-revlog` requirement

[Gregory Szorc]

On Mon, Apr 15, 2019 at 8:34 AM Augie Fackler  wrote:

> Greg: ping on these? I'm aware you had some unease on this topic, and
> I'd like to figure out if we should ship this. Thanks!
>

I've been enjoying my holiday away from the computer. I met with
Pierre-Yves and Georges yesterday and agreed to review these before the
freeze...


>
> On Tue, Apr 09, 2019 at 12:06:00PM +0200, Pierre-Yves David wrote:
> > # HG changeset patch
> > # User Pierre-Yves David 
> > # Date 1553707623 -3600
> > #  Wed Mar 27 18:27:03 2019 +0100
> > # Node ID 8ce788576265eb8b84cb697ea2e09b2246fb7b81
> > # Parent  456c37433c433ee59d321315da24eb944e495f08
> > # EXP-Topic zstd-revlog
> > # Available At https://bitbucket.org/octobus/mercurial-devel/
> > #  hg pull https://bitbucket.org/octobus/mercurial-devel/
> -r 8ce788576265
> > compression: introduce an official `zstd-revlog` requirement
> >
> > This requirement supersedes `exp-compression-zstd`. However, we keep
> support for
> > the old requirement.
> >
> > Strictly speaking, we do not need to add a new requirement, there are no
> logic
> > change making "new" repo incompatible with mercurial client using a
> version
> > that support `exp-compression-zstd`.
> >
> > The choice to introduce a new requirement is motivated by the following:
> >
> > * The previous requirement was explicitly "experimental". Using it by
> default
> >   could confuse users.
> >
> > * adding support for a hypothetical third compression engine will
> requires new
> >   code, and will comes with its own requirement tool.
> >
> > * We won't use it as the default for a while since I do not think we
> support
> >   zstd on all platform. I can imagine we'll gain another (unrelated but
> on my
> >   default) requirement by the time we turn this zstd by default.
>

So, the reason why I named things "compression-" was so we could
have a per-compressor requirement that could potentially allow *anything*
to use that compressor. For example, "compression-zstd" could allow random
on-disk files to be compressed with zstd.

Of course, *any* time you make a BC change to how the repository is read
from the filesystem, you need a new requirement, otherwise old clients may
attempt to open the repo and get confused or cause corruption. So even if
we have an e.g. "compression-zstd" requirement, we would still need
additional requirements to denote when zstd support is added to individual
components. e.g. we'd need a dedicated requirement for "zstd in revlogs."

I think I'm OK with the approach in this patch.



> >
> > diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
> > --- a/mercurial/localrepo.py
> > +++ b/mercurial/localrepo.py
> > @@ -645,6 +645,8 @@ def gathersupportedrequirements(ui):
> >  engine = util.compengines[name]
> >  if engine.revlogheader():
> >  supported.add(b'exp-compression-%s' % name)
> > +if engine.name() == 'zstd':
> > +supported.add(b'revlog-compression-zstd')
>

I'm still not a fan of special casing zstd: there should be something on
the compression engine API doing this. But I'm willing to look the other
way for now because doing it as part of the compression engine API means
stabilizing the "exp-compression-" bit and I don't want to do that
just yet.

There is also an existing bug where we fail to check engine.available().
This will result in Mercurial thinking it can open zstd repositories and
then failing when it tries to use zstd when it isn't available.
(Compression engines are always present/registered but they may not be
available at run-time due to missing dependencies.)

Could you please send a patch (either standalone or a v4 of this one) that
adds a check for engine.available() and skips engines that aren't usable?


> >
> >  return supported
> >
> > @@ -794,8 +796,13 @@ def resolverevlogstorevfsoptions(ui, req
> >  options[b'maxchainlen'] = maxchainlen
> >
> >  for r in requirements:
> > -if r.startswith(b'exp-compression-'):
> > -options[b'compengine'] = r[len(b'exp-compression-'):]
> > +# we allow multiple compression engine requirement to co-exist
> because
> > +# strickly speaking, revlog seems to support mixed compression
> style.
> > +#
> > +# The compression used for new entries will be "the last one"
> > +prefix = r.startswith
> > +if prefix('revlog-compression-') or prefix('exp-compression-'):
> > +options[b'compengine'] = r.split('-', 2)[2]
>

An existing problem with this code is that `requirements` is a set and
iteration order will be undefined. So if we have multiple compressors in
play, we could inconsistently compress with different compressors for
different transactions! We should have some way to sort the priorities. But
this is scope bloat and an existing issue, so I'm inclined to ignore it for
now. An inline comment would be useful, however.


> >
> >  options[b'zlib.level'] = ui.config