[PATCH 1 of 2 STABLE V2] convert: move recode() from convcmd to common

2017-11-26 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1511745478 18000
#  Sun Nov 26 20:17:58 2017 -0500
# Branch stable
# Node ID 7208fc36b7765c36d31d0ccddb7087081617208c
# Parent  02845f7441aff30bc01975a5881cabfa922c12d4
convert: move recode() from convcmd to common

The next patch will need 'orig_encoding' in the p4 module, but importing convcmd
there creates an import cycle.

diff --git a/hgext/convert/common.py b/hgext/convert/common.py
--- a/hgext/convert/common.py
+++ b/hgext/convert/common.py
@@ -37,6 +37,14 @@
 s = base64.decodestring(s)
 return pickle.loads(s)
 
+orig_encoding = 'ascii'
+
+def recode(s):
+if isinstance(s, unicode):
+return s.encode(orig_encoding, 'replace')
+else:
+return s.decode('utf-8').encode(orig_encoding, 'replace')
+
 class MissingTool(Exception):
 pass
 
diff --git a/hgext/convert/convcmd.py b/hgext/convert/convcmd.py
--- a/hgext/convert/convcmd.py
+++ b/hgext/convert/convcmd.py
@@ -49,13 +49,7 @@
 svn_sink = subversion.svn_sink
 svn_source = subversion.svn_source
 
-orig_encoding = 'ascii'
-
-def recode(s):
-if isinstance(s, unicode):
-return s.encode(orig_encoding, 'replace')
-else:
-return s.decode('utf-8').encode(orig_encoding, 'replace')
+recode = common.recode
 
 def mapbranch(branch, branchmap):
 '''
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2 STABLE V2] convert: avoid wrong lfconvert defaults by moving configitems to core

2017-11-26 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1511667169 18000
#  Sat Nov 25 22:32:49 2017 -0500
# Branch stable
# Node ID 229917a294a28a844ea08bd6729d9b5e54d86329
# Parent  7208fc36b7765c36d31d0ccddb7087081617208c
convert: avoid wrong lfconvert defaults by moving configitems to core

The `hg lfconvert --to-normal` command uses the convert extension internally to
work its magic, but that produced devel-warn messages if the convert extension
wasn't loaded by the user.  The test in fcd2f9b06629 (modified here) wasn't
showing the warnings because the convert extension was loaded via $HGRCPATH.
Most of the config options default to None/False, but 'hg.usebranchnames' and
'hg.tagsbranch' are supposed to default to True and 'default' respectively.

The first iteration of this was to ui.setconfig() inside lfconvert, to force the
convert extension to load.  But there really is no precedent for doing this, and
check-config complained that 'extensions.convert' isn't documented.  Yuya
suggested this alternative.

The p4.encoding lambda had to be ditched in this move to prevent a crash,
because ui._config only invokes the function if the default value is callable,
not dynamicdefault.  Since the orig_encoding value is only being set prior to
creating the sink, I think it is OK (and the only test failure locally is the
same as before the move).

diff --git a/hgext/convert/__init__.py b/hgext/convert/__init__.py
--- a/hgext/convert/__init__.py
+++ b/hgext/convert/__init__.py
@@ -28,103 +28,6 @@
 # leave the attribute unspecified.
 testedwith = 'ships-with-hg-core'
 
-configtable = {}
-configitem = registrar.configitem(configtable)
-
-configitem('convert', 'cvsps.cache',
-default=True,
-)
-configitem('convert', 'cvsps.fuzz',
-default=60,
-)
-configitem('convert', 'cvsps.logencoding',
-default=None,
-)
-configitem('convert', 'cvsps.mergefrom',
-default=None,
-)
-configitem('convert', 'cvsps.mergeto',
-default=None,
-)
-configitem('convert', 'git.committeractions',
-default=lambda: ['messagedifferent'],
-)
-configitem('convert', 'git.extrakeys',
-default=list,
-)
-configitem('convert', 'git.findcopiesharder',
-default=False,
-)
-configitem('convert', 'git.remoteprefix',
-default='remote',
-)
-configitem('convert', 'git.renamelimit',
-default=400,
-)
-configitem('convert', 'git.saverev',
-default=True,
-)
-configitem('convert', 'git.similarity',
-default=50,
-)
-configitem('convert', 'git.skipsubmodules',
-default=False,
-)
-configitem('convert', 'hg.clonebranches',
-default=False,
-)
-configitem('convert', 'hg.ignoreerrors',
-default=False,
-)
-configitem('convert', 'hg.revs',
-default=None,
-)
-configitem('convert', 'hg.saverev',
-default=False,
-)
-configitem('convert', 'hg.sourcename',
-default=None,
-)
-configitem('convert', 'hg.startrev',
-default=None,
-)
-configitem('convert', 'hg.tagsbranch',
-default='default',
-)
-configitem('convert', 'hg.usebranchnames',
-default=True,
-)
-configitem('convert', 'ignoreancestorcheck',
-default=False,
-)
-configitem('convert', 'localtimezone',
-default=False,
-)
-configitem('convert', 'p4.encoding',
-default=lambda: convcmd.orig_encoding,
-)
-configitem('convert', 'p4.startrev',
-default=0,
-)
-configitem('convert', 'skiptags',
-default=False,
-)
-configitem('convert', 'svn.debugsvnlog',
-default=True,
-)
-configitem('convert', 'svn.trunk',
-default=None,
-)
-configitem('convert', 'svn.tags',
-default=None,
-)
-configitem('convert', 'svn.branches',
-default=None,
-)
-configitem('convert', 'svn.startrev',
-default=0,
-)
-
 # Commands definition was moved elsewhere to ease demandload job.
 
 @command('convert',
diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -53,7 +53,8 @@
 common.checktool('p4', abort=False)
 
 self.revmap = {}
-self.encoding = self.ui.config('convert', 'p4.encoding')
+self.encoding = self.ui.config('convert', 'p4.encoding',
+   common.orig_encoding)
 self.re_type = re.compile(
 "([a-z]+)?(text|binary|symlink|apple|resource|unicode|utf\d+)"
 "(\+\w+)?$")
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -208,6 +208,99 @@
 default=None,
 generic=True,
 )
+coreconfigitem('convert', 'cvsps.cache',
+default=True,
+)
+coreconfigitem('convert', 'cvsps.fuzz',
+default=60,
+)
+coreconfigitem('convert', 'cvsps.logencoding',
+default=None,
+)
+coreconfigitem('convert', 'cvsps.mergefrom',
+default=None,
+)
+coreconfigitem('convert', 'cvsps.mergeto',
+default=None,
+)
+coreconfigitem('convert', 'git.committeractions',
+default=lambda: ['messagedifferent'],
+)
+coreconfigitem('convert', 'git.extrakeys',
+default=list,
+)
+coreconfigitem('convert',

Re: [PATCH 2 of 2] cat: do not instantiate subrepo if no potential match in it

2017-11-26 Thread Matt Harbison 

On Sun, 26 Nov 2017 05:38:53 -0500, Yuya Nishihara  wrote:


On Sat, 25 Nov 2017 23:53:45 -0500, Matt Harbison wrote:
On Sat, 25 Nov 2017 02:35:04 -0500, Yuya Nishihara   
wrote:


> # HG changeset patch
> # User Yuya Nishihara 
> # Date 1511593310 -32400
> #  Sat Nov 25 16:01:50 2017 +0900
> # Node ID b6e526ee5d2662ca30a555ca51264b3e371fe44e
> # Parent  38e952030f0c4746257280d47f0f94b6cb9ddb41
> cat: do not instantiate subrepo if no potential match in it
>
> This fixes the test failure in hg-git.
>
> https://bitbucket.org/durin42/hg-git/issues/227/
>
> diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
> --- a/mercurial/cmdutil.py
> +++ b/mercurial/cmdutil.py
> @@ -3068,6 +3068,8 @@ def cat(ui, repo, ctx, matcher, basefm,
>  err = 0
> for subpath in sorted(ctx.substate):
> +if not matcher.visitdir(subpath):
> +continue

The basematcher contract says the result is undefined if False is  
returned
for any parent directory.  Do we need a subrepo utility method to break  
up

the subpath and test top down?  Consider 'directory/subrepo', where
'directory' is excluded.


(+CC Martin)

Yeah the doc says that, but not all callers seem to obey the  
restriction. So
I take it visitdir() won't return Falsy value if there is a possible  
match,
even though it might return non-False for Falsy case if any parents  
return

False.


It's probably beyond the scope of what you were fixing, but should all
subrepo recursion be guarded like this?

>  sub = ctx.sub(subpath)
>  try:
>  submatch = matchmod.subdirmatcher(subpath, matcher)


Maybe we'll need ctx.walksub(matcher) which yields (subrepo, submatcher)  
pairs?


I like it.

I wonder if the exception handler that prints 'skipping..' can be rolled  
in too.  At least cmdutil.add() and cmdutil.cat() catch different things,  
and I have a vague recollection that there was also some uncaught  
exception in this area.  (I don't recall what it was, maybe the subrepo  
being completely missing?)

___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2] cat: record the current behavior of wildcard matches in subrepos

2017-11-26 Thread Matt Harbison 

On Sun, 26 Nov 2017 05:26:24 -0500, Yuya Nishihara  wrote:


On Sat, 25 Nov 2017 23:37:16 -0500, Matt Harbison wrote:
On Sat, 25 Nov 2017 02:35:03 -0500, Yuya Nishihara   
wrote:


> # HG changeset patch
> # User Yuya Nishihara 
> # Date 1511591374 -32400
> #  Sat Nov 25 15:29:34 2017 +0900
> # Node ID 38e952030f0c4746257280d47f0f94b6cb9ddb41
> # Parent  8287df8b7be545fdafa22b771012ac65f6264d12
> cat: record the current behavior of wildcard matches in subrepos
>
> It appears that Mercurial subrepo supports any match patterns.
>
> diff --git a/mercurial/help/subrepos.txt b/mercurial/help/subrepos.txt
> --- a/mercurial/help/subrepos.txt
> +++ b/mercurial/help/subrepos.txt
> @@ -90,7 +90,7 @@ Interaction with Mercurial Commands
>  :archive: archive does not recurse in subrepositories unless
>  -S/--subrepos is specified.
> -:cat: cat currently only handles exact file matches in subrepos.
> +:cat: cat currently only handles exact file matches in Git subrepos.

Should this be "cat currently only handles pattern matches in Mercurial
subrepos"?  The test above this new inexact match test is for an exact  
hg

match.  Git bails early on m.anypats().


Hmm, how about this?

  Git subrepositories only support exact file matches. Subversion
  subrepositories are currently ignored.

What I wanted to say is Git support is limited.


That sounds good to me, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 5 of 5] largefiles: explicitly set the source and sink types to 'hg' for lfconvert

2017-11-26 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1511418070 18000
#  Thu Nov 23 01:21:10 2017 -0500
# Node ID e251a82c7a7601ce3d7e4fcccb2b44cb7d599db5
# Parent  1dfe0c6abea0374834bbb100b5402361e9d7954a
largefiles: explicitly set the source and sink types to 'hg' for lfconvert

I stumbled into this prior to adding the type indicator on the source and sink,
but there's no reason to try to infer the types for this conversion.

diff --git a/hgext/largefiles/lfcommands.py b/hgext/largefiles/lfcommands.py
--- a/hgext/largefiles/lfcommands.py
+++ b/hgext/largefiles/lfcommands.py
@@ -177,7 +177,7 @@
 convcmd.converter = converter
 
 try:
-convcmd.convert(ui, src, dest)
+convcmd.convert(ui, src, dest, source_type='hg', 
dest_type='hg')
 finally:
 convcmd.converter = orig
 success = True
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 4 of 5] lfs: add a repo requirement for this extension when converting to lfs

2017-11-26 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1511408330 18000
#  Wed Nov 22 22:38:50 2017 -0500
# Node ID 1dfe0c6abea0374834bbb100b5402361e9d7954a
# Parent  652a95c77a386d6cd99480a7063d791b000229b3
lfs: add a repo requirement for this extension when converting to lfs

This covers both the vanilla repo -> lfs repo and largefiles -> lfs conversions.
The largefiles extension adds the requirement directly, because it has a
dedicated command to convert.  Using the convert extension is better, because it
supports more features.

I'd like ideas about how to ensure that converting away from lfs works on all
files.  (See comments in test-lfs.t)

diff --git a/hgext/lfs/__init__.py b/hgext/lfs/__init__.py
--- a/hgext/lfs/__init__.py
+++ b/hgext/lfs/__init__.py
@@ -128,6 +128,9 @@
 wrapfilelog(filelog.filelog)
 
 wrapfunction = extensions.wrapfunction
+
+wrapfunction(scmutil, 'wrapconvertsink', wrapper.convertsink)
+
 wrapfunction(changegroup,
  'supportedoutgoingversions',
  wrapper.supportedoutgoingversions)
diff --git a/hgext/lfs/wrapper.py b/hgext/lfs/wrapper.py
--- a/hgext/lfs/wrapper.py
+++ b/hgext/lfs/wrapper.py
@@ -165,6 +165,31 @@
 def filectxislfs(self):
 return _islfs(self.filelog(), self.filenode())
 
+def convertsink(orig, sink):
+sink = orig(sink)
+if sink.repotype == 'hg':
+class lfssink(sink.__class__):
+def putcommit(self, files, copies, parents, commit, source, revmap,
+  full, cleanp2):
+pc = super(lfssink, self).putcommit
+node = pc(files, copies, parents, commit, source, revmap, full,
+  cleanp2)
+
+if 'lfs' not in self.repo.requirements:
+ctx = self.repo[node]
+
+# The file list may contain removed files, so check for
+# membership before assuming it is in the context.
+if any(f in ctx and ctx[f].islfs() for f, n in files):
+self.repo.requirements.add('lfs')
+self.repo._writerequirements()
+
+return node
+
+sink.__class__ = lfssink
+
+return sink
+
 def vfsinit(orig, self, othervfs):
 orig(self, othervfs)
 # copy lfs related options
diff --git a/tests/test-lfs-largefiles.t b/tests/test-lfs-largefiles.t
--- a/tests/test-lfs-largefiles.t
+++ b/tests/test-lfs-largefiles.t
@@ -290,14 +290,16 @@
   0 remove large_by_size.bin
   $ cd nolargefiles
 
-BUG: This should have a requires line for 'lfs'
+The requirement is added to the destination repo
 
   $ cat .hg/requires
   dotencode
   fncache
   generaldelta
+  lfs
   revlogv1
   store
+
   $ hg log -r 'all()' -G -T '{rev} {join(lfs_files, ", ")} ({desc})\n'
   o  8  (remove large_by_size.bin)
   |
diff --git a/tests/test-lfs.t b/tests/test-lfs.t
--- a/tests/test-lfs.t
+++ b/tests/test-lfs.t
@@ -557,3 +557,82 @@
   repo: repo8
   repo: repo9
   repo: repo10
+
+lfs -> normal -> lfs round trip conversions are possible.  The threshold for 
the
+lfs destination is specified here because it was originally listed in the local
+.hgrc, and the global one is too high to trigger lfs usage.  For lfs -> normal,
+there's no 'lfs' destination repo requirement.  For normal -> lfs, there is.
+
+XXX: There's not a great way to ensure that the conversion to normal files
+actually converts _everything_ to normal.  The extension needs to be loaded for
+the source, but there's no way to disable it for the destination.  The best 
that
+can be done is to raise the threshold so that lfs isn't used on the 
destination.
+It doesn't like using '!' to unset the value on the command line.
+
+  $ hg --config extensions.convert= --config lfs.threshold=1000M \
+  >convert repo8 convert_normal
+  initializing destination convert_normal repository
+  scanning source...
+  sorting...
+  converting...
+  2 a
+  1 b
+  0 meta
+  $ grep 'lfs' convert_normal/.hg/requires
+  [1]
+  $ hg --cwd convert_normal debugdata a1 0
+  THIS-IS-LFS-BECAUSE-10-BYTES
+
+  $ hg --config extensions.convert= --config lfs.threshold=10B \
+  >convert convert_normal convert_lfs
+  initializing destination convert_lfs repository
+  scanning source...
+  sorting...
+  converting...
+  2 a
+  1 b
+  0 meta
+  $ hg --cwd convert_lfs debugdata a1 0
+  version https://git-lfs.github.com/spec/v1
+  oid sha256:5bb8341bee63b3649f222b2215bde37322bea075a30575aa685d8f8d21c77024
+  size 29
+  x-is-binary 0
+  $ grep 'lfs' convert_lfs/.hg/requires
+  lfs
+
+This convert is trickier, because it contains deleted files (via `hg mv`)
+
+  $ hg --config extensions.convert= --config lfs.threshold=1000M \
+  >convert repo3 convert_normal2
+  initializing destination convert_normal2 repository
+  scanning source...
+  sorting...
+  converting...
+  4 commit with lfs content
+  3 renames
+  2 large to small, small to large
+  1 random modifications
+  0

[PATCH 2 of 5] convert: save an indicator of the repo type for sources and sinks

2017-11-26 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1511401741 18000
#  Wed Nov 22 20:49:01 2017 -0500
# Node ID 2e7c87e0755eccfc367d12d28c1597bc18e70c27
# Parent  ccc7eae771760a638f9809caa8c3eb4747f94f01
convert: save an indicator of the repo type for sources and sinks

This seems like basic info to have, and will be used shortly when deciding
whether or not to wrap the class for lfs conversions.

The other option is to just add a function to each class.  But this seems better
in that the strings aren't duplicated, and the constructor for most of these
will run even if the VCS isn't installed, so it's easier to catch errors.

diff --git a/hgext/convert/bzr.py b/hgext/convert/bzr.py
--- a/hgext/convert/bzr.py
+++ b/hgext/convert/bzr.py
@@ -44,8 +44,8 @@
 class bzr_source(common.converter_source):
 """Reads Bazaar repositories by using the Bazaar Python libraries"""
 
-def __init__(self, ui, path, revs=None):
-super(bzr_source, self).__init__(ui, path, revs=revs)
+def __init__(self, ui, repotype, path, revs=None):
+super(bzr_source, self).__init__(ui, repotype, path, revs=revs)
 
 if not os.path.exists(os.path.join(path, '.bzr')):
 raise common.NoRepo(_('%s does not look like a Bazaar repository')
diff --git a/hgext/convert/common.py b/hgext/convert/common.py
--- a/hgext/convert/common.py
+++ b/hgext/convert/common.py
@@ -73,12 +73,13 @@
 class converter_source(object):
 """Conversion source interface"""
 
-def __init__(self, ui, path=None, revs=None):
+def __init__(self, ui, repotype, path=None, revs=None):
 """Initialize conversion source (or raise NoRepo("message")
 exception if path is not a valid repository)"""
 self.ui = ui
 self.path = path
 self.revs = revs
+self.repotype = repotype
 
 self.encoding = 'utf-8'
 
@@ -218,7 +219,7 @@
 class converter_sink(object):
 """Conversion sink (target) interface"""
 
-def __init__(self, ui, path):
+def __init__(self, ui, repotype, path):
 """Initialize conversion sink (or raise NoRepo("message")
 exception if path is not a valid repository)
 
@@ -227,6 +228,7 @@
 self.ui = ui
 self.path = path
 self.created = []
+self.repotype = repotype
 
 def revmapfile(self):
 """Path to a file that will contain lines
diff --git a/hgext/convert/convcmd.py b/hgext/convert/convcmd.py
--- a/hgext/convert/convcmd.py
+++ b/hgext/convert/convcmd.py
@@ -114,7 +114,7 @@
 for name, source, sortmode in source_converters:
 try:
 if not type or name == type:
-return source(ui, path, revs), sortmode
+return source(ui, name, path, revs), sortmode
 except (NoRepo, MissingTool) as inst:
 exceptions.append(inst)
 if not ui.quiet:
@@ -128,7 +128,7 @@
 for name, sink in sink_converters:
 try:
 if not type or name == type:
-return sink(ui, path)
+return sink(ui, name, path)
 except NoRepo as inst:
 ui.note(_("convert: %s\n") % inst)
 except MissingTool as inst:
diff --git a/hgext/convert/cvs.py b/hgext/convert/cvs.py
--- a/hgext/convert/cvs.py
+++ b/hgext/convert/cvs.py
@@ -32,8 +32,8 @@
 NoRepo = common.NoRepo
 
 class convert_cvs(converter_source):
-def __init__(self, ui, path, revs=None):
-super(convert_cvs, self).__init__(ui, path, revs=revs)
+def __init__(self, ui, repotype, path, revs=None):
+super(convert_cvs, self).__init__(ui, repotype, path, revs=revs)
 
 cvs = os.path.join(path, "CVS")
 if not os.path.exists(cvs):
diff --git a/hgext/convert/darcs.py b/hgext/convert/darcs.py
--- a/hgext/convert/darcs.py
+++ b/hgext/convert/darcs.py
@@ -40,8 +40,8 @@
 pass
 
 class darcs_source(common.converter_source, common.commandline):
-def __init__(self, ui, path, revs=None):
-common.converter_source.__init__(self, ui, path, revs=revs)
+def __init__(self, ui, repotype, path, revs=None):
+common.converter_source.__init__(self, ui, repotype, path, revs=revs)
 common.commandline.__init__(self, ui, 'darcs')
 
 # check for _darcs, ElementTree so that we can easily skip
diff --git a/hgext/convert/filemap.py b/hgext/convert/filemap.py
--- a/hgext/convert/filemap.py
+++ b/hgext/convert/filemap.py
@@ -172,7 +172,7 @@
 
 class filemap_source(common.converter_source):
 def __init__(self, ui, baseconverter, filemap):
-super(filemap_source, self).__init__(ui)
+super(filemap_source, self).__init__(ui, baseconverter.repotype)
 self.base = baseconverter
 self.filemapper = filemapper(ui, filemap)
 self.commits = {}
diff --git a/hgext/convert/git.py b/hgext/convert/git.py
--- a/hgext/convert/git.py
+++ b/hgext/convert/git.py
@@ -66,8 +66,8 @@
 def gitpipe(self, *args, **kwargs):
 return

[PATCH 3 of 5] convert: allow the sink object to be wrapped when the extension isn't loaded

2017-11-26 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1511726379 18000
#  Sun Nov 26 14:59:39 2017 -0500
# Node ID 652a95c77a386d6cd99480a7063d791b000229b3
# Parent  2e7c87e0755eccfc367d12d28c1597bc18e70c27
convert: allow the sink object to be wrapped when the extension isn't loaded

The next patch will wrap the conversion code, in order to write out a
requirement for 'lfs' when appropriate.  Wrapping convcmd.convertsink() in an
afterloaded callback works fine when the convert extension is enabled by the
user.  The problem here is that lfconvert uses the convert extension, whether or
not it was formally enabled by the user.

My first attempt was to have lfs install an afterloaded callback that would wrap
the convert sink if convert was loaded, or wrap lfconvert if it wasn't.  Then
the lfconvert override could install an afterloaded callback to try wrapping the
convert sink again, before calling the original lfconvert.  But that breaks down
if largefiles can't load the convert extension on the fly. [1]  Further, some
tests were failing with an error indicating that the size of the afterloaded
list changed while iterating it.

Yuya mentioned that maybe some bits of convert could be moved into core, but I'm
not sure where to draw that line.  The convertsink() method depends on the list
of sinks, which in turn depends on the sink classes.

[1] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-November/108038.html

diff --git a/hgext/convert/convcmd.py b/hgext/convert/convcmd.py
--- a/hgext/convert/convcmd.py
+++ b/hgext/convert/convcmd.py
@@ -15,6 +15,7 @@
 encoding,
 error,
 hg,
+scmutil,
 util,
 )
 
@@ -575,6 +576,7 @@
 ui.status(_("assuming destination %s\n") % dest)
 
 destc = convertsink(ui, dest, opts.get('dest_type'))
+destc = scmutil.wrapconvertsink(destc)
 
 try:
 srcc, defaultsort = convertsource(ui, src, opts.get('source_type'),
diff --git a/mercurial/scmutil.py b/mercurial/scmutil.py
--- a/mercurial/scmutil.py
+++ b/mercurial/scmutil.py
@@ -1271,3 +1271,9 @@
 else:
 revrange = '%s:%s' % (minrev, maxrev)
 repo.ui.status(_('new changesets %s\n') % revrange)
+
+def wrapconvertsink(sink):
+"""Allow extensions to wrap the sink returned by convcmd.convertsink()
+before it is used, whether or not the convert extension was formally 
loaded.
+"""
+return sink
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 1 of 5] lfs: add a repo requirement for this extension once an lfs file is committed

2017-11-26 Thread Matt Harbison 
# HG changeset patch
# User Matt Harbison 
# Date 1510807395 18000
#  Wed Nov 15 23:43:15 2017 -0500
# Node ID ccc7eae771760a638f9809caa8c3eb4747f94f01
# Parent  32bb27dd52825236ba1b6c06fe60e140d6b5ea45
lfs: add a repo requirement for this extension once an lfs file is committed

Largefiles does the same thing (also delayed until the first largefile commit),
to prevent access to the repo without the extension.  In the case of this
extension, not having the extension loaded while accessing an lfs file results
in cryptic errors about "missing processor for flag '0x2000'".  If enabled
locally but not remotely, the cryptic error message is about no common
changegroup version.  (It wants '03', which is currently experimental.)

The largefiles extension looks for any tracked file that starts with '.hglf/'.
Unfortunately, that doesn't work here.  I didn't see any way to get the files
that were just committed, without doing a full status.  But since there's no
secondary check on adding an lfs file once the extension is loaded and a
threshold set, the best practice is to only enable this locally on a repo that
needs it.  That should minimize the unnecessary overhead for repos without an
lfs file.

diff --git a/hgext/lfs/__init__.py b/hgext/lfs/__init__.py
--- a/hgext/lfs/__init__.py
+++ b/hgext/lfs/__init__.py
@@ -37,6 +37,7 @@
 exchange,
 extensions,
 filelog,
+localrepo,
 registrar,
 revlog,
 scmutil,
@@ -84,6 +85,13 @@
 
 templatekeyword = registrar.templatekeyword()
 
+def featuresetup(ui, supported):
+# don't die on seeing a repo with the lfs requirement
+supported |= {'lfs'}
+
+def uisetup(ui):
+localrepo.localrepository.featuresetupfuncs.add(featuresetup)
+
 def reposetup(ui, repo):
 # Nothing to do with a remote repo
 if not repo.local():
@@ -98,6 +106,17 @@
 # Push hook
 repo.prepushoutgoinghooks.add('lfs', wrapper.prepush)
 
+if 'lfs' not in repo.requirements:
+def checkrequireslfs(ui, repo, **kwargs):
+if 'lfs' not in repo.requirements:
+ctx = repo[kwargs['node']]
+# TODO: is there a way to just walk the files in the commit?
+if any(ctx[f].islfs() for f in ctx.files()):
+repo.requirements.add('lfs')
+repo._writerequirements()
+
+ui.setconfig('hooks', 'commit.lfs', checkrequireslfs, 'lfs')
+
 def wrapfilelog(filelog):
 wrapfunction = extensions.wrapfunction
 
diff --git a/tests/test-lfs.t b/tests/test-lfs.t
--- a/tests/test-lfs.t
+++ b/tests/test-lfs.t
@@ -20,7 +20,11 @@
 
 # Commit large file
   $ echo $LONG > largefile
+  $ grep lfs .hg/requires
+  [1]
   $ hg commit --traceback -Aqm "add large file"
+  $ grep lfs .hg/requires
+  lfs
 
 # Ensure metadata is stored
   $ hg debugdata largefile 0
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[Bug 5748] New: Unknown exception during commit

2017-11-26 Thread mercurial-bugs 
https://bz.mercurial-scm.org/show_bug.cgi?id=5748

Bug ID: 5748
   Summary: Unknown exception during commit
   Product: Mercurial
   Version: 4.4.1
  Hardware: PC
OS: Mac OS
Status: UNCONFIRMED
  Severity: bug
  Priority: wish
 Component: Mercurial
  Assignee: bugzi...@mercurial-scm.org
  Reporter: v...@uber.com
CC: mercurial-devel@mercurial-scm.org

Hi, I'm getting following exception while trying to commit changes on fairly
large repo. (simply running: hg commit -m "Commit foo")
Not sure what or why this is happening, would be great to get some help.

** unknown exception encountered, please report by visiting
** https://mercurial-scm.org/wiki/BugTracker
** Python 2.7.10 (default, Jul 15 2017, 17:16:57) [GCC 4.2.1 Compatible Apple
LLVM 9.0.0 (clang-900.0.31)]
** Mercurial Distributed SCM (version 4.4.1)
** Extensions loaded: convert, purge
Traceback (most recent call last):
  File "/usr/local/bin/hg", line 41, in 
dispatch.run()
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 85, in
run
status = (dispatch(req) or 0) & 255
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 173, in
dispatch
ret = _runcatch(req)
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 313, in
_runcatch
return _callcatch(ui, _runcatchfunc)
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 321, in
_callcatch
return scmutil.callcatch(ui, func)
  File "/Library/Python/2.7/site-packages/mercurial/scmutil.py", line 154, in
callcatch
return func()
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 303, in
_runcatchfunc
return _dispatch(req)
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 940, in
_dispatch
cmdpats, cmdoptions)
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 701, in
runcommand
ret = _runcommand(ui, options, cmd, d)
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 948, in
_runcommand
return cmdfunc()
  File "/Library/Python/2.7/site-packages/mercurial/dispatch.py", line 937, in

d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
  File "/Library/Python/2.7/site-packages/mercurial/util.py", line 1183, in
check
return func(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/mercurial/commands.py", line 1501, in
commit
return _docommit(ui, repo, *pats, **opts)
  File "/Library/Python/2.7/site-packages/mercurial/commands.py", line 1584, in
_docommit
node = cmdutil.commit(ui, repo, commitfunc, pats, opts)
  File "/Library/Python/2.7/site-packages/mercurial/cmdutil.py", line 3090, in
commit
return commitfunc(ui, repo, message, matcher, opts)
  File "/Library/Python/2.7/site-packages/mercurial/commands.py", line 1582, in
commitfunc
extra=extra)
  File "/Library/Python/2.7/site-packages/mercurial/localrepo.py", line 142, in
wrapper
return orig(repo.unfiltered(), *args, **kwargs)
  File "/Library/Python/2.7/site-packages/mercurial/localrepo.py", line 1946,
in commit
ret = self.commitctx(cctx, True)
  File "/Library/Python/2.7/site-packages/mercurial/localrepo.py", line 142, in
wrapper
return orig(repo.unfiltered(), *args, **kwargs)
  File "/Library/Python/2.7/site-packages/mercurial/localrepo.py", line 2013,
in commitctx
trp, changed)
  File "/Library/Python/2.7/site-packages/mercurial/localrepo.py", line 1778,
in _filecommit
if fparent2 != nullid or flog.cmp(fparent1, text) or meta:
  File "/Library/Python/2.7/site-packages/mercurial/filelog.py", line 101, in
cmp
if self.renamed(node):
  File "/Library/Python/2.7/site-packages/mercurial/filelog.py", line 65, in
renamed
return (m["copy"], revlog.bin(m["copyrev"]))
KeyError: 'copyrev'

-- 
You are receiving this mail because:
You are on the CC list for the bug.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1506: dagop: handle IndexError when using wdir() in dag range

2017-11-26 Thread yuja (Yuya Nishihara) 
yuja added a comment.


  In https://phab.mercurial-scm.org/D1506#25509, @ryanmce wrote:
  
  > but ideally, wouldn't this be supported?
  
  
  Yeah, this case, `includepath=True`, won't be difficult to handle. If 
reachableroots() raised
  WdirUnsupportedError, drop wdir revision from `y` of `x::y`, add parents 
instead, rerun
  `x::(y - wdir() + parents(wdir()))`, and append wdir to the result.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1506

To: swhitaker, #hg-reviewers, mbthomas, yuja
Cc: ryanmce, yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] mergetools: add nbdime merge tools to default rc

2017-11-26 Thread Yuya Nishihara 
On Sat, 25 Nov 2017 13:34:58 +0100, Vidar Tonaas Fauske wrote:
> exporting patch:
> # HG changeset patch
> # User Vidar Tonaas Fauske
> # Date 1510750467 -3600
> #  Wed Nov 15 13:54:27 2017 +0100
> # Node ID 7072463ab05996c7ae12ea4a6ccb9da305c54a53
> # Parent  75013952d8d9608f73cd45f68405fbd6ec112bf2
> mergetools: add nbdime merge tools to default rc
> 
> Adds default configuration for nbdime merge tools for the Jupyter notebook
> format (a structured JSON document). If the user has nbdime installed, this
> change will cause mercurial to use it by default (but with a low priority).
> 
> Why is it a good idea to use a different tool for Notebooks? Because the 
> default
> merge tool can produce invalid documents (non-valid JSON), meaning the user
> has to edit the JSON by hand (which is not the intenion). The nbdime merge 
> tool
> ensures that the output is always a valid notebook, even when conflicted.
> 
> diff -r 75013952d8d9 -r 7072463ab059 mercurial/default.d/mergetools.rc
> --- a/mercurial/default.d/mergetools.rc Fri Nov 10 19:14:06 2017 +0800
> +++ b/mercurial/default.d/mergetools.rc Wed Nov 15 13:54:27 2017 +0100
> @@ -144,3 +144,24 @@
>  UltraCompare.binary = True
>  UltraCompare.check = conflicts,changed
>  UltraCompare.diffargs=$child $parent -title1 $clabel -title2 $plabel1
> +
> +# Nbdime for merging Jupyter notebooks:
> +#   merge driver:

Nit: I'm not sure what this "merge driver" and "merge tool" mean, and the
term "merge driver" seems to have a different meaning in Mercurial. Can you
elaborate?

> +nbdime.priority = 2
> +nbdime.premerge = False
> +nbdime.executable = hg-nbmerge
> +nbdime.args = $base $local $other $output
> +#   merge tool:
> +nbdimeweb.priority = 1
> +nbdimeweb.premerge = False
> +nbdimeweb.executable = hg-nbmergeweb
> +nbdimeweb.args = --log-level ERROR $base $local $other $output
> +nbdimeweb.gui = True

These priorities seem too high compared to the other tools. I think these
tools should have quite low priority so they won't be selected as a general-
purpose merge tool.

> +# Default association of file types with merge tools
> +
> +[merge-patterns]
> +
> +**.ipynb = nbdime

Registering merge-tools by default is good, but I don't think we should force
users to select them by default. Can you send new patch that only registers
merge-tools?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] hgweb: display fate of obsolete changesets

2017-11-26 Thread Yuya Nishihara 
On Tue, 21 Nov 2017 18:14:42 +0800, Anton Shestakov wrote:
> # HG changeset patch
> # User Anton Shestakov 
> # Date 1511255021 -28800
> #  Tue Nov 21 17:03:41 2017 +0800
> # Node ID f8e398c42760482f02e6cfb6b8a1e6a752564c95
> # Parent  ff80efc8f3e469c90376603af4fa39012f328918
> # EXP-Topic hgweb-more-info
> hgweb: display fate of obsolete changesets

> diff --git a/mercurial/hgweb/webutil.py b/mercurial/hgweb/webutil.py
> --- a/mercurial/hgweb/webutil.py
> +++ b/mercurial/hgweb/webutil.py
> @@ -28,10 +28,12 @@ from .. import (
>  error,
>  match,
>  mdiff,
> +obsutil,
>  patch,
>  pathutil,
>  pycompat,
>  templatefilters,
> +templatekw,
>  ui as uimod,
>  util,
>  )
> @@ -351,6 +353,10 @@ def linerange(req):
>  def formatlinerange(fromline, toline):
>  return '%d:%d' % (fromline + 1, toline)
>  
> +def obsfate(repo, ctx, ui):
> +return [obsutil.obsfateprinter(x['successors'], x['markers'], ui)
> +for x in templatekw.showsuccsandmarkers(repo, ctx)]

Maybe we should first refactor obsfate utility to not depend on templatekw.
Boris, any progress on this?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2] obsolete: drop usage of changectx in '_computephasedivergentset'

2017-11-26 Thread Yuya Nishihara 
On Fri, 24 Nov 2017 16:34:11 -0500, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld 
> # Date 1511065392 -3600
> #  Sun Nov 19 05:23:12 2017 +0100
> # Node ID b76ac906d1ecea6bcac489834699d01273112fc8
> # Parent  75013952d8d9608f73cd45f68405fbd6ec112bf2
> # EXP-Topic instability-speed
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #  hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 
> b76ac906d1ec
> obsolete: drop usage of changectx in '_computephasedivergentset'

Looks good. Queued, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 3 of 3] test-lfs: allow the test server to be killed on Windows

2017-11-26 Thread Yuya Nishihara 
On Tue, 21 Nov 2017 01:31:14 -0500, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison 
> # Date 1511241849 18000
> #  Tue Nov 21 00:24:09 2017 -0500
> # Node ID cdc5b2fc8d54771402773287a241924878a3aebe
> # Parent  8e1c165cf1eef41c1db8359ed07424629c932c42
> test-lfs: allow the test server to be killed on Windows

Queued, thanks.

Appears that there isn't no easy way to make lfs-test-server dump a pid file.
https://github.com/git-lfs/lfs-test-server/blob/master/main.go
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 4] paper: remove css hack that made .branchname look like .branchhead

2017-11-26 Thread Yuya Nishihara 
On Sat, 25 Nov 2017 16:50:38 +0800, Anton Shestakov wrote:
> # HG changeset patch
> # User Anton Shestakov 
> # Date 1511593307 -28800
> #  Sat Nov 25 15:01:47 2017 +0800
> # Node ID b49cc79b7aab3a0ecae1eb8ce9cef87b17b296de
> # Parent  e17143d2181bb77ef8346b7dfe32b894bbf9a064
> # EXP-Topic hgweb-cleanup
> paper: remove css hack that made .branchname look like .branchhead

Queued the series, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] hgweb: show changeset age in more places (gitweb and monoblue)

2017-11-26 Thread Yuya Nishihara 
On Tue, 21 Nov 2017 22:44:30 +0800, Anton Shestakov wrote:
> # HG changeset patch
> # User Anton Shestakov 
> # Date 1511267337 -28800
> #  Tue Nov 21 20:28:57 2017 +0800
> # Node ID e17143d2181bb77ef8346b7dfe32b894bbf9a064
> # Parent  f8e398c42760482f02e6cfb6b8a1e6a752564c95
> # EXP-Topic hgweb-more-info
> hgweb: show changeset age in more places (gitweb and monoblue)

Queued, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH] hgweb: rename the main attribute of instabilities

2017-11-26 Thread Yuya Nishihara 
On Sun, 26 Nov 2017 14:23:25 +0800, Anton Shestakov wrote:
> # HG changeset patch
> # User Anton Shestakov 
> # Date 1511674158 -28800
> #  Sun Nov 26 13:29:18 2017 +0800
> # Node ID f38c91c7429458763f85c0ab6dd87727636417d9
> # Parent  8287df8b7be545fdafa22b771012ac65f6264d12
> hgweb: rename the main attribute of instabilities

Queued, thanks.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] cat: do not instantiate subrepo if no potential match in it

2017-11-26 Thread Yuya Nishihara 
On Sat, 25 Nov 2017 23:53:45 -0500, Matt Harbison wrote:
> On Sat, 25 Nov 2017 02:35:04 -0500, Yuya Nishihara  wrote:
> 
> > # HG changeset patch
> > # User Yuya Nishihara 
> > # Date 1511593310 -32400
> > #  Sat Nov 25 16:01:50 2017 +0900
> > # Node ID b6e526ee5d2662ca30a555ca51264b3e371fe44e
> > # Parent  38e952030f0c4746257280d47f0f94b6cb9ddb41
> > cat: do not instantiate subrepo if no potential match in it
> >
> > This fixes the test failure in hg-git.
> >
> > https://bitbucket.org/durin42/hg-git/issues/227/
> >
> > diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
> > --- a/mercurial/cmdutil.py
> > +++ b/mercurial/cmdutil.py
> > @@ -3068,6 +3068,8 @@ def cat(ui, repo, ctx, matcher, basefm,
> >  err = 0
> > for subpath in sorted(ctx.substate):
> > +if not matcher.visitdir(subpath):
> > +continue
> 
> The basematcher contract says the result is undefined if False is returned  
> for any parent directory.  Do we need a subrepo utility method to break up  
> the subpath and test top down?  Consider 'directory/subrepo', where  
> 'directory' is excluded.

(+CC Martin)

Yeah the doc says that, but not all callers seem to obey the restriction. So
I take it visitdir() won't return Falsy value if there is a possible match,
even though it might return non-False for Falsy case if any parents return
False.

> It's probably beyond the scope of what you were fixing, but should all  
> subrepo recursion be guarded like this?
> 
> >  sub = ctx.sub(subpath)
> >  try:
> >  submatch = matchmod.subdirmatcher(subpath, matcher)

Maybe we'll need ctx.walksub(matcher) which yields (subrepo, submatcher) pairs?
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 1 of 2] cat: record the current behavior of wildcard matches in subrepos

2017-11-26 Thread Yuya Nishihara 
On Sat, 25 Nov 2017 23:37:16 -0500, Matt Harbison wrote:
> On Sat, 25 Nov 2017 02:35:03 -0500, Yuya Nishihara  wrote:
> 
> > # HG changeset patch
> > # User Yuya Nishihara 
> > # Date 1511591374 -32400
> > #  Sat Nov 25 15:29:34 2017 +0900
> > # Node ID 38e952030f0c4746257280d47f0f94b6cb9ddb41
> > # Parent  8287df8b7be545fdafa22b771012ac65f6264d12
> > cat: record the current behavior of wildcard matches in subrepos
> >
> > It appears that Mercurial subrepo supports any match patterns.
> >
> > diff --git a/mercurial/help/subrepos.txt b/mercurial/help/subrepos.txt
> > --- a/mercurial/help/subrepos.txt
> > +++ b/mercurial/help/subrepos.txt
> > @@ -90,7 +90,7 @@ Interaction with Mercurial Commands
> >  :archive: archive does not recurse in subrepositories unless
> >  -S/--subrepos is specified.
> > -:cat: cat currently only handles exact file matches in subrepos.
> > +:cat: cat currently only handles exact file matches in Git subrepos.
> 
> Should this be "cat currently only handles pattern matches in Mercurial  
> subrepos"?  The test above this new inexact match test is for an exact hg  
> match.  Git bails early on m.anypats().

Hmm, how about this?

  Git subrepositories only support exact file matches. Subversion
  subrepositories are currently ignored.

What I wanted to say is Git support is limited.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH STABLE] convert: avoid wrong lfconvert defaults by moving configitems to core

2017-11-26 Thread Yuya Nishihara 
On Sat, 25 Nov 2017 22:51:12 -0500, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison 
> # Date 1511667169 18000
> #  Sat Nov 25 22:32:49 2017 -0500
> # Branch stable
> # Node ID a2a5049f83d484eafcef2372a57ce02737a398d8
> # Parent  02845f7441aff30bc01975a5881cabfa922c12d4
> convert: avoid wrong lfconvert defaults by moving configitems to core

> +coreconfigitem('convert', 'p4.encoding',
> +default=lambda: convcmd.orig_encoding,
> +)

Missing attribute here. Perhaps this has to be a dynamicdefault and resolved
in convert/p4.py.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

D1506: dagop: handle IndexError when using wdir() in dag range

2017-11-26 Thread ryanmce (Ryan McElroy) 
ryanmce added a comment.


  I understand that fixing the crash is a good first step, but ideally, 
wouldn't this be supported? Is there a reason it's particularly hard to support?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1506

To: swhitaker, #hg-reviewers, mbthomas, yuja
Cc: ryanmce, yuja, mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel