Re: [PATCH 2 of 7 upgraderepo V2] debugcommands: stub for debugupgraderepo command

2016-12-13 Thread Pierre-Yves David



On 12/13/2016 06:18 AM, Gregory Szorc wrote:

On Mon, Dec 12, 2016 at 2:17 AM, Pierre-Yves David
mailto:pierre-yves.da...@ens-lyon.org>>
wrote:



On 11/25/2016 04:28 AM, Gregory Szorc wrote:

# HG changeset patch
# User Gregory Szorc mailto:gregory.sz...@gmail.com>>
# Date 1480033449 28800
#  Thu Nov 24 16:24:09 2016 -0800
# Node ID 4fffbdc2bca22e666482b55562c0e592c8a68027
# Parent  c60995fce14b0e34bd1416dab3712a6c33bb29bb
debugcommands: stub for debugupgraderepo command

Currently, if Mercurial introduces a new repository/store feature or
changes behavior of an existing feature, users must perform an
`hg clone` to create a new repository with hopefully the
correct/optimal settings. Unfortunately, even `hg clone` may not
give the correct results. For example, if you do a local `hg clone`,
you may get hardlinks to revlog files that inherit the old state.
If you `hg clone` from a remote or `hg clone --pull`, changegroup
application may bypass some optimization, such as converting to
generaldelta.

Optimizing a repository is harder than it seems and requires more
than a simple `hg` command invocation.

This commit starts the process of changing that. We introduce
`hg debugupgraderepo`, a command that performs an in-place upgrade
of a repository to use new, optimal features. The command is just
a stub right now. Features will be added in subsequent commits.


I like the idea of this commit, this should probably be the first
one in the stack. The revlog cloning feature can be delayed later in
the stack until we actually start performing upgrade.

This commit does foreshadow some of the behavior of the new command,
notably that it doesn't do anything by default and that it takes
arguments that influence what actions it performs. These will be
explained more in subsequent commits.

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -849,3 +849,28 @@ def debugdeltachain(ui, repo, file_=None
  extradist=extradist, extraratio=extraratio)

 fm.end()
+
+@command('debugupgraderepo', [
+('o', 'optimize', [], _('extra optimization to perform'),
_('NAME')),
+('', 'run', False, _('performs an upgrade')),


We should drop these option from the command until we actually use
them this will allow to discuss them with their actual behavior.

+])
+def debugupgraderepo(ui, repo, run=False, optimize=None):
+"""upgrade a repository to use different features
+
+If no arguments are specified, the repository is evaluated
for upgrade
+and a list of problems and potential optimizations is printed.
+
+With ``--run``, a repository upgrade is performed. Behavior
of the upgrade
+can be influenced via additional arguments. More details
will be provided
+by the command output when run without ``--run``.


(I would also delayed the help about --run until we actually run
something)

+
+During the upgrade, the repository will be locked and no
writes will be
+allowed.
+
+At the end of the upgrade, the repository may not be
readable while new
+repository data is swapped in. This window will be as long
as it takes to
+rename some directories inside the ``.hg`` directory. On
most machines, this
+should complete almost instantaneously and the chances of a
consumer being
+unable to access the repository should be low.


(I would also delay that doc until we actually behave that way)
I know it would bot solve "everything" but we discussed the option
of having a "repositorybeingupgraded" (or similar) in .hg/required
to reduce the change for a reader to access the repository at that time.


I added all this code and documentation to foreshadow the feature so a
reviewer isn't guessing where things are going. It also cuts down on the
amount of patches and/or churn in future patches, making code review
easier, IMO. I think it is busy work to change it at this stage. But if
you insist, I will.


This definitely does not help me. Sure, this gives some hint of what 
might happen in the future, but this also make use take an early 
decision with very partial information on something we might end up 
doing different.


For example the --optimize flag looks strange to me and I'm not sure it 
will survive later review. So gluing the 'we have a debugupgraderepo 
command' steps with 'we have an --optimize flag' increase the odd that 
the whole t

Re: [PATCH 2 of 7 upgraderepo V2] debugcommands: stub for debugupgraderepo command

2016-12-13 Thread Augie Fackler
On Mon, Dec 12, 2016 at 09:18:51PM -0800, Gregory Szorc wrote:
> On Mon, Dec 12, 2016 at 2:17 AM, Pierre-Yves David <
> pierre-yves.da...@ens-lyon.org> wrote:
>
> >
> >
> > On 11/25/2016 04:28 AM, Gregory Szorc wrote:
> >
> >> # HG changeset patch
> >> # User Gregory Szorc 
> >> # Date 1480033449 28800
> >> #  Thu Nov 24 16:24:09 2016 -0800
> >> # Node ID 4fffbdc2bca22e666482b55562c0e592c8a68027
> >> # Parent  c60995fce14b0e34bd1416dab3712a6c33bb29bb
> >> debugcommands: stub for debugupgraderepo command
> >>
> >> Currently, if Mercurial introduces a new repository/store feature or
> >> changes behavior of an existing feature, users must perform an
> >> `hg clone` to create a new repository with hopefully the
> >> correct/optimal settings. Unfortunately, even `hg clone` may not
> >> give the correct results. For example, if you do a local `hg clone`,
> >> you may get hardlinks to revlog files that inherit the old state.
> >> If you `hg clone` from a remote or `hg clone --pull`, changegroup
> >> application may bypass some optimization, such as converting to
> >> generaldelta.
> >>
> >> Optimizing a repository is harder than it seems and requires more
> >> than a simple `hg` command invocation.
> >>
> >> This commit starts the process of changing that. We introduce
> >> `hg debugupgraderepo`, a command that performs an in-place upgrade
> >> of a repository to use new, optimal features. The command is just
> >> a stub right now. Features will be added in subsequent commits.
> >>
> >
> > I like the idea of this commit, this should probably be the first one in
> > the stack. The revlog cloning feature can be delayed later in the stack
> > until we actually start performing upgrade.
> >
> > This commit does foreshadow some of the behavior of the new command,
> >> notably that it doesn't do anything by default and that it takes
> >> arguments that influence what actions it performs. These will be
> >> explained more in subsequent commits.
> >>
> >> diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
> >> --- a/mercurial/debugcommands.py
> >> +++ b/mercurial/debugcommands.py
> >> @@ -849,3 +849,28 @@ def debugdeltachain(ui, repo, file_=None
> >>   extradist=extradist, extraratio=extraratio)
> >>
> >>  fm.end()
> >> +
> >> +@command('debugupgraderepo', [
> >> +('o', 'optimize', [], _('extra optimization to perform'), _('NAME')),
> >> +('', 'run', False, _('performs an upgrade')),
> >>
> >
> > We should drop these option from the command until we actually use them
> > this will allow to discuss them with their actual behavior.
> >
> > +])
> >> +def debugupgraderepo(ui, repo, run=False, optimize=None):
> >> +"""upgrade a repository to use different features
> >> +
> >> +If no arguments are specified, the repository is evaluated for
> >> upgrade
> >> +and a list of problems and potential optimizations is printed.
> >> +
> >> +With ``--run``, a repository upgrade is performed. Behavior of the
> >> upgrade
> >> +can be influenced via additional arguments. More details will be
> >> provided
> >> +by the command output when run without ``--run``.
> >>
> >
> > (I would also delayed the help about --run until we actually run something)
> >
> > +
> >> +During the upgrade, the repository will be locked and no writes will
> >> be
> >> +allowed.
> >> +
> >> +At the end of the upgrade, the repository may not be readable while
> >> new
> >> +repository data is swapped in. This window will be as long as it
> >> takes to
> >> +rename some directories inside the ``.hg`` directory. On most
> >> machines, this
> >> +should complete almost instantaneously and the chances of a consumer
> >> being
> >> +unable to access the repository should be low.
> >>
> >
> > (I would also delay that doc until we actually behave that way)
> > I know it would bot solve "everything" but we discussed the option of
> > having a "repositorybeingupgraded" (or similar) in .hg/required to reduce
> > the change for a reader to access the repository at that time.
> >
>
> I added all this code and documentation to foreshadow the feature so a
> reviewer isn't guessing where things are going. It also cuts down on the
> amount of patches and/or churn in future patches, making code review
> easier, IMO. I think it is busy work to change it at this stage. But if you
> insist, I will.

I liked having the documentation early in the stack, for what it's worth.

>
>
> >
> > +"""
> >> +raise error.Abort(_('not yet implemented'))
> >> diff --git a/tests/test-completion.t b/tests/test-completion.t
> >> --- a/tests/test-completion.t
> >> +++ b/tests/test-completion.t
> >> @@ -109,6 +109,7 @@ Show debug commands if there are no othe
> >>debugsub
> >>debugsuccessorssets
> >>debugtemplate
> >> +  debugupgraderepo
> >>debugwalk
> >>debugwireargs
> >>
> >> @@ -274,6 +275,7 @@ Show all commands + options
> >>debugsub: rev
> >>debugsuccessorssets:

Re: [PATCH 2 of 7 upgraderepo V2] debugcommands: stub for debugupgraderepo command

2016-12-12 Thread Gregory Szorc
On Mon, Dec 12, 2016 at 2:17 AM, Pierre-Yves David <
pierre-yves.da...@ens-lyon.org> wrote:

>
>
> On 11/25/2016 04:28 AM, Gregory Szorc wrote:
>
>> # HG changeset patch
>> # User Gregory Szorc 
>> # Date 1480033449 28800
>> #  Thu Nov 24 16:24:09 2016 -0800
>> # Node ID 4fffbdc2bca22e666482b55562c0e592c8a68027
>> # Parent  c60995fce14b0e34bd1416dab3712a6c33bb29bb
>> debugcommands: stub for debugupgraderepo command
>>
>> Currently, if Mercurial introduces a new repository/store feature or
>> changes behavior of an existing feature, users must perform an
>> `hg clone` to create a new repository with hopefully the
>> correct/optimal settings. Unfortunately, even `hg clone` may not
>> give the correct results. For example, if you do a local `hg clone`,
>> you may get hardlinks to revlog files that inherit the old state.
>> If you `hg clone` from a remote or `hg clone --pull`, changegroup
>> application may bypass some optimization, such as converting to
>> generaldelta.
>>
>> Optimizing a repository is harder than it seems and requires more
>> than a simple `hg` command invocation.
>>
>> This commit starts the process of changing that. We introduce
>> `hg debugupgraderepo`, a command that performs an in-place upgrade
>> of a repository to use new, optimal features. The command is just
>> a stub right now. Features will be added in subsequent commits.
>>
>
> I like the idea of this commit, this should probably be the first one in
> the stack. The revlog cloning feature can be delayed later in the stack
> until we actually start performing upgrade.
>
> This commit does foreshadow some of the behavior of the new command,
>> notably that it doesn't do anything by default and that it takes
>> arguments that influence what actions it performs. These will be
>> explained more in subsequent commits.
>>
>> diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
>> --- a/mercurial/debugcommands.py
>> +++ b/mercurial/debugcommands.py
>> @@ -849,3 +849,28 @@ def debugdeltachain(ui, repo, file_=None
>>   extradist=extradist, extraratio=extraratio)
>>
>>  fm.end()
>> +
>> +@command('debugupgraderepo', [
>> +('o', 'optimize', [], _('extra optimization to perform'), _('NAME')),
>> +('', 'run', False, _('performs an upgrade')),
>>
>
> We should drop these option from the command until we actually use them
> this will allow to discuss them with their actual behavior.
>
> +])
>> +def debugupgraderepo(ui, repo, run=False, optimize=None):
>> +"""upgrade a repository to use different features
>> +
>> +If no arguments are specified, the repository is evaluated for
>> upgrade
>> +and a list of problems and potential optimizations is printed.
>> +
>> +With ``--run``, a repository upgrade is performed. Behavior of the
>> upgrade
>> +can be influenced via additional arguments. More details will be
>> provided
>> +by the command output when run without ``--run``.
>>
>
> (I would also delayed the help about --run until we actually run something)
>
> +
>> +During the upgrade, the repository will be locked and no writes will
>> be
>> +allowed.
>> +
>> +At the end of the upgrade, the repository may not be readable while
>> new
>> +repository data is swapped in. This window will be as long as it
>> takes to
>> +rename some directories inside the ``.hg`` directory. On most
>> machines, this
>> +should complete almost instantaneously and the chances of a consumer
>> being
>> +unable to access the repository should be low.
>>
>
> (I would also delay that doc until we actually behave that way)
> I know it would bot solve "everything" but we discussed the option of
> having a "repositorybeingupgraded" (or similar) in .hg/required to reduce
> the change for a reader to access the repository at that time.
>

I added all this code and documentation to foreshadow the feature so a
reviewer isn't guessing where things are going. It also cuts down on the
amount of patches and/or churn in future patches, making code review
easier, IMO. I think it is busy work to change it at this stage. But if you
insist, I will.


>
> +"""
>> +raise error.Abort(_('not yet implemented'))
>> diff --git a/tests/test-completion.t b/tests/test-completion.t
>> --- a/tests/test-completion.t
>> +++ b/tests/test-completion.t
>> @@ -109,6 +109,7 @@ Show debug commands if there are no othe
>>debugsub
>>debugsuccessorssets
>>debugtemplate
>> +  debugupgraderepo
>>debugwalk
>>debugwireargs
>>
>> @@ -274,6 +275,7 @@ Show all commands + options
>>debugsub: rev
>>debugsuccessorssets:
>>debugtemplate: rev, define
>> +  debugupgraderepo: optimize, run
>>debugwalk: include, exclude
>>debugwireargs: three, four, five, ssh, remotecmd, insecure
>>files: rev, print0, include, exclude, template, subrepos
>> diff --git a/tests/test-help.t b/tests/test-help.t
>> --- a/tests/test-help.t
>> +++ b/tests/test-help.t
>> @@ -916,6 +916,8 @@ Test list of i

Re: [PATCH 2 of 7 upgraderepo V2] debugcommands: stub for debugupgraderepo command

2016-12-12 Thread Pierre-Yves David



On 11/25/2016 04:28 AM, Gregory Szorc wrote:

# HG changeset patch
# User Gregory Szorc 
# Date 1480033449 28800
#  Thu Nov 24 16:24:09 2016 -0800
# Node ID 4fffbdc2bca22e666482b55562c0e592c8a68027
# Parent  c60995fce14b0e34bd1416dab3712a6c33bb29bb
debugcommands: stub for debugupgraderepo command

Currently, if Mercurial introduces a new repository/store feature or
changes behavior of an existing feature, users must perform an
`hg clone` to create a new repository with hopefully the
correct/optimal settings. Unfortunately, even `hg clone` may not
give the correct results. For example, if you do a local `hg clone`,
you may get hardlinks to revlog files that inherit the old state.
If you `hg clone` from a remote or `hg clone --pull`, changegroup
application may bypass some optimization, such as converting to
generaldelta.

Optimizing a repository is harder than it seems and requires more
than a simple `hg` command invocation.

This commit starts the process of changing that. We introduce
`hg debugupgraderepo`, a command that performs an in-place upgrade
of a repository to use new, optimal features. The command is just
a stub right now. Features will be added in subsequent commits.


I like the idea of this commit, this should probably be the first one in 
the stack. The revlog cloning feature can be delayed later in the stack 
until we actually start performing upgrade.



This commit does foreshadow some of the behavior of the new command,
notably that it doesn't do anything by default and that it takes
arguments that influence what actions it performs. These will be
explained more in subsequent commits.

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -849,3 +849,28 @@ def debugdeltachain(ui, repo, file_=None
  extradist=extradist, extraratio=extraratio)

 fm.end()
+
+@command('debugupgraderepo', [
+('o', 'optimize', [], _('extra optimization to perform'), _('NAME')),
+('', 'run', False, _('performs an upgrade')),


We should drop these option from the command until we actually use them 
this will allow to discuss them with their actual behavior.



+])
+def debugupgraderepo(ui, repo, run=False, optimize=None):
+"""upgrade a repository to use different features
+
+If no arguments are specified, the repository is evaluated for upgrade
+and a list of problems and potential optimizations is printed.
+
+With ``--run``, a repository upgrade is performed. Behavior of the upgrade
+can be influenced via additional arguments. More details will be provided
+by the command output when run without ``--run``.


(I would also delayed the help about --run until we actually run something)


+
+During the upgrade, the repository will be locked and no writes will be
+allowed.
+
+At the end of the upgrade, the repository may not be readable while new
+repository data is swapped in. This window will be as long as it takes to
+rename some directories inside the ``.hg`` directory. On most machines, 
this
+should complete almost instantaneously and the chances of a consumer being
+unable to access the repository should be low.


(I would also delay that doc until we actually behave that way)
I know it would bot solve "everything" but we discussed the option of 
having a "repositorybeingupgraded" (or similar) in .hg/required to 
reduce the change for a reader to access the repository at that time.



+"""
+raise error.Abort(_('not yet implemented'))
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -109,6 +109,7 @@ Show debug commands if there are no othe
   debugsub
   debugsuccessorssets
   debugtemplate
+  debugupgraderepo
   debugwalk
   debugwireargs

@@ -274,6 +275,7 @@ Show all commands + options
   debugsub: rev
   debugsuccessorssets:
   debugtemplate: rev, define
+  debugupgraderepo: optimize, run
   debugwalk: include, exclude
   debugwireargs: three, four, five, ssh, remotecmd, insecure
   files: rev, print0, include, exclude, template, subrepos
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -916,6 +916,8 @@ Test list of internal help commands
  show set of successors for revision
debugtemplate
  parse and apply a template
+   debugupgraderepo
+ upgrade a repository to use different features
debugwalk show how files match on given patterns
debugwireargs
  (no help text available)
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
new file mode 100644
--- /dev/null
+++ b/tests/test-upgrade-repo.t
@@ -0,0 +1,5 @@
+  $ hg init empty
+  $ cd empty
+  $ hg debugupgraderepo
+  abort: not yet implemented
+  [255]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.or

[PATCH 2 of 7 upgraderepo V2] debugcommands: stub for debugupgraderepo command

2016-11-24 Thread Gregory Szorc
# HG changeset patch
# User Gregory Szorc 
# Date 1480033449 28800
#  Thu Nov 24 16:24:09 2016 -0800
# Node ID 4fffbdc2bca22e666482b55562c0e592c8a68027
# Parent  c60995fce14b0e34bd1416dab3712a6c33bb29bb
debugcommands: stub for debugupgraderepo command

Currently, if Mercurial introduces a new repository/store feature or
changes behavior of an existing feature, users must perform an
`hg clone` to create a new repository with hopefully the
correct/optimal settings. Unfortunately, even `hg clone` may not
give the correct results. For example, if you do a local `hg clone`,
you may get hardlinks to revlog files that inherit the old state.
If you `hg clone` from a remote or `hg clone --pull`, changegroup
application may bypass some optimization, such as converting to
generaldelta.

Optimizing a repository is harder than it seems and requires more
than a simple `hg` command invocation.

This commit starts the process of changing that. We introduce
`hg debugupgraderepo`, a command that performs an in-place upgrade
of a repository to use new, optimal features. The command is just
a stub right now. Features will be added in subsequent commits.

This commit does foreshadow some of the behavior of the new command,
notably that it doesn't do anything by default and that it takes
arguments that influence what actions it performs. These will be
explained more in subsequent commits.

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -849,3 +849,28 @@ def debugdeltachain(ui, repo, file_=None
  extradist=extradist, extraratio=extraratio)
 
 fm.end()
+
+@command('debugupgraderepo', [
+('o', 'optimize', [], _('extra optimization to perform'), _('NAME')),
+('', 'run', False, _('performs an upgrade')),
+])
+def debugupgraderepo(ui, repo, run=False, optimize=None):
+"""upgrade a repository to use different features
+
+If no arguments are specified, the repository is evaluated for upgrade
+and a list of problems and potential optimizations is printed.
+
+With ``--run``, a repository upgrade is performed. Behavior of the upgrade
+can be influenced via additional arguments. More details will be provided
+by the command output when run without ``--run``.
+
+During the upgrade, the repository will be locked and no writes will be
+allowed.
+
+At the end of the upgrade, the repository may not be readable while new
+repository data is swapped in. This window will be as long as it takes to
+rename some directories inside the ``.hg`` directory. On most machines, 
this
+should complete almost instantaneously and the chances of a consumer being
+unable to access the repository should be low.
+"""
+raise error.Abort(_('not yet implemented'))
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -109,6 +109,7 @@ Show debug commands if there are no othe
   debugsub
   debugsuccessorssets
   debugtemplate
+  debugupgraderepo
   debugwalk
   debugwireargs
 
@@ -274,6 +275,7 @@ Show all commands + options
   debugsub: rev
   debugsuccessorssets: 
   debugtemplate: rev, define
+  debugupgraderepo: optimize, run
   debugwalk: include, exclude
   debugwireargs: three, four, five, ssh, remotecmd, insecure
   files: rev, print0, include, exclude, template, subrepos
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -916,6 +916,8 @@ Test list of internal help commands
  show set of successors for revision
debugtemplate
  parse and apply a template
+   debugupgraderepo
+ upgrade a repository to use different features
debugwalk show how files match on given patterns
debugwireargs
  (no help text available)
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
new file mode 100644
--- /dev/null
+++ b/tests/test-upgrade-repo.t
@@ -0,0 +1,5 @@
+  $ hg init empty
+  $ cd empty
+  $ hg debugupgraderepo
+  abort: not yet implemented
+  [255]
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel