Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Nicholas Chammas]

Response from GitHub staff regarding using their Importer
 to import CPython:

Unfortunately, the repository is too large to migrate using the importer.
I’d recommend converting it to git locally using something like
hg-fast-export. Due to its size, you’ll need to push the local repo to
GitHub in chunks.

So it seems like Importer is a no-go.

Nick
​
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Petr Viktorin]

On 02/14/2016 06:24 PM, Pierre-Yves David wrote:
> On 02/13/2016 02:56 AM, Nick Coghlan wrote:
>> On 13 February 2016 at 11:15, Brett Cannon  wrote:
>>> I don't remember the story behind cpython-fullhistory, but it's
>>> obviously
>>> incomplete since just stopped post conversion. You will need to find
>>> someone
>>> who knows (I'd ask on python-dev).
>>>
>>> Also realize that this will be  our fourth VCS (cvs, svn, hg, now
>>> git). This
>>> is not going to be a perfect history of the semantic actions of
>>> commits from
>>> the beginning of time just due to the fact that these VCS tools all use
>>> different concepts.
>>
>> There's also the fact that prior to the move to SourceForge in 2000,
>> all changes had to be funneled through the half dozen or so people
>> with write access to the CVS tree:
>> https://docs.python.org/3/whatsnew/2.0.html#new-development-process
>>
>> I think it's definitely OK if future code archaeologists need to dig
>> into the SVN repository to get a more complete view of CPython's
>> history.
> 
> I've never met a project who did not regret such decision at some point.
> Keeping older history is usually valuable. Mercurial have powerful
> enough tool to let you get all the history back together, I assume git
> probably have that power too.
> 
> This is your call, but I strongly recommend taking advantage of this
> migration to put everything back together.
> 

While "putting everything back together" would be great, it doesn't
*have* to block the migration. Git has a command called "git replace"
that lets you do this later.

The Linux kernel (which switched to Git before Git migration tools
existed) has a separate "early history" repo that you can "prepend" to
the main one. Then, in your local copy, it looks like one unbroken
history. Since Git commits are snapshots and not deltas, this works
amazingly well -- it's just telling Git's object retrieval routine to
retrieve  instead of . The disadvantage is that it
has to be done in each clone individually -- no one can rewrite history
for others.

Two commands every future historian would have to do:
git fetch 
git replace --graft 



___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Nicolás Alvarez]

2016-02-14 1:27 GMT-03:00 Senthil Kumaran :
> On Fri, Feb 12, 2016 at 6:20 AM, Pierre-Yves David
>  wrote:
>>
>> We, at Facebook, have been happily using hg-git in production for
>> bidirectional conversion of repositories of insane sizes.
>
>
> I played with hg-git tool and was able to migrate the hg cpython default
> branch to git successfully. I have documented the steps here:
> https://github.com/orsenthil/cpython-hg-to-git/blob/master/README.md

Did you repack the git repository after importing with hg-git and
before pushing it? Every repository import tool I know of produces
extremely suboptimal pack files. fast-export initially gave me a 3GB
monstrosity before I ran "git gc --aggressive".

-- 
Nicolás
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Oleg Broytman]

Hi!

On Fri, Feb 12, 2016 at 06:04:19AM +, Martin Panter  
wrote:
> One thing I???m slightly curious about is how much the result differs
> from  or other results, and if so,
> what the differences are. The differences could be serious (mangled
> history), or they could be trivial things like stripping trailing
> newlines from commit messages, or skipping commits that don???t change
> any files.

   git-remote-hg that I have been using can be configured to produce
commits compatible with hg-git. If the repo above was created with
hg-git I can rerun my conversion with hg-git compatibility turned on.
Then it may be compared with the mentioned repository.

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Senthil Kumaran]

On Thu, Feb 11, 2016 at 4:04 PM, Matthias Bussonnier <
bussonniermatth...@gmail.com> wrote:

> I just saw the GitHub released a tool to import Mercurial repo:
>
> https://github.com/blog/2110-migrate-your-code-with-the-github-importer
>


Turns out that our CPython repository size is too big for this importer
tool to handle.
It fails for me consistently at 78% and I tried more than 10 times to push
the limit to 78%.

As Brett mentioned, I will report this to github.com team and let's
continue our evaluation of other tools
which can successfully complete the hg-> git translation of the repo.

-- 
Senthil
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Senthil Kumaran]

Hi Pierre,

  

Do you have external documents that you could point us to so that it help us
with the research?

Thanks for sharing this detail on hg-git already working for a huge
repo.

  

\--  
Senthil

  

> On Feb 12 2016, at 6:28 am, Pierre-Yves David pierre-yves.david@ens-
lyon.org wrote:  

>

> On 02/06/2016 01:57 AM, Brett Cannon wrote:  
 https://www.python.org/dev/peps/pep-0512/#define-commands-to-move-a
-mercurial-repository-to-git  
  
 There appear to be multiple ways to convert hg repos to git, but no  
 clear winner. It would be great if some one/people took on the task of  
 evaluating the tools available out there by converting the cpython repo  
 and seeing which one has the best results.

>

> We, at Facebook, have been happily using hg-git in production for  
bidirectional conversion of repositories of insane sizes.

>

> I would look in that direction for your conversion.

>

> \--  
Pierre-Yves David  
___  
core-workflow mailing list  
core-workflow@python.org  
https://mail.python.org/mailman/listinfo/core-workflow  
This list is governed by the PSF Code of Conduct:
https://www.python.org/psf/codeofconduct

___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Pierre-Yves David]

On 02/06/2016 01:57 AM, Brett Cannon wrote:

https://www.python.org/dev/peps/pep-0512/#define-commands-to-move-a-mercurial-repository-to-git

There appear to be multiple ways to convert hg repos to git, but no
clear winner. It would be great if some one/people took on the task of
evaluating the tools available out there by converting the cpython repo
and seeing which one has the best results.


We, at Facebook, have been happily using hg-git in production for 
bidirectional conversion of repositories of insane sizes.


I would look in that direction for your conversion.

--
Pierre-Yves David
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Pierre-Yves David]

On 02/12/2016 02:37 PM, Senthil Kumaran wrote:

Hi Pierre,

Do you have external documents that you could point us to so that it
help us with the research?
Thanks for sharing this detail on hg<->git already working for a huge repo.


I would expect the hg-git documentation to be the place to start.

--
Pierre-Yves David
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Ryan Gonzalez]

How did you manage to push the repo somewhere? hg-git successfully exported all 
the changesets for me, but I couldn't push to a directory, and my computer ran 
out of memory twice during the adding objects stage when trying to push to 
GitHub.

On February 12, 2016 8:41:08 AM CST, Pierre-Yves David 
 wrote:
>
>
>On 02/12/2016 02:37 PM, Senthil Kumaran wrote:
>> Hi Pierre,
>>
>> Do you have external documents that you could point us to so that it
>> help us with the research?
>> Thanks for sharing this detail on hg<->git already working for a huge
>repo.
>
>I would expect the hg-git documentation to be the place to start.
>
>-- 
>Pierre-Yves David
>___
>core-workflow mailing list
>core-workflow@python.org
>https://mail.python.org/mailman/listinfo/core-workflow
>This list is governed by the PSF Code of Conduct:
>https://www.python.org/psf/codeofconduct

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Martin Panter]

On 12 February 2016 at 03:07, Brett Cannon  wrote:
> On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez 
> wrote:
>> I tried fast-export, and I don't really see anything wrong with the
>> repository. The size is 221MB.

One thing I’m slightly curious about is how much the result differs
from  or other results, and if so,
what the differences are. The differences could be serious (mangled
history), or they could be trivial things like stripping trailing
newlines from commit messages, or skipping commits that don’t change
any files.

>> It depends on how crazy you want to go. For example, SVN-era merges
>> don't appear as merges, but looks like some SVN-era branches don't
>> exist in Hg to begin with (Would I need to get cpython-fullhistory?
>> Cloning it gives me a 400 Bad Request). Do we care about that?
>
> Good question. If you are not an even clone it then that shows how much
> people who are. Honestly I wouldn't worry since we have the history in the
> hg repo (converting from svn was necessary to have it available without the
> server).

I care a bit. If I get the time, I would like to figure out a robust
way to convert the Subversion history to Git so that the svnmerge
information is included as proper merges.

Another concern for me is that some of the useful history is not even
in Mercurial. For example  is an
svnmerge from ^/python/branches/io-c into ^/python/branches/py3k, but
the Mercurial repository doesn’t have the branch history, so all the
merged-in Subversion revisions such as r68683 are missing.

Some other highlights on my quest to investigate the holy Subversion
respository (I can post my full notes somewhere if ppl are
interested):

* It is nice to have a local mirror of the Subversion repository so
that experimenting with different options and programs isn’t horribly
slow. But I don’t want to mirror everything or overload the server
because there are other projects stored in the repository that seem to
take up a lot of space (and download time).

* What is the story with the cpython-fullhistory Mercurial repository?
On the surface it almost looks like an out-of-date copy of the main
repository, but I notice some subtle differences, e.g. revision ids
for early tags are different, v1.0.0 tag is added.

* Some Subversion revisions actually merge stuff from outside the
Python tree (e.g.  from
^/sandbox/trunk/2to3/lib2to3 into
^/branches/release27-maint/Lib/lib2to3. Not sure if it is worth trying
to salvage these merges; I never noticed them when working on Python.

>> Or, changes that come from non-committers could have their Author
>> field modified, maybe based on the ACKS file modification. It's
>> feasible but will take time and manual work. Do we care about that?
>
> That would be great but too much effort.

I think it would not be worth it, and could even be detrimenal. You
would be trying to guess based on incomplete and unreliable
information. Maybe one person wrote a test, another wrote the
implementation, and a third wrote the documentation, but it was all
committed at once. Maybe the author was already in ACKS and the
committer did not mention who the author was in the message. I think
it is safer to not pretend the author field is alway accurate.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Nicolás Alvarez]

2016-02-12 3:04 GMT-03:00 Martin Panter :
> On 12 February 2016 at 03:07, Brett Cannon  wrote:
>> On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez 
>> wrote:
>>> It depends on how crazy you want to go. For example, SVN-era merges
>>> don't appear as merges, but looks like some SVN-era branches don't
>>> exist in Hg to begin with (Would I need to get cpython-fullhistory?
>>> Cloning it gives me a 400 Bad Request). Do we care about that?
>>
>> Good question. If you are not an even clone it then that shows how much
>> people who are. Honestly I wouldn't worry since we have the history in the
>> hg repo (converting from svn was necessary to have it available without the
>> server).
>
> I care a bit. If I get the time, I would like to figure out a robust
> way to convert the Subversion history to Git so that the svnmerge
> information is included as proper merges.

I migrated most of KDE from SVN to Git, progressively converting a
single million-revision repository into hundreds of per-app Git
repositories.

Is it still possible to access the SVN repository? :)

> Some other highlights on my quest to investigate the holy Subversion
> respository (I can post my full notes somewhere if ppl are
> interested):
>
> * It is nice to have a local mirror of the Subversion repository so
> that experimenting with different options and programs isn’t horribly
> slow. But I don’t want to mirror everything or overload the server
> because there are other projects stored in the repository that seem to
> take up a lot of space (and download time).

The svn2git tool we used in KDE *requires* the repository to be local,
because the libsvn API it uses works with repositories, not server
URLs.

-- 
Nicolás
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Martin Panter]

On 12 February 2016 at 06:27, Nicolás Alvarez  wrote:
> 2016-02-12 3:04 GMT-03:00 Martin Panter :
>> On 12 February 2016 at 03:07, Brett Cannon  wrote:
>>> On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez 
>>> wrote:
 It depends on how crazy you want to go. For example, SVN-era merges
 don't appear as merges, but looks like some SVN-era branches don't
 exist in Hg to begin with (Would I need to get cpython-fullhistory?
 Cloning it gives me a 400 Bad Request). Do we care about that?
>>>
>>> Good question. If you are not an even clone it then that shows how much
>>> people who are. Honestly I wouldn't worry since we have the history in the
>>> hg repo (converting from svn was necessary to have it available without the
>>> server).
>>
>> I care a bit. If I get the time, I would like to figure out a robust
>> way to convert the Subversion history to Git so that the svnmerge
>> information is included as proper merges.
>
> I migrated most of KDE from SVN to Git, progressively converting a
> single million-revision repository into hundreds of per-app Git
> repositories.
>
> Is it still possible to access the SVN repository? :)

svn info https://svn.python.org/projects/python

>> Some other highlights on my quest to investigate the holy Subversion
>> respository (I can post my full notes somewhere if ppl are
>> interested):
>>
>> * It is nice to have a local mirror of the Subversion repository so
>> that experimenting with different options and programs isn’t horribly
>> slow. But I don’t want to mirror everything or overload the server
>> because there are other projects stored in the repository that seem to
>> take up a lot of space (and download time).
>
> The svn2git tool we used in KDE *requires* the repository to be local,
> because the libsvn API it uses works with repositories, not server
> URLs.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Matthias Bussonnier]

Hey,

I just saw the GitHub released a tool to import Mercurial repo:

https://github.com/blog/2110-migrate-your-code-with-the-github-importer

Thought it might be usefull for this thread.

I'm currently trying to import to see how it looks, have been stuck at
0% for a few minutes now.

-- 
M

Will be at there if import succeed.
https://github.com/Carreau/cpython-test



On Mon, Feb 8, 2016 at 10:23 AM, Brett Cannon  wrote:
>
>
> On Mon, 8 Feb 2016 at 09:54 Senthil Kumaran  wrote:
>>
>> Hi Brett,
>>
>> On Sat, Feb 6, 2016 at 11:42 AM, Brett Cannon  wrote:
>>>
>>> It seems there are people definitely willing to try out the tools and
>>> report back, but I'm looking for someone/people to organize the effort and
>>> come back to me with a thought-out solution so I don't have to. :)
>>
>>
>> I wanted to get involved with this process. I am ready to help you here.
>>
>> https://github.com/orsenthil/cpython-hg-to-git
>>
>> Here I started documenting our discussion so far. I will continue to
>> evaluate the tools (with other developers this list) and we'll try to come
>> up with a conclusive suggestion for our migration.
>>
>
> Thanks to much, Senthil! When I get a chance I will update the PEP to say
> this step of the migration is in process and you as in charge of it.
>
> -Brett
>
> P.S.: For this whole migration I'm going to try and delegate where
> appropriate, so this won't be the last time I reach out for help. I
> anticipate helping with bugs.python.org is going to be the biggest stumbling
> block/need. And for those that wanted GitLab over GitHub, a vast majority of
> this migration is Git host-agnostic, and so this is necessary for after we
> leave GitHub (either because we're unhappy with GitHub or because it's
> simply time to move; nothing is forever).
>
> ___
> core-workflow mailing list
> core-workflow@python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Nicholas Chammas]

> I'm currently trying to import to see how it looks, have been stuck at
0% for a few minutes now.

Doing the same myself. Got to 73% and it restarted. Am back at 73% now.

Already reached out to GitHub to make them aware of the issue.

Will report here when/if I have results.

Nick
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Brett Cannon]

On Thu, 11 Feb 2016 at 17:07 Senthil Kumaran  wrote:

>
> On Thu, Feb 11, 2016 at 4:04 PM, Matthias Bussonnier <
> bussonniermatth...@gmail.com> wrote:
>
>> I just saw the GitHub released a tool to import Mercurial repo:
>>
>> https://github.com/blog/2110-migrate-your-code-with-the-github-importer
>>
>> Thought it might be usefull for this thread.
>>
>> I'm currently trying to import to see how it looks, have been stuck at
>> 0% for a few minutes now.
>>
>
> Yeah, I noticed it too and I has said that it will send an email once the
> import it done.
> Let's see if this feature provides all the things are looking for.
>
> This is interesting one for our needs.
>

Two things. One, I just updated listing Senthil as in charge of evaluating
the tools. Should go live in an hour or so (aside: I'm going to delete my
GitHub repo for the PEP so I don't have to keep track of two copies; can't
wait until we get this done enough to move the peps repo over).

Two, thanks for everyone who jumped on the GitHub blog post and posting
here! I noticed myself and was coming to comment when I noticed Matthias,
Nick, Nicolás, and Senthil had beaten me to it. :) As Senthil said, it's
very interesting to see this option open up to us and will need to see how
they do. Hopefully it will work out (or at least we can help GitHub fix
their tooling if they can't handle the cpython repo's deep history).
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Ethan Furman]

On 02/11/2016 07:07 PM, Brett Cannon wrote:
> On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez wrote:

>> It depends on how crazy you want to go. For example, SVN-era merges
>> don't appear as merges, but looks like some SVN-era branches don't
>> exist in Hg to begin with (Would I need to get cpython-fullhistory?
>> Cloning it gives me a 400 Bad Request). Do we care about that?

> If you are not an even clone it then that shows how much
> people who are.

Um, could you repeat that?  In English?  :)

--
~Ethan~
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Brett Cannon]

On Thu, Feb 11, 2016, 19:27 Ethan Furman  wrote:

> On 02/11/2016 07:07 PM, Brett Cannon wrote:
>  > On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez wrote:
>
>  >> It depends on how crazy you want to go. For example, SVN-era merges
>  >> don't appear as merges, but looks like some SVN-era branches don't
>  >> exist in Hg to begin with (Would I need to get cpython-fullhistory?
>  >> Cloning it gives me a 400 Bad Request). Do we care about that?
>
>  > If you are not an even clone it then that shows how much
>  > people who are.
>
> Um, could you repeat that?  In English?  :)
>

If you aren't able to even clone it then that shows how much people care.

Brett


> --
> ~Ethan~
> ___
> core-workflow mailing list
> core-workflow@python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Michiel Overtoom]

Hi,

ESR wrote a repository migration tool which probably could be used for this, 
Reposurgeon:

  http://www.catb.org/esr/reposurgeon/

He used it (among other things) to migrate the huge emacs repository to git: 
http://esr.ibiblio.org/?p=5634

PS. Reposurgon is written in Python ;-)

Greetings,

___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Brett Cannon]

On Mon, 8 Feb 2016 at 01:19 Michiel Overtoom  wrote:

>
> Hi,
>
> ESR wrote a repository migration tool which probably could be used for
> this, Reposurgeon:
>
>   http://www.catb.org/esr/reposurgeon/
>
> He used it (among other things) to migrate the huge emacs repository to
> git: http://esr.ibiblio.org/?p=5634
>
> PS. Reposurgon is written in Python ;-)
>

If you (or anyone else) wants to try converting the cpython repo using the
tool that would be appreciated!
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Brett Cannon]

On Mon, 8 Feb 2016 at 09:54 Senthil Kumaran  wrote:

> Hi Brett,
>
> On Sat, Feb 6, 2016 at 11:42 AM, Brett Cannon  wrote:
>
>> It seems there are people definitely willing to try out the tools and
>> report back, but I'm looking for someone/people to organize the effort and
>> come back to me with a thought-out solution so I don't have to. :)
>>
>
> I wanted to get involved with this process. I am ready to help you here.
>
> https://github.com/orsenthil/cpython-hg-to-git
>
> Here I started documenting our discussion so far. I will continue to
> evaluate the tools (with other developers this list) and we'll try to come
> up with a conclusive suggestion for our migration.
>
>
Thanks to much, Senthil! When I get a chance I will update the PEP to say
this step of the migration is in process and you as in charge of it.

-Brett

P.S.: For this whole migration I'm going to try and delegate where
appropriate, so this won't be the last time I reach out for help. I
anticipate helping with bugs.python.org is going to be the biggest
stumbling block/need. And for those that wanted GitLab over GitHub, a vast
majority of this migration is Git host-agnostic, and so this is necessary
for after we leave GitHub (either because we're unhappy with GitHub or
because it's simply time to move; nothing is forever).
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Senthil Kumaran]

Hi Brett,

On Sat, Feb 6, 2016 at 11:42 AM, Brett Cannon  wrote:

> It seems there are people definitely willing to try out the tools and
> report back, but I'm looking for someone/people to organize the effort and
> come back to me with a thought-out solution so I don't have to. :)
>

I wanted to get involved with this process. I am ready to help you here.

https://github.com/orsenthil/cpython-hg-to-git

Here I started documenting our discussion so far. I will continue to
evaluate the tools (with other developers this list) and we'll try to come
up with a conclusive suggestion for our migration.

Thank you,
Senthil
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Ryan Gonzalez]

Well, hg-git might not work...

I cloned the CPython repo. I then ran `hg gexport`, which converts the
repository to Git. Took three hours, but it worked!

However, actually attempted to push the result to a Git repo failed
miserably. After five hours, Mercurial ran out of memory in the "adding
objects" stage. And I have 6 GB of RAM!

Right now, I'm trying to see if I can work around it. This may not work
out, though.

On Sat, Feb 6, 2016 at 10:37 AM, Ryan Gonzalez  wrote:

> I'll try hg-git!
>
> On February 5, 2016 7:57:15 PM CST, Brett Cannon  wrote:
>>
>>
>> https://www.python.org/dev/peps/pep-0512/#define-commands-to-move-a-mercurial-repository-to-git
>>
>> There appear to be multiple ways to convert hg repos to git, but no clear
>> winner. It would be great if some one/people took on the task of evaluating
>> the tools available out there by converting the cpython repo and seeing
>> which one has the best results.
>>
>> --
>>
>> core-workflow mailing list
>> core-workflow@python.org
>> https://mail.python.org/mailman/listinfo/core-workflow
>> This list is governed by the PSF Code of Conduct: 
>> https://www.python.org/psf/codeofconduct
>>
>>
> --
> Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.
>



-- 
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something’s wrong.
http://kirbyfan64.github.io/
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Ryan Gonzalez]

I'll try hg-git!

On February 5, 2016 7:57:15 PM CST, Brett Cannon  wrote:
>https://www.python.org/dev/peps/pep-0512/#define-commands-to-move-a-mercurial-repository-to-git
>
>There appear to be multiple ways to convert hg repos to git, but no
>clear
>winner. It would be great if some one/people took on the task of
>evaluating
>the tools available out there by converting the cpython repo and seeing
>which one has the best results.
>
>
>
>
>___
>core-workflow mailing list
>core-workflow@python.org
>https://mail.python.org/mailman/listinfo/core-workflow
>This list is governed by the PSF Code of Conduct:
>https://www.python.org/psf/codeofconduct

-- 
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Petr Viktorin]

On 02/06/2016 05:48 PM, Oleg Broytman wrote:
> On Sat, Feb 06, 2016 at 04:50:51PM +0100, Oleg Broytman  
> wrote:
>> Hello!
>>
>> On Sat, Feb 06, 2016 at 01:57:15AM +, Brett Cannon  
>> wrote:
>>> https://www.python.org/dev/peps/pep-0512/#define-commands-to-move-a-mercurial-repository-to-git
>>>
>>> There appear to be multiple ways to convert hg repos to git, but no clear
>>> winner. It would be great if some one/people took on the task of evaluating
>>> the tools available out there by converting the cpython repo and seeing
>>> which one has the best results.
>>
>>Let's me try git-remote-hg transport.
>>
>> $ hg clone https://hg.python.org/cpython/ cpython-hg
>>
>> $ time git clone hg::cpython-hg cpython-git
>> real  39m44.600s
>> user  45m54.192s
>> sys   1m4.184s
>>
>> $ cd cpython-git/
> 
> $ git remote add gh g...@github.com:phdru/cpython.git
> 
> $ git push --all gh
> 
>See the result at https://github.com/phdru/cpython

You might also want to try something like

   $ git repack -a -d -f --depth=250 --window=250

after importing to decrease the size of the repository for everyone
pulling from you.

(Reference: https://gcc.gnu.org/ml/gcc/2007-12/msg00165.html)

___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Oleg Broytman]

On Sat, Feb 06, 2016 at 04:50:51PM +0100, Oleg Broytman  wrote:
> Hello!
> 
> On Sat, Feb 06, 2016 at 01:57:15AM +, Brett Cannon  
> wrote:
> > https://www.python.org/dev/peps/pep-0512/#define-commands-to-move-a-mercurial-repository-to-git
> > 
> > There appear to be multiple ways to convert hg repos to git, but no clear
> > winner. It would be great if some one/people took on the task of evaluating
> > the tools available out there by converting the cpython repo and seeing
> > which one has the best results.
> 
>Let's me try git-remote-hg transport.
> 
> $ hg clone https://hg.python.org/cpython/ cpython-hg
> 
> $ time git clone hg::cpython-hg cpython-git
> real  39m44.600s
> user  45m54.192s
> sys   1m4.184s
> 
> $ cd cpython-git/

$ git remote add gh g...@github.com:phdru/cpython.git

$ git push --all gh

   See the result at https://github.com/phdru/cpython

PS. I am `phdru' at Github.

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

[Oleg Broytman]

On Sat, Feb 06, 2016 at 05:57:47PM +0100, Petr Viktorin  
wrote:
> On 02/06/2016 05:48 PM, Oleg Broytman wrote:
> > $ git remote add gh g...@github.com:phdru/cpython.git
> > 
> > $ git push --all gh
> > 
> >See the result at https://github.com/phdru/cpython
> 
> You might also want to try something like
> 
>$ git repack -a -d -f --depth=250 --window=250
> 
> after importing to decrease the size of the repository for everyone
> pulling from you.
> 
> (Reference: https://gcc.gnu.org/ml/gcc/2007-12/msg00165.html)

   See the discussion of this old and outdated message at PEP 103:
https://www.python.org/dev/peps/pep-0103/#database-maintenance

   The recommended parameters for git repack are --depth=20 --window=250
See http://vcscompare.blogspot.ru/2008/06/git-repack-parameters.html

   But yes, you're right, git gc/repack is very much recommended.

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct