Re: [core-workflow] Help needed: best way to convert hg repos to git?

2016-02-11 Thread Martin Panter 
On 12 February 2016 at 03:07, Brett Cannon  wrote:
> On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez 
> wrote:
>> I tried fast-export, and I don't really see anything wrong with the
>> repository. The size is 221MB.

One thing I’m slightly curious about is how much the result differs
from  or other results, and if so,
what the differences are. The differences could be serious (mangled
history), or they could be trivial things like stripping trailing
newlines from commit messages, or skipping commits that don’t change
any files.

>> It depends on how crazy you want to go. For example, SVN-era merges
>> don't appear as merges, but looks like some SVN-era branches don't
>> exist in Hg to begin with (Would I need to get cpython-fullhistory?
>> Cloning it gives me a 400 Bad Request). Do we care about that?
>
> Good question. If you are not an even clone it then that shows how much
> people who are. Honestly I wouldn't worry since we have the history in the
> hg repo (converting from svn was necessary to have it available without the
> server).

I care a bit. If I get the time, I would like to figure out a robust
way to convert the Subversion history to Git so that the svnmerge
information is included as proper merges.

Another concern for me is that some of the useful history is not even
in Mercurial. For example  is an
svnmerge from ^/python/branches/io-c into ^/python/branches/py3k, but
the Mercurial repository doesn’t have the branch history, so all the
merged-in Subversion revisions such as r68683 are missing.

Some other highlights on my quest to investigate the holy Subversion
respository (I can post my full notes somewhere if ppl are
interested):

* It is nice to have a local mirror of the Subversion repository so
that experimenting with different options and programs isn’t horribly
slow. But I don’t want to mirror everything or overload the server
because there are other projects stored in the repository that seem to
take up a lot of space (and download time).

* What is the story with the cpython-fullhistory Mercurial repository?
On the surface it almost looks like an out-of-date copy of the main
repository, but I notice some subtle differences, e.g. revision ids
for early tags are different, v1.0.0 tag is added.

* Some Subversion revisions actually merge stuff from outside the
Python tree (e.g.  from
^/sandbox/trunk/2to3/lib2to3 into
^/branches/release27-maint/Lib/lib2to3. Not sure if it is worth trying
to salvage these merges; I never noticed them when working on Python.

>> Or, changes that come from non-committers could have their Author
>> field modified, maybe based on the ACKS file modification. It's
>> feasible but will take time and manual work. Do we care about that?
>
> That would be great but too much effort.

I think it would not be worth it, and could even be detrimenal. You
would be trying to guess based on incomplete and unreliable
information. Maybe one person wrote a test, another wrote the
implementation, and a third wrote the documentation, but it was all
committed at once. Maybe the author was already in ACKS and the
committer did not mention who the author was in the message. I think
it is safer to not pretend the author field is alway accurate.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

2016-02-11 Thread Nicolás Alvarez 
2016-02-12 3:04 GMT-03:00 Martin Panter :
> On 12 February 2016 at 03:07, Brett Cannon  wrote:
>> On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez 
>> wrote:
>>> It depends on how crazy you want to go. For example, SVN-era merges
>>> don't appear as merges, but looks like some SVN-era branches don't
>>> exist in Hg to begin with (Would I need to get cpython-fullhistory?
>>> Cloning it gives me a 400 Bad Request). Do we care about that?
>>
>> Good question. If you are not an even clone it then that shows how much
>> people who are. Honestly I wouldn't worry since we have the history in the
>> hg repo (converting from svn was necessary to have it available without the
>> server).
>
> I care a bit. If I get the time, I would like to figure out a robust
> way to convert the Subversion history to Git so that the svnmerge
> information is included as proper merges.

I migrated most of KDE from SVN to Git, progressively converting a
single million-revision repository into hundreds of per-app Git
repositories.

Is it still possible to access the SVN repository? :)

> Some other highlights on my quest to investigate the holy Subversion
> respository (I can post my full notes somewhere if ppl are
> interested):
>
> * It is nice to have a local mirror of the Subversion repository so
> that experimenting with different options and programs isn’t horribly
> slow. But I don’t want to mirror everything or overload the server
> because there are other projects stored in the repository that seem to
> take up a lot of space (and download time).

The svn2git tool we used in KDE *requires* the repository to be local,
because the libsvn API it uses works with repositories, not server
URLs.

-- 
Nicolás
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

2016-02-11 Thread Martin Panter 
On 12 February 2016 at 06:27, Nicolás Alvarez  wrote:
> 2016-02-12 3:04 GMT-03:00 Martin Panter :
>> On 12 February 2016 at 03:07, Brett Cannon  wrote:
>>> On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez 
>>> wrote:
 It depends on how crazy you want to go. For example, SVN-era merges
 don't appear as merges, but looks like some SVN-era branches don't
 exist in Hg to begin with (Would I need to get cpython-fullhistory?
 Cloning it gives me a 400 Bad Request). Do we care about that?
>>>
>>> Good question. If you are not an even clone it then that shows how much
>>> people who are. Honestly I wouldn't worry since we have the history in the
>>> hg repo (converting from svn was necessary to have it available without the
>>> server).
>>
>> I care a bit. If I get the time, I would like to figure out a robust
>> way to convert the Subversion history to Git so that the svnmerge
>> information is included as proper merges.
>
> I migrated most of KDE from SVN to Git, progressively converting a
> single million-revision repository into hundreds of per-app Git
> repositories.
>
> Is it still possible to access the SVN repository? :)

svn info https://svn.python.org/projects/python

>> Some other highlights on my quest to investigate the holy Subversion
>> respository (I can post my full notes somewhere if ppl are
>> interested):
>>
>> * It is nice to have a local mirror of the Subversion repository so
>> that experimenting with different options and programs isn’t horribly
>> slow. But I don’t want to mirror everything or overload the server
>> because there are other projects stored in the repository that seem to
>> take up a lot of space (and download time).
>
> The svn2git tool we used in KDE *requires* the repository to be local,
> because the libsvn API it uses works with repositories, not server
> URLs.
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

2016-02-11 Thread Matthias Bussonnier 
Hey,

I just saw the GitHub released a tool to import Mercurial repo:

https://github.com/blog/2110-migrate-your-code-with-the-github-importer

Thought it might be usefull for this thread.

I'm currently trying to import to see how it looks, have been stuck at
0% for a few minutes now.

-- 
M

Will be at there if import succeed.
https://github.com/Carreau/cpython-test



On Mon, Feb 8, 2016 at 10:23 AM, Brett Cannon  wrote:
>
>
> On Mon, 8 Feb 2016 at 09:54 Senthil Kumaran  wrote:
>>
>> Hi Brett,
>>
>> On Sat, Feb 6, 2016 at 11:42 AM, Brett Cannon  wrote:
>>>
>>> It seems there are people definitely willing to try out the tools and
>>> report back, but I'm looking for someone/people to organize the effort and
>>> come back to me with a thought-out solution so I don't have to. :)
>>
>>
>> I wanted to get involved with this process. I am ready to help you here.
>>
>> https://github.com/orsenthil/cpython-hg-to-git
>>
>> Here I started documenting our discussion so far. I will continue to
>> evaluate the tools (with other developers this list) and we'll try to come
>> up with a conclusive suggestion for our migration.
>>
>
> Thanks to much, Senthil! When I get a chance I will update the PEP to say
> this step of the migration is in process and you as in charge of it.
>
> -Brett
>
> P.S.: For this whole migration I'm going to try and delegate where
> appropriate, so this won't be the last time I reach out for help. I
> anticipate helping with bugs.python.org is going to be the biggest stumbling
> block/need. And for those that wanted GitLab over GitHub, a vast majority of
> this migration is Git host-agnostic, and so this is necessary for after we
> leave GitHub (either because we're unhappy with GitHub or because it's
> simply time to move; nothing is forever).
>
> ___
> core-workflow mailing list
> core-workflow@python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

2016-02-11 Thread Nicholas Chammas 
> I'm currently trying to import to see how it looks, have been stuck at
0% for a few minutes now.

Doing the same myself. Got to 73% and it restarted. Am back at 73% now.

Already reached out to GitHub to make them aware of the issue.

Will report here when/if I have results.

Nick
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

2016-02-11 Thread Brett Cannon 
On Thu, 11 Feb 2016 at 17:07 Senthil Kumaran  wrote:

>
> On Thu, Feb 11, 2016 at 4:04 PM, Matthias Bussonnier <
> bussonniermatth...@gmail.com> wrote:
>
>> I just saw the GitHub released a tool to import Mercurial repo:
>>
>> https://github.com/blog/2110-migrate-your-code-with-the-github-importer
>>
>> Thought it might be usefull for this thread.
>>
>> I'm currently trying to import to see how it looks, have been stuck at
>> 0% for a few minutes now.
>>
>
> Yeah, I noticed it too and I has said that it will send an email once the
> import it done.
> Let's see if this feature provides all the things are looking for.
>
> This is interesting one for our needs.
>

Two things. One, I just updated listing Senthil as in charge of evaluating
the tools. Should go live in an hour or so (aside: I'm going to delete my
GitHub repo for the PEP so I don't have to keep track of two copies; can't
wait until we get this done enough to move the peps repo over).

Two, thanks for everyone who jumped on the GitHub blog post and posting
here! I noticed myself and was coming to comment when I noticed Matthias,
Nick, Nicolás, and Senthil had beaten me to it. :) As Senthil said, it's
very interesting to see this option open up to us and will need to see how
they do. Hopefully it will work out (or at least we can help GitHub fix
their tooling if they can't handle the cpython repo's deep history).
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

2016-02-11 Thread Ethan Furman 

On 02/11/2016 07:07 PM, Brett Cannon wrote:
> On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez wrote:

>> It depends on how crazy you want to go. For example, SVN-era merges
>> don't appear as merges, but looks like some SVN-era branches don't
>> exist in Hg to begin with (Would I need to get cpython-fullhistory?
>> Cloning it gives me a 400 Bad Request). Do we care about that?

> If you are not an even clone it then that shows how much
> people who are.

Um, could you repeat that?  In English?  :)

--
~Ethan~
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct

Re: [core-workflow] Help needed: best way to convert hg repos to git?

2016-02-11 Thread Brett Cannon 
On Thu, Feb 11, 2016, 19:27 Ethan Furman  wrote:

> On 02/11/2016 07:07 PM, Brett Cannon wrote:
>  > On Thu, Feb 11, 2016, 16:43 Nicolás Alvarez wrote:
>
>  >> It depends on how crazy you want to go. For example, SVN-era merges
>  >> don't appear as merges, but looks like some SVN-era branches don't
>  >> exist in Hg to begin with (Would I need to get cpython-fullhistory?
>  >> Cloning it gives me a 400 Bad Request). Do we care about that?
>
>  > If you are not an even clone it then that shows how much
>  > people who are.
>
> Um, could you repeat that?  In English?  :)
>

If you aren't able to even clone it then that shows how much people care.

Brett


> --
> ~Ethan~
> ___
> core-workflow mailing list
> core-workflow@python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
___
core-workflow mailing list
core-workflow@python.org
https://mail.python.org/mailman/listinfo/core-workflow
This list is governed by the PSF Code of Conduct: 
https://www.python.org/psf/codeofconduct