Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
Hi Chad, As we discussed on IRC, my suggestion is the following: On 5 June 2013 16:49, Chad Horohoe choro...@wikimedia.org wrote: Everyone seems to be ok with doing this sooner rather than later, so how about we pencil in the date of June 14th (next Friday)? That gives us enough time to figure out the parts we don't know as well as get notifications out to the relevant places. Let's get the conversion working, but without allowing patches in Gerrit. This allows us to do the migration in a somewhat slower pace, updating e.g. the nightly generation and making it easier to check if the conversion works as expected. This also makes it easier to get feedback on the conversion, hopefully. As a timeline, I would suggest to have continuous updates from svn to git, roughly weekly updates to the conversion code and a transition period of maybe 6 weeks, starting next friday? I think it is realistic as a time frame to update most things that depend on svn. Things I think we need to 1) Exact repo structure. I think we've gotten a good discussion started on the wiki, but I'd like to see more input (and please, feel free to edit it mercilessly, this was all written based on my naïve understanding of how you guys work) I have added a new proposed structure on the wiki. Basically, I'd like to formally separate trunk and rewrite, and to rename them to something like 'compat' and 'core'. After discussing it with you and translatewiki people, I think having i18n as submodule is the easiest option. 3) What to do with submodules. A concise list of all the current externals will make this easier. I think we should not have the same method with external libraries svn:external'ed in. I'd be more interested in including a pip-style requirements.txt, with an optional 'external libraries' zip package. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
On Fri, Jun 7, 2013 at 4:29 PM, Merlijn van Deen valhall...@arctus.nlwrote: Hi Chad, As we discussed on IRC, my suggestion is the following: On 5 June 2013 16:49, Chad Horohoe choro...@wikimedia.org wrote: Everyone seems to be ok with doing this sooner rather than later, so how about we pencil in the date of June 14th (next Friday)? That gives us enough time to figure out the parts we don't know as well as get notifications out to the relevant places. Let's get the conversion working, but without allowing patches in Gerrit. This allows us to do the migration in a somewhat slower pace, updating e.g. the nightly generation and making it easier to check if the conversion works as expected. This also makes it easier to get feedback on the conversion, hopefully. As a timeline, I would suggest to have continuous updates from svn to git, roughly weekly updates to the conversion code and a transition period of maybe 6 weeks, starting next friday? I think it is realistic as a time frame to update most things that depend on svn. This sounds reasonable to me. Things I think we need to 1) Exact repo structure. I think we've gotten a good discussion started on the wiki, but I'd like to see more input (and please, feel free to edit it mercilessly, this was all written based on my naïve understanding of how you guys work) I have added a new proposed structure on the wiki. Basically, I'd like to formally separate trunk and rewrite, and to rename them to something like 'compat' and 'core'. After discussing it with you and translatewiki people, I think having i18n as submodule is the easiest option. This also looks good. 3) What to do with submodules. A concise list of all the current externals will make this easier. I think we should not have the same method with external libraries svn:external'ed in. I'd be more interested in including a pip-style requirements.txt, with an optional 'external libraries' zip package. Up to you guys. Depending on the size of the dependency and ease of packaging, some things can just be copy+pasted in (with licenses!) as needed. I think we're all in agreement on 2013 committers as being the initial group to have +2. I've updated the wiki to reflect this. -Chad ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
Everyone seems to be ok with doing this sooner rather than later, so how about we pencil in the date of June 14th (next Friday)? That gives us enough time to figure out the parts we don't know as well as get notifications out to the relevant places. Things I think we need to figure out: 1) Exact repo structure. I think we've gotten a good discussion started on the wiki, but I'd like to see more input (and please, feel free to edit it mercilessly, this was all written based on my naïve understanding of how you guys work) 2) Who gets initial permissions on the repo(s). I've seen suggestions for a select list, all committers in 2013...any of these is fine by me. Remember this is permission for +2'ing and merging, all registered users will be able to submit changes for review (just like everywhere else in Gerrit) 3) What to do with submodules. A concise list of all the current externals will make this easier. Thanks everyone! -Chad On Tue, Jun 4, 2013 at 8:30 AM, Amir Ladsgroup ladsgr...@gmail.com wrote: I agree with sooner is better, I will help in updating documentations and everything I can Tell me what I can do :) Best On 6/2/13, Chad Horohoe choro...@wikimedia.org wrote: On Sun, Jun 2, 2013 at 5:32 AM, legoktm legoktm.wikipe...@gmail.com wrote: One thing I didn't notice on the plan is that right now we have a lot of externals set up, some of which are also dependent on the Toolserver. Should we be converting these to git submodules? Or what? If they're available via Git too, then yes, we can switch them over to Git submodules once everything's migrated. For stuff coming from SVN, we'll have to find another way (but it's not impossible). Also, I added a comment on the page about giving all current commiters +2 since that wasn't explicitly stated, and drafted a list on the talk page of some things we should do/update after the migration (update docs, update nightlies source, etc) As I said on-wiki, that sounds like a reasonable list of people to start with for having +2 access on the repo. The group will be self-managing, so you guys can add new people to it as you see fit. The docs/nightlies stuff is more in your guys' court, but it's good to keep a list of such things, sure. -Chad -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
This is list of trunk branch externals: --'i18n' --'externals/spelling' --'externals/simplejson' --'externals/pycolorname' --'externals/opencv' On 6/5/13, Chad Horohoe choro...@wikimedia.org wrote: Everyone seems to be ok with doing this sooner rather than later, so how about we pencil in the date of June 14th (next Friday)? That gives us enough time to figure out the parts we don't know as well as get notifications out to the relevant places. Things I think we need to figure out: 1) Exact repo structure. I think we've gotten a good discussion started on the wiki, but I'd like to see more input (and please, feel free to edit it mercilessly, this was all written based on my naïve understanding of how you guys work) 2) Who gets initial permissions on the repo(s). I've seen suggestions for a select list, all committers in 2013...any of these is fine by me. Remember this is permission for +2'ing and merging, all registered users will be able to submit changes for review (just like everywhere else in Gerrit) 3) What to do with submodules. A concise list of all the current externals will make this easier. Thanks everyone! -Chad On Tue, Jun 4, 2013 at 8:30 AM, Amir Ladsgroup ladsgr...@gmail.com wrote: I agree with sooner is better, I will help in updating documentations and everything I can Tell me what I can do :) Best On 6/2/13, Chad Horohoe choro...@wikimedia.org wrote: On Sun, Jun 2, 2013 at 5:32 AM, legoktm legoktm.wikipe...@gmail.com wrote: One thing I didn't notice on the plan is that right now we have a lot of externals set up, some of which are also dependent on the Toolserver. Should we be converting these to git submodules? Or what? If they're available via Git too, then yes, we can switch them over to Git submodules once everything's migrated. For stuff coming from SVN, we'll have to find another way (but it's not impossible). Also, I added a comment on the page about giving all current commiters +2 since that wasn't explicitly stated, and drafted a list on the talk page of some things we should do/update after the migration (update docs, update nightlies source, etc) As I said on-wiki, that sounds like a reasonable list of people to start with for having +2 access on the repo. The group will be self-managing, so you guys can add new people to it as you see fit. The docs/nightlies stuff is more in your guys' court, but it's good to keep a list of such things, sure. -Chad -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
I agree about having an overlapping test period. Also, we've already agreed to move to Bugzilla, but it was decided that we would focus on moving to Gerrit first. Hazard-SJ From: Maarten Dammers maar...@mdammers.nl To: pywikipedia-l@lists.wikimedia.org Sent: Wednesday, June 5, 2013 3:38 PM Subject: Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed Hi Chad, Op 5-6-2013 16:49, Chad Horohoe schreef: Everyone seems to be ok with doing this sooner rather than later, so how about we pencil in the date of June 14th (next Friday)? Hold your horses please. I appreciate you putting effort in this, but I don't want to rush this. We should at least have: * Good plan * A timeline * An overlapping test period Also all our bugs are still at Sourceforge. If we're going to move, it would be good to move that to bugzilla so we have the integration. We should really think things through and not make hasty decisions. Maarten ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
On Thu, Jun 6, 2013 at 1:08 AM, Maarten Dammers maar...@mdammers.nl wrote: Hi Chad, Op 5-6-2013 16:49, Chad Horohoe schreef: Everyone seems to be ok with doing this sooner rather than later, so how about we pencil in the date of June 14th (next Friday)? Hold your horses please. I appreciate you putting effort in this, but I don't want to rush this. We should at least have: * Good plan * A timeline * An overlapping test period Also all our bugs are still at Sourceforge. If we're going to move, it would be good to move that to bugzilla so we have the integration. We should really think things through and not make hasty decisions. About migrating to bugzilla see this www.mediawiki.org/wiki/Manual:Pywikipediabot/Migrating_to_bugzillaI like to packing up these migrations but It seems other devs don't Maarten __**_ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.**org Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/**mailman/listinfo/pywikipedia-lhttps://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
I agree with sooner is better, I will help in updating documentations and everything I can Tell me what I can do :) Best On 6/2/13, Chad Horohoe choro...@wikimedia.org wrote: On Sun, Jun 2, 2013 at 5:32 AM, legoktm legoktm.wikipe...@gmail.com wrote: One thing I didn't notice on the plan is that right now we have a lot of externals set up, some of which are also dependent on the Toolserver. Should we be converting these to git submodules? Or what? If they're available via Git too, then yes, we can switch them over to Git submodules once everything's migrated. For stuff coming from SVN, we'll have to find another way (but it's not impossible). Also, I added a comment on the page about giving all current commiters +2 since that wasn't explicitly stated, and drafted a list on the talk page of some things we should do/update after the migration (update docs, update nightlies source, etc) As I said on-wiki, that sounds like a reasonable list of people to start with for having +2 access on the repo. The group will be self-managing, so you guys can add new people to it as you see fit. The docs/nightlies stuff is more in your guys' court, but it's good to keep a list of such things, sure. -Chad -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
One thing I didn't notice on the plan is that right now we have a lot of externals set up, some of which are also dependent on the Toolserver. Should we be converting these to git submodules? Or what? Also, I added a comment on the page about giving all current commiters +2 since that wasn't explicitly stated, and drafted a list on the talk page of some things we should do/update after the migration (update docs, update nightlies source, etc) -- Legoktm On Sat, Jun 1, 2013 at 5:10 AM, Chad Horohoe choro...@wikimedia.org wrote: On Wed, May 15, 2013 at 4:19 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi Chad, In December, I think the consensus was to move to Gerrit [1] - basically with the goal to keep it as close as possible to mediawiki, hopefully also easing contributions (to mw for pwb developers and vice versa). I have put some initial work into the migration (see https://github.com/pywikibot/svn2git ), but I have not had the time to smooth out the wrinkles. Will you be at the Hackathon next week? I think this would be a good topic to discuss there. Merlijn [1] http://lists.wikimedia.org/pipermail/pywikipedia-l/2012-December/007657.html On 15 May 2013 01:37, Chad Horohoe choro...@wikimedia.org wrote: Hi all, The time has finally come upon us--I'm finally moving forward with shutting down SVN and making it a read-only service. As Pywikipedia is the only consumer of SVN anymore, I wanted to reach out to the community to find out what everyone wants to do. As I see it, there's three courses of action that Pywikipedia can go in: 1) Move to Gerrit 2) Move to Git elsewhere (Github, Google Code, etc) 3) Move to some other SVN service I'm more than willing to help with any of these choices--the first two would involve a conversion of the history to Git, along with importing it to the destination of choice. Staying with SVN is also potentially possible, I'm more than happy to provide full SVN dumps if someone's wanting to setup that service elsewhere. What are people's thoughts? I've not come up with a firm date yet, but coming to consensus sooner rather than later would be nice. Thanks! So, this can be done easily, I'm just wondering when we should go ahead with things? It'll only take a few hours, so it's just a matter of picking a date that people are ok with. Also, if anyone has anything they want to add to the page on mw.org, I'd welcome the input. https://www.mediawiki.org/wiki/Git/Conversion/pywikipedia -Chad ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
On Sun, Jun 2, 2013 at 5:32 AM, legoktm legoktm.wikipe...@gmail.com wrote: One thing I didn't notice on the plan is that right now we have a lot of externals set up, some of which are also dependent on the Toolserver. Should we be converting these to git submodules? Or what? If they're available via Git too, then yes, we can switch them over to Git submodules once everything's migrated. For stuff coming from SVN, we'll have to find another way (but it's not impossible). Also, I added a comment on the page about giving all current commiters +2 since that wasn't explicitly stated, and drafted a list on the talk page of some things we should do/update after the migration (update docs, update nightlies source, etc) As I said on-wiki, that sounds like a reasonable list of people to start with for having +2 access on the repo. The group will be self-managing, so you guys can add new people to it as you see fit. The docs/nightlies stuff is more in your guys' court, but it's good to keep a list of such things, sure. -Chad ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
On Wed, May 15, 2013 at 4:19 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi Chad, In December, I think the consensus was to move to Gerrit [1] - basically with the goal to keep it as close as possible to mediawiki, hopefully also easing contributions (to mw for pwb developers and vice versa). I have put some initial work into the migration (see https://github.com/pywikibot/svn2git ), but I have not had the time to smooth out the wrinkles. Will you be at the Hackathon next week? I think this would be a good topic to discuss there. Merlijn [1] http://lists.wikimedia.org/pipermail/pywikipedia-l/2012-December/007657.html On 15 May 2013 01:37, Chad Horohoe choro...@wikimedia.org wrote: Hi all, The time has finally come upon us--I'm finally moving forward with shutting down SVN and making it a read-only service. As Pywikipedia is the only consumer of SVN anymore, I wanted to reach out to the community to find out what everyone wants to do. As I see it, there's three courses of action that Pywikipedia can go in: 1) Move to Gerrit 2) Move to Git elsewhere (Github, Google Code, etc) 3) Move to some other SVN service I'm more than willing to help with any of these choices--the first two would involve a conversion of the history to Git, along with importing it to the destination of choice. Staying with SVN is also potentially possible, I'm more than happy to provide full SVN dumps if someone's wanting to setup that service elsewhere. What are people's thoughts? I've not come up with a firm date yet, but coming to consensus sooner rather than later would be nice. Thanks! So, this can be done easily, I'm just wondering when we should go ahead with things? It'll only take a few hours, so it's just a matter of picking a date that people are ok with. Also, if anyone has anything they want to add to the page on mw.org, I'd welcome the input. https://www.mediawiki.org/wiki/Git/Conversion/pywikipedia -Chad ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
Looks fine by me, and the sooner, the better (or so I want to think :P). Hazard-SJ From: Chad Horohoe choro...@wikimedia.org To: Merlijn van Deen valhall...@arctus.nl Cc: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Sent: Saturday, June 1, 2013 7:10 AM Subject: Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed On Wed, May 15, 2013 at 4:19 AM, Merlijn van Deen valhall...@arctus.nl wrote: Hi Chad, In December, I think the consensus was to move to Gerrit [1] - basically with the goal to keep it as close as possible to mediawiki, hopefully also easing contributions (to mw for pwb developers and vice versa). I have put some initial work into the migration (see https://github.com/pywikibot/svn2git ), but I have not had the time to smooth out the wrinkles. Will you be at the Hackathon next week? I think this would be a good topic to discuss there. Merlijn [1] http://lists.wikimedia.org/pipermail/pywikipedia-l/2012-December/007657.html On 15 May 2013 01:37, Chad Horohoe choro...@wikimedia.org wrote: Hi all, The time has finally come upon us--I'm finally moving forward with shutting down SVN and making it a read-only service. As Pywikipedia is the only consumer of SVN anymore, I wanted to reach out to the community to find out what everyone wants to do. As I see it, there's three courses of action that Pywikipedia can go in: 1) Move to Gerrit 2) Move to Git elsewhere (Github, Google Code, etc) 3) Move to some other SVN service I'm more than willing to help with any of these choices--the first two would involve a conversion of the history to Git, along with importing it to the destination of choice. Staying with SVN is also potentially possible, I'm more than happy to provide full SVN dumps if someone's wanting to setup that service elsewhere. What are people's thoughts? I've not come up with a firm date yet, but coming to consensus sooner rather than later would be nice. Thanks! So, this can be done easily, I'm just wondering when we should go ahead with things? It'll only take a few hours, so it's just a matter of picking a date that people are ok with. Also, if anyone has anything they want to add to the page on mw.org, I'd welcome the input. https://www.mediawiki.org/wiki/Git/Conversion/pywikipedia -Chad ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
I'm guessing we could have pywikipedia/trunk and pywikipedia/rewrite in Gerrit. Hazard-SJ From: Chad Horohoe choro...@wikimedia.org To: Merlijn van Deen valhall...@arctus.nl Cc: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Sent: Wednesday, May 15, 2013 6:33 AM Subject: Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed On Wed, May 15, 2013 at 4:19 AM, Merlijn van Deen valhall...@arctus.nl wrote: Hi Chad, In December, I think the consensus was to move to Gerrit [1] - basically with the goal to keep it as close as possible to mediawiki, hopefully also easing contributions (to mw for pwb developers and vice versa). I have put some initial work into the migration (see https://github.com/pywikibot/svn2git ), but I have not had the time to smooth out the wrinkles. Ooh, you found my tool :) Cleaning up those rule files should be pretty trivial, then we can convert all of the wanted history pretty quickly. I've gone ahead and drafted a page on mw.org based on the rules you wrote along with a survey of the existing SVN layout. https://www.mediawiki.org/wiki/Git/Conversion/pywikipedia Will you be at the Hackathon next week? I think this would be a good topic to discuss there. I'm afraid I won't be making it this year. -Chad ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] SVN reaching end of life - migration plan needed
On Wed, May 15, 2013 at 4:19 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi Chad, In December, I think the consensus was to move to Gerrit [1] - basically with the goal to keep it as close as possible to mediawiki, hopefully also easing contributions (to mw for pwb developers and vice versa). I have put some initial work into the migration (see https://github.com/pywikibot/svn2git ), but I have not had the time to smooth out the wrinkles. Ooh, you found my tool :) Cleaning up those rule files should be pretty trivial, then we can convert all of the wanted history pretty quickly. I've gone ahead and drafted a page on mw.org based on the rules you wrote along with a survey of the existing SVN layout. https://www.mediawiki.org/wiki/Git/Conversion/pywikipedia Will you be at the Hackathon next week? I think this would be a good topic to discuss there. I'm afraid I won't be making it this year. -Chad ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
