[git-users] Large SVN to GIT migration
i'm in the process of coming up with a strategy to convert a very large project from SVN to GIT. i'm experimented with git svn clone but have some questions. how large is to large. current SVN repo 80K+ revisions. suffers from poor SCM practices current structure in SVN is using cascading hierarchy. Essentially each release branch becomes the trunk (not officially named as trunk) and then next release branches to start next development release branch and so on, Think of a stairway. Essentially current code sitting in Trunk is extremely old, relevant code is at the end of the branching staircase. My first issue i'm trying to sort out. Should i migrate the entire SVN repo into a staging GIT repo and then clean up the GIT repo before pushing to eventual network repo for all developers. Will it even clone at this size? Or i could clone only the latest release branch and start this as my Master in GIT. Questions with this approach are how do i keep it from walking the branches back through entire SVN repo? Only way I've seen so far is to specify SVN revision. Is there another approach i'm overlooking. Advantage here is smaller conversion but i'm loosing history or have to maintain a legacy SVN repo for historical. (maybe its not important?) any experience or suggestions with the above are appreciated. -- You received this message because you are subscribed to the Google Groups "Git for human beings" group. To unsubscribe from this group and stop receiving emails from it, send an email to git-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [git-users] Large SVN to GIT migration
Thanks Magnus, i will explore subgit, but i think i'm leaning towards simplifying my repo and trimming un-needed branches and history. Kevin On Thursday, November 3, 2016 at 1:18:53 AM UTC-7, Magnus Therning wrote: > > > Kevin Norton > writes: > > > i'm in the process of coming up with a strategy to convert a very large > > project from SVN to GIT. > > > > i'm experimented with git svn clone but have some questions. > > > > how large is to large. > > > > current SVN repo > > 80K+ revisions. > > suffers from poor SCM practices > > current structure in SVN is using cascading hierarchy. Essentially each > > release branch becomes the trunk (not officially named as trunk) and > then > > next release branches to start next development release branch and so > on, > > Think of a stairway. > > Essentially current code sitting in Trunk is extremely old, relevant > code > > is at the end of the branching staircase. > > > > My first issue i'm trying to sort out. > > Should i migrate the entire SVN repo into a staging GIT repo and then > clean > > up the GIT repo before pushing to eventual network repo for all > developers. > > Will it even clone at this size? > > Or i could clone only the latest release branch and start this as my > Master > > in GIT. > > Questions with this approach are how do i keep it from walking the > branches > > back through entire SVN repo? > > Only way I've seen so far is to specify SVN revision. Is there another > > approach i'm overlooking. > > Advantage here is smaller conversion but i'm loosing history or have to > > maintain a legacy SVN repo for historical. (maybe its not important?) > > > > any experience or suggestions with the above are appreciated. > > Just bumped into this too > https://github.com/svn-all-fast-export/svn2git. In particular this > passage sounds like it could be interesting: > > The svn2git repository gets you an application that will do the actual > conversion. The conversion exists of looping over each and every > commit in the subversion repository and matching the changes to a > ruleset after which the changes are applied to a certain path in a git > repo. > > I don't know, but maybe you can come up with rules that'll convert your > "staircase development" in SVN to a more common "single branch > development with release branches"? > > /M > > -- > Magnus Therning OpenPGP: 0x927912051716CE39 > email: mag...@therning.orgjabber: mag...@therning.org > > twitter: magthe http://therning.org/magnus > > We act as though comfort and luxury were the chief requirements of > life, when all that we need to make us happy is something to be > enthusiastic about. > — Albert Einstein > -- You received this message because you are subscribed to the Google Groups "Git for human beings" group. To unsubscribe from this group and stop receiving emails from it, send an email to git-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[git-users] Re: Large SVN to GIT migration
Thanks Nando, I have attempted clone of entire repo. Basically i believe it will complete most likely afer a few days of restarts after failures. But most likely i need to take this opportunity to trim my repo and only clone what is necessary. Old versions of code are most likely not valuable and this is a good opportunity to take a fresh start. My goal is to introduce better SCM practices, so the cleaner the repo, most likely the better chance of success. . On Thursday, November 3, 2016 at 12:11:34 AM UTC-7, Ferdinando Santacroce wrote: > > Hi Kevin. > > Il giorno mercoledì 2 novembre 2016 19:48:58 UTC+1, Kevin Norton ha > scritto: >> >> i'm in the process of coming up with a strategy to convert a very large >> project from SVN to GIT. >> >> i'm experimented with git svn clone but have some questions. >> >> how large is to large. >> > >> current SVN repo >> 80K+ revisions. >> suffers from poor SCM practices >> current structure in SVN is using cascading hierarchy. Essentially each >> release branch becomes the trunk (not officially named as trunk) and then >> next release branches to start next development release branch and so on, >> Think of a stairway. >> Essentially current code sitting in Trunk is extremely old, relevant code >> is at the end of the branching staircase. >> >> My first issue i'm trying to sort out. >> Should i migrate the entire SVN repo into a staging GIT repo and then >> clean up the GIT repo before pushing to eventual network repo for all >> developers. >> Will it even clone at this size? >> > > I converted some SVN repo before, but no so big stuff (about 1k revisions). > Have you tried to run the git svn clone command just to see what happens? > It will take hours (please use powerful machine and reliable network), but > I think there's no better way to figure out if it will work. > > >> Or i could clone only the latest release branch and start this as my >> Master in GIT. >> > > This can be a way to set a "cut-line" and start with a new repo and a > better versioning strategy and flow. > > >> Questions with this approach are how do i keep it from walking the >> branches back through entire SVN repo? >> Only way I've seen so far is to specify SVN revision. Is there another >> approach i'm overlooking. >> > > Take a look at --ignore-paths option (https://git-scm.com/docs/git-svn); > you can use Perl regex to restrict the field of your clone command. > > >> Advantage here is smaller conversion but i'm loosing history or have to >> maintain a legacy SVN repo for historical. (maybe its not important?) >> > > Try to do a little retrospective: how much times do you look for old > commits and branches in you actual SVN repo? > Do an year or later old branch has some value for you today? > Or it is only a matter of historical purpose? > I you realize you never look at oldest branches, then I think you can > freeze the actual SVN repo (or convert it to Git but leave it apart) and > then start with a new Git repo with the latest 2-3 branches only. > > >> any experience or suggestions with the above are appreciated. >> > > Software evolves, and long-living repos often are just an heavy load to > drag than a real value. > Starting with a new tool can also be a chance to review your flow a choose > a better strategy (see GiFlow, GitHub flow, GitLab flow, etc.). > If I were in your shoes, I will take this chance and start with a new > smaller repo wit only the latest 2 branches/versions of the software. > > My 2 cents :) > Nando > -- You received this message because you are subscribed to the Google Groups "Git for human beings" group. To unsubscribe from this group and stop receiving emails from it, send an email to git-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [git-users] Large SVN to GIT migration
Thanks Mark, All advice so far seems to trend towards taking opportunity to clean up repository. I'm doing a walk through Clone of latest Dev Branch into GIT. I want to see how to manage the Clone and then keep synchronized. And then evenutally push or clone into a central repository that Developers would connect to. Kevin On Thursday, November 3, 2016 at 3:58:12 PM UTC-7, Mark Waite wrote: > > > > On Thursday, November 3, 2016 at 10:31:08 AM UTC-6, Kevin Norton wrote: >> >> Thanks Magnus, >> >> i will explore subgit, but i think i'm leaning towards simplifying my >> repo and trimming un-needed branches and history. >> > > When we did a major transition from a previous source control system to > git, it was easiest and simplest to take a snapshot of the tip of the > relevant branches from the source repository, then place them into the new > git repository. Developers who want to see the history before the > transition referred to the earlier repository. Developers who want the > history after the transition referred to the git repository. > > I think that had several benefits: > >1. Don't mislead developers to think that the history in git is >authoritative for those commits before git was used. The authoritative >history is in the original repository >2. Don't clutter the new repository with history that precedes the >transition to git >3. Don't spend time trying to create and revise a translation system >when the translated result will not be authoritative > > I'm sure there are business conditions where that is not acceptable > (transition from a licensed, proprietary source control system which will > be unavailable after the transition), but it worked well for us, and it > wasn't too long before searches in the old repository dropped to almost > zero. > > > Mark Waite > > >> Kevin >> >> On Thursday, November 3, 2016 at 1:18:53 AM UTC-7, Magnus Therning wrote: >>> >>> >>> Kevin Norton writes: >>> >>> > i'm in the process of coming up with a strategy to convert a very >>> large >>> > project from SVN to GIT. >>> > >>> > i'm experimented with git svn clone but have some questions. >>> > >>> > how large is to large. >>> > >>> > current SVN repo >>> > 80K+ revisions. >>> > suffers from poor SCM practices >>> > current structure in SVN is using cascading hierarchy. Essentially >>> each >>> > release branch becomes the trunk (not officially named as trunk) and >>> then >>> > next release branches to start next development release branch and so >>> on, >>> > Think of a stairway. >>> > Essentially current code sitting in Trunk is extremely old, relevant >>> code >>> > is at the end of the branching staircase. >>> > >>> > My first issue i'm trying to sort out. >>> > Should i migrate the entire SVN repo into a staging GIT repo and then >>> clean >>> > up the GIT repo before pushing to eventual network repo for all >>> developers. >>> > Will it even clone at this size? >>> > Or i could clone only the latest release branch and start this as my >>> Master >>> > in GIT. >>> > Questions with this approach are how do i keep it from walking the >>> branches >>> > back through entire SVN repo? >>> > Only way I've seen so far is to specify SVN revision. Is there another >>> > approach i'm overlooking. >>> > Advantage here is smaller conversion but i'm loosing history or have >>> to >>> > maintain a legacy SVN repo for historical. (maybe its not important?) >>> > >>> > any experience or suggestions with the above are appreciated. >>> >>> Just bumped into this too >>> https://github.com/svn-all-fast-export/svn2git. In particular this >>> passage sounds like it could be interesting: >>> >>> The svn2git repository gets you an application that will do the actual >>> conversion. The conversion exists of looping over each and every >>> commit in the subversion repository and matching the changes to a >>> ruleset after which the changes are applied to a certain path in a git >>> repo. >>> >>> I don't know, but maybe you can come up with rules that'll convert your >>> "staircase development" in SVN to a more common "single branch >>> development with release branches"? >>> >>> /M >>> >>> -- >>> Magnus Therning OpenPGP: 0x927912051716CE39 >>> email: mag...@therning.org jabber: mag...@therning.org >>> twitter: magthe http://therning.org/magnus >>> >>> We act as though comfort and luxury were the chief requirements of >>> life, when all that we need to make us happy is something to be >>> enthusiastic about. >>> — Albert Einstein >>> >> -- You received this message because you are subscribed to the Google Groups "Git for human beings" group. To unsubscribe from this group and stop receiving emails from it, send an email to git-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[git-users] Re: Large SVN to GIT migration
I have been experiencing alot of buggy behavior while trying to clone with git-svn. for example things like "git svn clone malformed index error". i generally try many things until i finally get a successful clone. i was able to do a Clone from my SVN repo. I directly cloned my latest branch using an SVN revision to limit the amount of history that i'm pulling. I cloned the branch to my GIT trunk. I also created an authors file that seems to be working as expected. i'm wondering if i can specify a revision in the git config file so that i dont' have to remember to limit my Fetch command each time I run Fetch. And what is the correct way to update my GIT repo from SVN. I assume i can just run git svn fetch to keep updating. Kevin On Friday, November 4, 2016 at 8:29:32 AM UTC-7, Ferdinando Santacroce wrote: > > You're welcome. > So, if you decide to save only the latest branch, doing a convertion is > not so difficult. > If you need help, I can try to guide you in checking out the latest branch > as master, preserving authors and ignored file list. > Please let me know. > > Bye, > Nando :) > > Il giorno giovedì 3 novembre 2016 17:29:27 UTC+1, Kevin Norton ha scritto: >> >> Thanks Nando, >> >> I have attempted clone of entire repo. Basically i believe it will >> complete most likely afer a few days of restarts after failures. But most >> likely i need to take this opportunity to trim my repo and only clone what >> is necessary. >> >> Old versions of code are most likely not valuable and this is a good >> opportunity to take a fresh start. >> My goal is to introduce better SCM practices, so the cleaner the repo, >> most likely the better chance of success. >> . >> >> >> >> On Thursday, November 3, 2016 at 12:11:34 AM UTC-7, Ferdinando Santacroce >> wrote: >>> >>> Hi Kevin. >>> >>> Il giorno mercoledì 2 novembre 2016 19:48:58 UTC+1, Kevin Norton ha >>> scritto: >>>> >>>> i'm in the process of coming up with a strategy to convert a very large >>>> project from SVN to GIT. >>>> >>>> i'm experimented with git svn clone but have some questions. >>>> >>>> how large is to large. >>>> >>> >>>> current SVN repo >>>> 80K+ revisions. >>>> suffers from poor SCM practices >>>> current structure in SVN is using cascading hierarchy. Essentially each >>>> release branch becomes the trunk (not officially named as trunk) and then >>>> next release branches to start next development release branch and so on, >>>> Think of a stairway. >>>> Essentially current code sitting in Trunk is extremely old, relevant >>>> code is at the end of the branching staircase. >>>> >>>> My first issue i'm trying to sort out. >>>> Should i migrate the entire SVN repo into a staging GIT repo and then >>>> clean up the GIT repo before pushing to eventual network repo for all >>>> developers. >>>> Will it even clone at this size? >>>> >>> >>> I converted some SVN repo before, but no so big stuff (about 1k >>> revisions). >>> Have you tried to run the git svn clone command just to see what >>> happens? >>> It will take hours (please use powerful machine and reliable network), >>> but I think there's no better way to figure out if it will work. >>> >>> >>>> Or i could clone only the latest release branch and start this as my >>>> Master in GIT. >>>> >>> >>> This can be a way to set a "cut-line" and start with a new repo and a >>> better versioning strategy and flow. >>> >>> >>>> Questions with this approach are how do i keep it from walking the >>>> branches back through entire SVN repo? >>>> Only way I've seen so far is to specify SVN revision. Is there another >>>> approach i'm overlooking. >>>> >>> >>> Take a look at --ignore-paths option (https://git-scm.com/docs/git-svn); >>> you can use Perl regex to restrict the field of your clone command. >>> >>> >>>> Advantage here is smaller conversion but i'm loosing history or have to >>>> maintain a legacy SVN repo for historical. (maybe its not important?) >>>> >>> >>> Try to do a little retrospective: how much times do you look for old >>> commits and branches in you actual SVN rep
