upgrading server
We have an old subversion version 1.4.3 (r23084) running on Solaris. We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), possibly version 1.8.x or 1.9.x Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go through dump and load of the repository as described in various online post and documentations. The repository is quite large…guessing the size to be in the order of 20-40GB Before we start undertaking such tasks 1. Does anyone know if there are there any problem/gotcha in migrating the repository? 2. Does anyone know how long it would take to export the repository of this size? This will give us an estimate how long to schedule down time and cut off time. Thanks for any insight.
Re: upgrading server
On Tue, Jul 25, 2017 at 2:00 PM, Andy So wrote: > We have an old subversion *version 1.4.3 (r23084) *running on Solaris. > > We would like to upgrade to use new hardware on Linux based OS (CentOS > 6.9), possibly version 1.8.x or 1.9.x > > > > Our plan is to installed and configure the latest SVN on CentOS 6.9. Then > go through dump and load of the repository as described in various online > post and documentations. The repository is quite large…guessing the size > to be in the order of 20-40GB > > > > Before we start undertaking such tasks > > 1. Does anyone know if there are there any problem/gotcha in > migrating the repository? > > 2. Does anyone know how long it would take to export the repository > of this size? This will give us an estimate how long to schedule down time > and cut off time. > > > > Thanks for any insight. > Dump and load is a good idea because it lets the repository be rewritten using the newer code and repository format which will give you a smaller repository that will run a little faster. That said you do not HAVE to dump/load. You have other options: 1. Just move the repository folder to the new server. Perhaps using tar and then moving the archive. 2. Instead of using dump/load, use svnsync. This gives all the benefits of the dump/load but allows you to shrink your downtime to almost nothing. Just svnsync the repository to the new server. This will probably take a long time, but it does not matter since the original server can be running while it happens. At the time of your choosing, do a final svnsync, and then shutdown the old server and use the new one. 3. Do option 1 now and then do a dump/load or svnsync at some future time that is more convenient for downtime. It will probably run faster since it is on new and better hardware too. There are some gotchas no matter what: 1. Does the new server have a new hostname or do you intend to update DNS to point to new server? If you are not doing the latter, then all of your existing working copies and scripts have to be modified for the new server. Also any use of svn:externals property has to be modified. 2. With an old repository there is a good chance you will run into bugs in your data that cause svnsync or load to fail. There are workaround for different failures but be prepared to run into them and account for finding the solutions. SVN 1.8 and 1.9 have added various options to let you workaround some of the known bugs. A common problem is having svn: properties that are not UTF-8 encoded or do not have LF line endings. svnsync has this option to work around the encoding problem: --source-prop-encoding ARG : convert translatable properties from encoding ARG to UTF-8. If not specified, then properties are presumed to be encoded in UTF-8. And it automatically fixes the LF problems. svnadmin load does not have any options to fix the problems, but you can add the --bypass-prop-validation option to ignore them and just carry the problems into your new repository. -- Thanks Mark Phippard http://markphip.blogspot.com/
Re: upgrading server
On 7/25/2017 11:00 AM, Andy So wrote: We have an old subversion /version 1.4.3 (r23084) /running on Solaris. We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), possibly version 1.8.x or 1.9.x Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go through dump and load of the repository as described in various online post and documentations. The repository is quite large…guessing the size to be in the order of 20-40GB Before we start undertaking such tasks 1.Does anyone know if there are there any problem/gotcha in migrating the repository? 2.Does anyone know how long it would take to export the repository of this size? This will give us an estimate how long to schedule down time and cut off time. Thanks for any insight. I recommend CentOS 7.x; CentOS 6.x is nearing the end of its support lifetime. I also installed an SSD on my new machine - much faster than rotating media, and Subversion's "write once" philosophy (old revisions are essentially immutable) works well on an SSD. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA Software Development Done Right. www.chapman-consulting-sj.com
RE: upgrading server
>> Does anyone know how long it would take to export the repository of this >> size? This will give us an estimate how long to schedule down time and cut >> off time. Svnsync is the easy option. If you insist on doing a dump/load, then a) you can time a test run of a dump/load, and b) "svnadmin dump" allows you to specify revision ranges which means you can do incremental dumps. You can dump/load the bulk of the repo now and then during the migration, run another dump/load to catch any new commits. http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate The only caveat I can think of is if someone changes revision properties (e.g. the commit message) between the time of the initial dump and the migration. But you can track/prevent those with a hook script. (And is another reason why svnsync is preferred.)
Re: upgrading server
Don't forget that svnadmin dump/load and svnsync don't preserve hook scripts or other ancillary data that might be in your repository directory on the server, so if you have hook scripts, authorization rules or other customizations, carry those over to the new server manually. And of course any configuration files for httpd or svnserve.
Re: upgrading server
On Tue, Jul 25, 2017 at 2:00 PM, Andy So wrote: > We have an old subversion version 1.4.3 (r23084) running on Solaris. > > We would like to upgrade to use new hardware on Linux based OS (CentOS 6.9), > possibly version 1.8.x or 1.9.x If you're bumping Subversion release to 1.8.x or 1.9.x, I'd urge you to go to CentOS 7, which at least starts with Subverison 1.7, and get the latest RPM's from Wandisco. > Our plan is to installed and configure the latest SVN on CentOS 6.9. Then go > through dump and load of the repository as described in various online post > and documentations. The repository is quite large…guessing the size to be > in the order of 20-40GB See above. Installing the latest Subversion on CentOS 6 gets into some fairly awkward library dependencies, especially for the "serf" libraries which are notably out of date on CentOS 6. > Before we start undertaking such tasks > > 1. Does anyone know if there are there any problem/gotcha in migrating > the repository? > > 2. Does anyone know how long it would take to export the repository of > this size? This will give us an estimate how long to schedule down time and > cut off time. Definitely use svnsync, if possible, so you have no downtime and can write-lock the original repository and do a DNS based switchover. Rollback, if there are *any* mismatched writes to the new repository, means starting over with all working copies, which can be quite painful. > Thanks for any insight. You might consider less of an extreme version update. The more versions you update between, I've noted more likelihood of incompatible workflows such as the changes in the "svn:external" formats.
Re: upgrading server
On Tue, Jul 25, 2017 at 9:00 PM, Mark Phippard wrote: > On Tue, Jul 25, 2017 at 2:00 PM, Andy So wrote: >> >> We have an old subversion version 1.4.3 (r23084) running on Solaris. >> >> We would like to upgrade to use new hardware on Linux based OS (CentOS >> 6.9), possibly version 1.8.x or 1.9.x >> >> >> >> Our plan is to installed and configure the latest SVN on CentOS 6.9. Then >> go through dump and load of the repository as described in various online >> post and documentations. The repository is quite large…guessing the size to >> be in the order of 20-40GB >> >> >> >> Before we start undertaking such tasks >> >> 1. Does anyone know if there are there any problem/gotcha in >> migrating the repository? >> >> 2. Does anyone know how long it would take to export the repository >> of this size? This will give us an estimate how long to schedule down time >> and cut off time. >> >> >> >> Thanks for any insight. > > > > Dump and load is a good idea because it lets the repository be rewritten > using the newer code and repository format which will give you a smaller > repository that will run a little faster. That said you do not HAVE to > dump/load. You have other options: > > 1. Just move the repository folder to the new server. Perhaps using tar and > then moving the archive. > > 2. Instead of using dump/load, use svnsync. This gives all the benefits of > the dump/load but allows you to shrink your downtime to almost nothing. > Just svnsync the repository to the new server. This will probably take a > long time, but it does not matter since the original server can be running > while it happens. At the time of your choosing, do a final svnsync, and > then shutdown the old server and use the new one. > > 3. Do option 1 now and then do a dump/load or svnsync at some future time > that is more convenient for downtime. It will probably run faster since it > is on new and better hardware too. > > There are some gotchas no matter what: > > 1. Does the new server have a new hostname or do you intend to update DNS > to point to new server? If you are not doing the latter, then all of your > existing working copies and scripts have to be modified for the new server. > Also any use of svn:externals property has to be modified. > > 2. With an old repository there is a good chance you will run into bugs in > your data that cause svnsync or load to fail. There are workaround for > different failures but be prepared to run into them and account for finding > the solutions. SVN 1.8 and 1.9 have added various options to let you > workaround some of the known bugs. A common problem is having svn: > properties that are not UTF-8 encoded or do not have LF line endings. > svnsync has this option to work around the encoding problem: > > --source-prop-encoding ARG : convert translatable properties from encoding > ARG > to UTF-8. If not specified, then properties are > presumed to be encoded in UTF-8. > > And it automatically fixes the LF problems. > > svnadmin load does not have any options to fix the problems, but you can add > the --bypass-prop-validation option to ignore them and just carry the > problems into your new repository. Indeed, dump/load is a bit more limited than svnsync regarding normalization. Also, be careful: both dump/load and svnsync do not migrate your hook scripts, config files and locks (server side locks of the svn:needs-lock type). I've just extended the FAQ entry about dump/load with some of the known caveats, and explained a procedure to do dump/load with minimal downtime. See: http://subversion.apache.org/faq.html#dumpload -- Johan
svn merge issues after upgrading server from 1.4.3 to 1.6.6 - unexpected property changes (deleted svn:mergeinfo)
-All, This is my first time posting to this alias; I apologize in advance for any protocol mistakes, but we are in a serious situation and I need to get some guidance/help as soon as possible. I tried searching through the issues database but could not find anything that appeared to be the issue we are encountering, so I am posting this request. We recently upgraded our subversion server software and are having major problems with merging after moving the subversion server from 1.4.2 to 1.6.6. I am asking the user community for any help they can give us. The details of the prior and current software versions are detailed at the end of this email. The problem we are having is with 'svn merge' with a workspace that was checked out using the subversion 1.4.2 server: - cd - svn merge -r revision1:revision2 SOURCE_SVN_URL . This results in many unrelated files are having their properties change, and many unrelated files being included in the merge. It makes it very difficult to verify that the merge was successful when many unrelated files are included. A small example: An attempt to merge one change from sw1.0 to sw2.0 branch. A folder called Docs contains files that have not been changed. An svn diff produces the following: % cd /data/source/branches/sw2.0 % svn stat -q Docs/ M Docs/WDS-RIS-Blueprint.odt M Docs/WDS-RIS-Script-Usage.odt % svn diff Docs/ Property changes on: Docs/WDS-RIS-Blueprint.odt ___ Deleted: svn:mergeinfo Property changes on: Docs/WDS-RIS-Script-Usage.odt ___ Deleted: svn:mergeinfo A mix of subversion clients have been in use, including 1.4, 1.5 and 1.6 based clients. Those have not changed, and this behavior occurs even using a 1.6.9 client. Can anyone provide any guidance, clues, pointers, etc. to what we need to do to address this issue?? Thank you very much in advance, -GmG Prior to last Monday 22-Mar-2010, we had a system running the following: - RHEL-5.3 64-bit (Linux 2.6.18-128.1.6.el5 #1 SMP Tue Mar 24 12:05:57 EDT 2009 x86_64) - subversion 1.4.2 (r22196, Sep 1 2008) (subversion-1.4.2-4.el5, subversion-devel-1.4.2-4.el5, subversion-javahl-1.4.2-4.el5, subversion-perl-1.4.2-4.el5, subversion-ruby-1.4.2-4.el5) - openssl OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008 (mod_ssl-2.2.3-22.el5, openssl097a-0.9.7a-9.el5_2.1, openssl-0.9.8e-7.el5, openssl-devel-0.9.8e-7.el5) - httpd Apache/2.2.3 (Nov 12 2008) (httpd-2.2.3-22.el5, httpd-devel-2.2.3-22.el5, system-config-httpd-1.3.3.3-1.el5) - Berkeley DB 4.3.29 (db4-4.3.29-9.fc6, db4-devel-4.3.29-9.fc6) The subversion repository is format "3" We upgraded the system to: - RHEL-5.4 64-bit (Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 04:15:13 EDT 2009 x86_64) - subversion 1.6.6 (r40053, Oct 22 2009) (subversion-1.6.6-0.1.el5.rf, subversion-devel-1.6.6-0.1.el5.rf, subversion-perl-1.6.6-0.1.el5.rf) - openssl OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008 (Mod_ssl-2.2.3-31.0.1.el5, openssl-0.9.8e-12.el5_4.1, openssl-devel-0.9.8e-12.el5_4.1, openssl-perl-0.9.8e-12.el5_4.1) - httpd Apache/2.2.3 (Sep 3 2009) (httpd-2.2.3-31.0.1.el5, httpd-manual-2.2.3-31.0.1.el5, system-config-httpd-1.3.3.1-1.el5.0.1) - Berkeley DB 4.3.29 (db4-4.3.29-10.el5, db4-devel-4.3.29-10.el5) The subversion repository was not upgraded and is still in format "3".
RE: svn merge issues after upgrading server from 1.4.3 to 1.6.6 - unexpected property changes (deleted svn:mergeinfo)
> We recently upgraded our subversion server software and are having major > problems with merging after moving the subversion server from 1.4.2 to > 1.6.6. I am asking the user community for any help they can give us. > > The details of the prior and current software versions are detailed at the > end of this email. > > The problem we are having is with 'svn merge' with a workspace that was > checked out using the subversion 1.4.2 server: Eww... not sure how great an idea it is to use a pre 1.5 server with 1.5 and 1.6 clients. Although I'm not 100% sure of the ramifications... it could just be a performance issue and the client has to do more work to walk the tree of mergeinfo properties. Perhaps someone that knows a bit more about that issue will chime in here. > > - cd > - svn merge -r revision1:revision2 SOURCE_SVN_URL . > > This results in many unrelated files are having their properties change, > and many unrelated files being included in the merge. It makes it very > difficult to verify that the merge was successful when many unrelated > files are included. > > A small example: > > An attempt to merge one change from sw1.0 to sw2.0 branch. > A folder called Docs contains files that have not been changed. > An svn diff produces the following: > % cd /data/source/branches/sw2.0 > % svn stat -q Docs/ > M Docs/WDS-RIS-Blueprint.odt > M Docs/WDS-RIS-Script-Usage.odt > As you know a " M" (Space in first col M in second column) indicates that the properties where changed but not the file itself. > % svn diff Docs/ > > Property changes on: Docs/WDS-RIS-Blueprint.odt > ___ > Deleted: svn:mergeinfo > > > Property changes on: Docs/WDS-RIS-Script-Usage.odt > ___ > Deleted: svn:mergeinfo This shows that you do have mergeinfo on your files. So, at some time someone did a merge probably at the Docs level or directly to a file which added the mergeinfo. Now you are doing it at a higher folder level so the data is being elided. There's not really anything wrong here. You might want to train your devs to always do merges at the same folder level to prevent all the merge data on child folders and files. If they weren't there then they wouldn't be elided. > A mix of subversion clients have been in use, including 1.4, 1.5 and 1.6 > based clients. Those have not changed, and this behavior occurs even using > a 1.6.9 client. I assume it doesn't happen when you use a 1.4 client... since this version doesn't include the merge tracking functionality. > Can anyone provide any guidance, clues, pointers, etc. to what we need to > do to address this issue?? I think there is nothing you can do to address it. But, to solve your problem you may just want to pipe the output to grep (or something) and ignore files with a status of " M" You might also want to upgrade your server... 1.4.x is pretty old and not supported with updates. I hope this helps.. sorry AFAIK there is no magic answer to make this easier. BOb
Re: svn merge issues after upgrading server from 1.4.3 to 1.6.6 - unexpected property changes (deleted svn:mergeinfo)
-Hi Bob, First off, thank you very much for your quick reply. We were running a 1.4.2 subversion server, and upgraded the subversion server to 1.6.6. The clients have not changed in any way, and were a mix of 1.4, 1.5 and 1.6 clients that used "http" only to communicate with the subversion server. We have not had any compatibility issues of any kind until we upgraded the server to 1.6.6. The merge problem did not show up when the server was 1.4.2, but is now showing up with the 1.6.6 server. We did not upgrade the database repository - it is still at version "3". I read somewhere in the 1.5 release notes that the "merge tracking" added to 1.5 will not be enabled unless the repository is upgraded, which it has not. If I understand your reply correctly, what's happening is old svn:mergeinfo data that is no longer needed is being removed (property change only). Even though a large number of files are listed in the merge/commit, besides files that really were merged, the rest are just property changes. Thank you very much. -GmG On 03/25/10 12:04, Bob Archer wrote: We recently upgraded our subversion server software and are having major problems with merging after moving the subversion server from 1.4.2 to 1.6.6. I am asking the user community for any help they can give us. The details of the prior and current software versions are detailed at the end of this email. The problem we are having is with 'svn merge' with a workspace that was checked out using the subversion 1.4.2 server: Eww... not sure how great an idea it is to use a pre 1.5 server with 1.5 and 1.6 clients. Although I'm not 100% sure of the ramifications... it could just be a performance issue and the client has to do more work to walk the tree of mergeinfo properties. Perhaps someone that knows a bit more about that issue will chime in here. - cd - svn merge -r revision1:revision2 SOURCE_SVN_URL . This results in many unrelated files are having their properties change, and many unrelated files being included in the merge. It makes it very difficult to verify that the merge was successful when many unrelated files are included. A small example: An attempt to merge one change from sw1.0 to sw2.0 branch. A folder called Docs contains files that have not been changed. An svn diff produces the following: % cd /data/source/branches/sw2.0 % svn stat -q Docs/ M Docs/WDS-RIS-Blueprint.odt M Docs/WDS-RIS-Script-Usage.odt As you know a " M" (Space in first col M in second column) indicates that the properties where changed but not the file itself. % svn diff Docs/ Property changes on: Docs/WDS-RIS-Blueprint.odt ___ Deleted: svn:mergeinfo Property changes on: Docs/WDS-RIS-Script-Usage.odt ___ Deleted: svn:mergeinfo This shows that you do have mergeinfo on your files. So, at some time someone did a merge probably at the Docs level or directly to a file which added the mergeinfo. Now you are doing it at a higher folder level so the data is being elided. There's not really anything wrong here. You might want to train your devs to always do merges at the same folder level to prevent all the merge data on child folders and files. If they weren't there then they wouldn't be elided. A mix of subversion clients have been in use, including 1.4, 1.5 and 1.6 based clients. Those have not changed, and this behavior occurs even using a 1.6.9 client. I assume it doesn't happen when you use a 1.4 client... since this version doesn't include the merge tracking functionality. Can anyone provide any guidance, clues, pointers, etc. to what we need to do to address this issue?? I think there is nothing you can do to address it. But, to solve your problem you may just want to pipe the output to grep (or something) and ignore files with a status of " M" You might also want to upgrade your server... 1.4.x is pretty old and not supported with updates. I hope this helps.. sorry AFAIK there is no magic answer to make this easier. BOb