Re: Moving Repositories to New server
Around about 26/07/11 15:35, Phil Pinkerton typed ... Questions : Switch vs relocate ? Effects of load into a new Subversion version ? That all sounds fine¹; you will need 'svn switch --relocate new-url' to maintain each existing working copy, as you surmised. The only thing you might have to watch out for above what you listed is that the new server has the same tool set that the old one does, so that the copied hook scripts still work. ¹ - However I have rarely done this myself :) -- [neil@fnx ~]# rm -f .signature [neil@fnx ~]# ls -l .signature ls: .signature: No such file or directory [neil@fnx ~]# exit
Re: Moving Repositories to New server
On Tue, Jul 26, 2011 at 10:35:02AM -0400, Phil Pinkerton wrote: Are there any know issues with regards to moving Repositories from one platform to another ? Will the old Repositories maintain their current SVN revision ? Current platform Sun Solaris 10: SVN 1.6.5 Target platform Red Hat Enterprise 5 SVN 1.6.17 ( Subversion Edge ) Planned steps (Creating a script for the dumps and loads as there as a few hundred Repositories to move.) (1) Freeze the repository . Take a dump of the repository. (2) Verify the file is dumped correctly by making sure that the last version dumped is the same as the one in the live repository. Also check the return code of the svnadmin dump command. (3) Copy the dump file over to the new server. Verify that the file is copied over correctly. (4) Load the dump (5) Verify the load. If your repositories are in FSFS format (check the file db/fs-type) you don't need to perform a dump/load cycle. It doesn't hurt, but it can take longer than copying the repositories directly. Run 'svnadmin verify' on all repositories before you start moving them. If all is good, disable commit, and run 'svnadmin verify' again for revisions you didn't have in the previous run (see the -r option). Then use 'svnadmin hotcopy' to copy the repository to a disk, transfer it to the new server, run 'svnadmin verify' on it again, and enable commit at the new location. (6) Migrate and hook scripts or authorization files over. 'svnadmin hotcopy' will copy all files in a repository (except versions prior to 1.6.11 which do not copy db/conf/fsfs.conf because of a bug). (7) Verify the scripts and configuration files work. (8) If you have a name for the server for accessing it. You might have to point the name to the new server. a. use switch ? b. or relocate ? You need relocate (on the 1.6.x command line, this is confusingly called 'svn switch --relocate') if the hostname or the access protocol (http://, svn:// etc.) of your new server is not the same as the old one.
Re: Moving Repositories to New server
On 26/07/11 15:35, Phil Pinkerton wrote: Are there any know issues with regards to moving Repositories from one platform to another ? Will the old Repositories maintain their current SVN revision ? Current platform Sun Solaris 10: SVN 1.6.5 Target platform Red Hat Enterprise 5 SVN 1.6.17 ( Subversion Edge ) Planned steps (Creating a script for the dumps and loads as there as a few hundred Repositories to move.) (1) Freeze the repository . Take a dump of the repository. (2) Verify the file is dumped correctly by making sure that the last version dumped is the same as the one in the live repository. Also check the return code of the svnadmin dump command. (3) Copy the dump file over to the new server. Verify that the file is copied over correctly. (4) Load the dump (5) Verify the load. (6) Migrate and hook scripts or authorization files over. (7) Verify the scripts and configuration files work. (8) If you have a name for the server for accessing it. You might have to point the name to the new server. a. use switch ? b. or relocate ? (9) Unfreeze the repository. Questions : Switch vs relocate ? Effects of load into a new Subversion version ? To minimise down-time I would suggest looking into using svnsync to create the new repository, I think something on the following lines should work 1. create the new repository and sync it with the master one 2. copy over the hooks and scripts 3. inform all users that the old repository is not available any longer and any commit will not work and possibly be lost 4. lock down the master repository. This may just be stopping Apache serving it, nothing drastic, like removing it, is neceserry yet. Besides, it's safer to keep it just in case 5. set the new repository as master. This involves deleting some revision properties on revision 0 that Subversion uses to sync the repository. As the new repository will not be synced any more these are no more necessary. Moreover, they will prevent the repository to work as a normal one rather than a synchronised copy. 6. inform all users to use svn switch --relocate to point their WCs to the new repository As you can see the old repository comes down only at step 4, and steps 5 takes only few seconds. I used this set-up for my DR solution. I had never had the chance to use it, but I did some testing and it seemed to work. Giulio
Re: Moving Repositories to New server
On 7/26/2011 8:08 AM, Stefan Sperling wrote: On Tue, Jul 26, 2011 at 10:35:02AM -0400, Phil Pinkerton wrote: Are there any know issues with regards to moving Repositories from one platform to another ? Will the old Repositories maintain their current SVN revision ? Current platform Sun Solaris 10: SVN 1.6.5 Target platform Red Hat Enterprise 5 SVN 1.6.17 ( Subversion Edge ) snip If your repositories are in FSFS format (check the file db/fs-type) you don't need to perform a dump/load cycle. It doesn't hurt, but it can take longer than copying the repositories directly. If the processor architectures differ, copying the repositories directly won't work unless changes to the repository format have been made recently. I had a problem when copying a repository from a 64-bit x86 machine to a 32-bit x86 machine, for example. (I think this was in 1.4.x, but it was years ago and I don't remember the details.) Solaris 10 suggests a SPARC machine as the source. Because of the byte ordering difference, I'd expect major trouble when copying a repository directly from a SPARC machine to an x86 machine. I've used (and recommend) the svnsync approach recommended by another poster. Downtime can be even less than when copying repository directories directly. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
Re: Moving Repositories to New server
On Tue, Jul 26, 2011 at 08:35:31AM -0700, David Chapman wrote: If the processor architectures differ, copying the repositories directly won't work unless changes to the repository format have been made recently. I had a problem when copying a repository from a 64-bit x86 machine to a 32-bit x86 machine, for example. (I think this was in 1.4.x, but it was years ago and I don't remember the details.) Solaris 10 suggests a SPARC machine as the source. Because of the byte ordering difference, I'd expect major trouble when copying a repository directly from a SPARC machine to an x86 machine. This only applies to repositories using the BDB backend. There is no such problem with the FSFS backend because it uses flat files. I've used (and recommend) the svnsync approach recommended by another poster. Downtime can be even less than when copying repository directories directly. Yes, it's another good alternative. But don't forget to run 'svnadmin verify' on the synced copies, just to be safe.
Re: Moving Repositories to New server
On 7/26/2011 10:44 AM, Stefan Sperling wrote: On Tue, Jul 26, 2011 at 08:35:31AM -0700, David Chapman wrote: If the processor architectures differ, copying the repositories directly won't work unless changes to the repository format have been made recently. I had a problem when copying a repository from a 64-bit x86 machine to a 32-bit x86 machine, for example. (I think this was in 1.4.x, but it was years ago and I don't remember the details.) Solaris 10 suggests a SPARC machine as the source. Because of the byte ordering difference, I'd expect major trouble when copying a repository directly from a SPARC machine to an x86 machine. This only applies to repositories using the BDB backend. There is no such problem with the FSFS backend because it uses flat files. I've used (and recommend) the svnsync approach recommended by another poster. Downtime can be even less than when copying repository directories directly. Yes, it's another good alternative. But don't forget to run 'svnadmin verify' on the synced copies, just to be safe. In a situation where copying directly will work, you can use 'rsync -a' to copy the live repositories and do the bulk of the work with the old system still active. Then when ready to cut over, stop svnserve/http access and repeat the command with the --delete option. That will complete quickly because it only has to duplicate the changes since the previous run. -- Les Mikesell lesmikes...@gmail.com
Re: Moving Repositories to New server
On Tue, Jul 26, 2011 at 09:22:04AM -0700, David Chapman wrote: On 7/26/2011 8:44 AM, Stefan Sperling wrote: On Tue, Jul 26, 2011 at 08:35:31AM -0700, David Chapman wrote: If the processor architectures differ, copying the repositories directly won't work unless changes to the repository format have been made recently. I had a problem when copying a repository from a 64-bit x86 machine to a 32-bit x86 machine, for example. (I think this was in 1.4.x, but it was years ago and I don't remember the details.) Solaris 10 suggests a SPARC machine as the source. Because of the byte ordering difference, I'd expect major trouble when copying a repository directly from a SPARC machine to an x86 machine. This only applies to repositories using the BDB backend. There is no such problem with the FSFS backend because it uses flat files. The bad copy was a FSFS repository. IIRC, the problem was writing binary data (e.g. integers) into files. The 64-bit machine wrote 64-bit integers into some of the files, and the 32-bit machine got confused. I don't think there are any platform dependent bits in an FSFS revision file itself. It's much like you can copy, say, a PDF document or a jpg image via the network from one platform to another and have it work. Mounting a big-endian filesystem that was written on a sparc on an x86 box is of course a different story. It might work, or it might not. There is nothing an application can do to account for that though. It would be good to know what really went wrong in your case. Maybe something went wrong during the copy process? How was the repository transferred? Over the network (good idea), or via a disk that used a big-endian filesystem (possibly a bad idea if the x86 box has no support for reading it properly despite the byte ordering difference)? Unless FSFS writes no binary data into its files, or else it is careful to do so in a platform-independent matter, there will be trouble. Not being an official Subversion developer, I can't comment on the internals of the FSFS format, but I would be very surprised if its files were truly platform-independent because this would slow down repository operations. See the spec here if you want to know the details: https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/structure Some related links: https://svn.apache.org/repos/asf/subversion/trunk/notes/svndiff https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_base/notes/structure (BDB spec) https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_base/notes/fs-history Given that there is an official portable transfer file format (the dump file), I would expect the Subversion developers to use more-efficient non-portable code within the repository files. It is more about being able to switch between backends and to allow upgrades to versions that change the repository format in non-trivial ways. You can certainly try to copy one repository directly from one platform to the other, then run svnadmin verify. That *should* tell you if there was a problem. Yes, indeed. But I wouldn't trust anything worth money on an inter-platform repository file copy. I suppose it depends on how that copy is done.
Re: Moving Repositories to New server
Phil Pinkerton pcpinker...@gmail.com wrote on 07/26/2011 09:35:02 AM: Are there any know issues with regards to moving Repositories from one platform to another ? Will the old Repositories maintain their current SVN revision ? Current platform Sun Solaris 10: SVN 1.6.5 Target platform Red Hat Enterprise 5 SVN 1.6.17 ( Subversion Edge ) Planned steps (Creating a script for the dumps and loads as there as a few hundred Repositories to move.) (1) Freeze the repository . Take a dump of the repository. (2) Verify the file is dumped correctly by making sure that the last version dumped is the same as the one in the live repository. Also check the return code of the svnadmin dump command. (3) Copy the dump file over to the new server. Verify that the file is copied over correctly. (4) Load the dump (5) Verify the load. (6) Migrate and hook scripts or authorization files over. (7) Verify the scripts and configuration files work. (8) If you have a name for the server for accessing it. You might have to point the name to the new server. a. use switch ? b. or relocate ? (9) Unfreeze the repository. As others have stated there are multiple ways. Be careful if your users use file locking, since most of the options will ignore lock tokens. (Just as most of the options ignore hook scripts and authorization files.) I've gone from Red Hat 3 to Solaris 10 (both x64) by just rsyncing the fsfs filesystem. (It is best to ignore transactions and sync the current file first if you plan on doing the initial rsync with a live database.) I've gone from Win 2003 x86 to Win 2008 x64 by svnsync. I've gone from Solaris to Windows using dumpfiles. I've gone from Windows to Red Hat with svnsync. I've played with hot copies, but not used them for much. They all have worked. Dumpfiles are by far the slowest and will take considerable temporary space if you have repositories of any significant size. However, I would consider dumpfiles the safest since you essentially will have a complete separate portable copy of the whole repository. (minus locks, hooks, and auth stuff mentioned above.) svnsync can also be quite slow, but would probably require less downtime since you can sync and re-sync live repositories. (Just make sure you either have rev-prop changing disabled or have a way to identify already sync'd rev-prop changes.) Kevin R.
Re: Moving Repositories to New server
On 7/26/2011 9:48 AM, Stefan Sperling wrote: On Tue, Jul 26, 2011 at 09:22:04AM -0700, David Chapman wrote: On 7/26/2011 8:44 AM, Stefan Sperling wrote: On Tue, Jul 26, 2011 at 08:35:31AM -0700, David Chapman wrote: If the processor architectures differ, copying the repositories directly won't work unless changes to the repository format have been made recently. I had a problem when copying a repository from a 64-bit x86 machine to a 32-bit x86 machine, for example. (I think this was in 1.4.x, but it was years ago and I don't remember the details.) Solaris 10 suggests a SPARC machine as the source. Because of the byte ordering difference, I'd expect major trouble when copying a repository directly from a SPARC machine to an x86 machine. This only applies to repositories using the BDB backend. There is no such problem with the FSFS backend because it uses flat files. The bad copy was a FSFS repository. IIRC, the problem was writing binary data (e.g. integers) into files. The 64-bit machine wrote 64-bit integers into some of the files, and the 32-bit machine got confused. I don't think there are any platform dependent bits in an FSFS revision file itself. It's much like you can copy, say, a PDF document or a jpg image via the network from one platform to another and have it work. Mounting a big-endian filesystem that was written on a sparc on an x86 box is of course a different story. It might work, or it might not. There is nothing an application can do to account for that though. It would be good to know what really went wrong in your case. Maybe something went wrong during the copy process? How was the repository transferred? Over the network (good idea), or via a disk that used a big-endian filesystem (possibly a bad idea if the x86 box has no support for reading it properly despite the byte ordering difference)? The original copy was over a local area network, using tar cf - | ssh on the 64-bit machine and pushing the files to the 32-bit machine. My notes then say something to the effect of trouble - redone with dump and load cycle. This was in January 2007 with Subversion 1.3.0 on both ends. I never tried the directory copy again. I don't have better notes about what problems I found. The FSFS spec links are interesting and on first reading suggest portability, but I don't have enough time to study them in detail. I'd have to look at all of the specs for repository files to be sure. For now I back up my repositories nightly using hot copies and full dumps (about 2 GB total dump size, so still feasible). One of the repositories came from a remote hosting service via svnsync; we decided that local hosting was better. I don't use svnsync locally now because all but one of my machines are powered off at night, but it worked without any problems then. -- David Chapman dcchap...@acm.org Chapman Consulting -- San Jose, CA
