Re: [BackupPC-users] How to move the Backuppc pool
Matthias writes: For the last 4 weeks, I have been doing everything I could to try to move our current backuppc pool (Pool is 79.18GB comprising 1132632 files and 4369 directories) from a RAID0 to a RAID5 having different filesystem sizes. The operating system is OpenBSD, so the filesystem is FFS, and I have tried so far: * tar * gtar (GNU version of tar) * pax * dump -- restore And none of them was able to successfully copy the pool. I suspect this has to do with the huge number of hardlinks within the pool, and requiring too much memory to catalog them all. dd is not an option as the filesystems cannot be the same size. Has anyone a clue on how to solve my problem? Thanks, and happy new year :-) As Les mentions, starting over is often the easiest approach. One other option is to use BackupPC_tarPCCopy in BackupPC 3.0.0beta3. You start by copying the pool itself (ie: TOPDIR/cpool), plus the config and log directories, using any copy method you prefer, without worrying about hardlinks. BackupPC_tarPCCopy is used to copy the pc subdirectory. It traverses each directory and generates a tar file containing hardlinks to the pool, or the full file if it can't find the file in the pool. You unpack this on the target file system and the hardlinks are recreated as the tar archive is extracted. It figures out what hardlink each file matches by applying the BackupPC pool checksum to each file. It's definitely not fast, but it does work without eating huge amounts of memory. It does cache inodes for each pc host, but you can turn that off with the -c option. Beyond memory usage, the other reason copying the pool is slow is that sequential files in a directory often have quite widely dispersed inodes (because of the effect of pooling of many backups), meaning there are a lot of disk seeks. Perhaps sorting by inodes will make processing each directory faster. Craig - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] How to move the Backuppc pool
Hello list, For the last 4 weeks, I have been doing everything I could to try to move our current backuppc pool (Pool is 79.18GB comprising 1132632 files and 4369 directories) from a RAID0 to a RAID5 having different filesystem sizes. The operating system is OpenBSD, so the filesystem is FFS, and I have tried so far: * tar * gtar (GNU version of tar) * pax * dump -- restore And none of them was able to successfully copy the pool. I suspect this has to do with the huge number of hardlinks within the pool, and requiring too much memory to catalog them all. dd is not an option as the filesystems cannot be the same size. Has anyone a clue on how to solve my problem? Thanks, and happy new year :-) Matthias - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] How to move the Backuppc pool
On Fri, Jan 05, 2007 at 03:45:43PM +0100, Matthias Bertschy wrote: For the last 4 weeks, I have been doing everything I could to try to move our current backuppc pool (Pool is 79.18GB comprising 1132632 files and 4369 directories) from a RAID0 to a RAID5 having different filesystem sizes. The operating system is OpenBSD, so the filesystem is FFS, and I have tried so far: * tar * gtar (GNU version of tar) * pax * dump -- restore And none of them was able to successfully copy the pool. I suspect this has to do with the huge number of hardlinks within the pool, and requiring too much memory to catalog them all. Yes, the hardlinks are difficult to cope with. dd is not an option as the filesystems cannot be the same size. Could you dd, then resize the file system? Tino. -- www.quantenfeuerwerk.de www.spiritualdesign-chemnitz.de www.lebensraum11.de - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] How to move the Backuppc pool
Tino Schwarze wrote: dd is not an option as the filesystems cannot be the same size. Could you dd, then resize the file system? Unfortunately no, because the RAID0 is huge ~1TB and we don't want such a size for the RAID5, mainly for performance reasons. Furthermore, downsizing a FFS partition is not supported on OpenBSD. Matthias - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] How to move the Backuppc pool
On Fri, 2007-01-05 at 15:45 +0100, Matthias Bertschy wrote: For the last 4 weeks, I have been doing everything I could to try to move our current backuppc pool (Pool is 79.18GB comprising 1132632 files and 4369 directories) from a RAID0 to a RAID5 having different filesystem sizes. The operating system is OpenBSD, so the filesystem is FFS, and I have tried so far: * tar * gtar (GNU version of tar) * pax * dump -- restore And none of them was able to successfully copy the pool. I suspect this has to do with the huge number of hardlinks within the pool, and requiring too much memory to catalog them all. dd is not an option as the filesystems cannot be the same size. Has anyone a clue on how to solve my problem? Thanks, and happy new year :-) The straightforward way is to just copy the configurations over and let the new system start from scratch, keeping the old drives around for some interval so you could restore a historical copy if required. In this case that's an especially good idea since you may see much worse performance on the RAID5 setup. -- Les Mikesell [EMAIL PROTECTED] - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] How to move the Backuppc pool
Les Mikesell a écrit : On Fri, 2007-01-05 at 15:45 +0100, Matthias Bertschy wrote: For the last 4 weeks, I have been doing everything I could to try to move our current backuppc pool (Pool is 79.18GB comprising 1132632 files and 4369 directories) from a RAID0 to a RAID5 having different filesystem sizes. The operating system is OpenBSD, so the filesystem is FFS, and I have tried so far: * tar * gtar (GNU version of tar) * pax * dump -- restore And none of them was able to successfully copy the pool. I suspect this has to do with the huge number of hardlinks within the pool, and requiring too much memory to catalog them all. dd is not an option as the filesystems cannot be the same size. Has anyone a clue on how to solve my problem? Thanks, and happy new year :-) The straightforward way is to just copy the configurations over and let the new system start from scratch, keeping the old drives around for some interval so you could restore a historical copy if required. In this case that's an especially good idea since you may see much worse performance on the RAID5 setup. Try afio, very efficient tool. Usage is similar to cpio. Yves -- Yves Trudeau, Ph. D., MCSE, OCP Analyste Senior Révolution Linux 819-780-8955 poste *104 Toutes les opinions et les prises de position exprimées dans ce courriel sont celles de son auteur et ne répresentent pas nécessairement celles de Révolution Linux Any views and opinions expressed in this email are solely those of the author and do not necessarily represent those of Revolution Linux - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] How to move the Backuppc pool
Matthias Bertschy escreveu: Hello list, For the last 4 weeks, I have been doing everything I could to try to move our current backuppc pool (Pool is 79.18GB comprising 1132632 files and 4369 directories) from a RAID0 to a RAID5 having different filesystem sizes. The operating system is OpenBSD, so the filesystem is FFS, and I have tried so far: * tar * gtar (GNU version of tar) * pax * dump -- restore And none of them was able to successfully copy the pool. I suspect this has to do with the huge number of hardlinks within the pool, and requiring too much memory to catalog them all. dd is not an option as the filesystems cannot be the same size. Has anyone a clue on how to solve my problem? Thanks, and happy new year :-) Matthias - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/ cp -a worked for me in a 20GB pool... - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
