Re: [BackupPC-users] backing up over WAN
On Wednesday 07 July 2010 09:32:25 Udo Rader wrote: If I understand BackupPC's pooling concept correctly, pooling takes place after the file has been transferred to the server and is compared to other files in the pool. No, a checksum is calculated for a file and then compared against the pool. If it doesn't exist in the pool, it is copied to the pc tree. After that is done, it is inserted into the pool by the linking process. Corrections, anyone? Regards, Tyler -- In a time of universal deceit, telling the truth is a revolutionary act. -- George Orwell -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] backing up over WAN
Udo Rader wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, we need to setup a reasonable backup solution for one of our customers, consisting of a HQ office and two branches. The most important requirement is off-site backup. Backup volume will be approximately 50 GB per office for a full backup, meaning 150GB in total. So far, we have used backuppc for almost any backup solution, but this time I am not 100% sure because of the WAN side of life. What I am wondering is if it was possible to retrieve files from the clients more intelligently: If I understand BackupPC's pooling concept correctly, pooling takes place after the file has been transferred to the server and is compared to other files in the pool. Yet what I have in mind would be don't transfer a file if we already have it in the pool, thus drastically reducing the amount of data transferred over the net. IIRC, the rsync protocol per se should just allow that, but I am unsure if BackupPC utilizes it at that level. So is this transfer minimization doable with BackupPC? Rsync compares only against the last full backup tree on the same host, transferring the differences. Once you get started it will work the way you want, but each new machine will transfer everything on the first run and if the same file is added on many machines, it will be copied separately from each, then merged in the pool. One thing you might do to to help get started would be to take the server on-site to each office for the first full run, then move to its permanent location. Or if you have the bandwidth, perhaps you can do the initial run over a weekend. -- Les Mikesell lesmikes...@gmail.com -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] backing up over WAN
Yes, this is what we do with 4 of our sites that are off our fiber ring. Although 2 have the bandwidth to complete the initial backup the other 2 don't. So we just take the backup server on site and let it get an initial full. As Les stated, after the initial full it will work the way you intend. Donny B. On 7/7/10, Les Mikesell lesmikes...@gmail.com wrote: Udo Rader wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, we need to setup a reasonable backup solution for one of our customers, consisting of a HQ office and two branches. The most important requirement is off-site backup. Backup volume will be approximately 50 GB per office for a full backup, meaning 150GB in total. So far, we have used backuppc for almost any backup solution, but this time I am not 100% sure because of the WAN side of life. What I am wondering is if it was possible to retrieve files from the clients more intelligently: If I understand BackupPC's pooling concept correctly, pooling takes place after the file has been transferred to the server and is compared to other files in the pool. Yet what I have in mind would be don't transfer a file if we already have it in the pool, thus drastically reducing the amount of data transferred over the net. IIRC, the rsync protocol per se should just allow that, but I am unsure if BackupPC utilizes it at that level. So is this transfer minimization doable with BackupPC? Rsync compares only against the last full backup tree on the same host, transferring the differences. Once you get started it will work the way you want, but each new machine will transfer everything on the first run and if the same file is added on many machines, it will be copied separately from each, then merged in the pool. One thing you might do to to help get started would be to take the server on-site to each office for the first full run, then move to its permanent location. Or if you have the bandwidth, perhaps you can do the initial run over a weekend. -- Les Mikesell lesmikes...@gmail.com -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Sent from my mobile device -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] backing up over WAN
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07.07.2010 16:57, Kameleon wrote: Yet what I have in mind would be don't transfer a file if we already have it in the pool, thus drastically reducing the amount of data transferred over the net. IIRC, the rsync protocol per se should just allow that, but I am unsure if BackupPC utilizes it at that level. So is this transfer minimization doable with BackupPC? Rsync compares only against the last full backup tree on the same host, transferring the differences. Once you get started it will work the way you want, but each new machine will transfer everything on the first run and if the same file is added on many machines, it will be copied separately from each, then merged in the pool. One thing you might do to to help get started would be to take the server on-site to each office for the first full run, then move to its permanent location. Or if you have the bandwidth, perhaps you can do the initial run over a weekend. Thanks to everyone for the clearance. While not 100% the thing I had in mind, I think I can live with the backup server travelling around a bit for the initial backups then. - -- Udo Rader | CEO BestSolution.at EDV Systemhaus GmbH | company Eduard-Bodem-Gasse 5 | address A-6020 Innsbruck | city +43 512 935834 | phone http://www.bestsolution.at | web -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mandriva - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMNKKRAAoJEJA9QoEqa9WByiMH/0MKpKJ3R7F8U11UNOZAwRuW LHklLDPthv5AwNPaId3jvkeIelXccc6ZIVQDJtxESrGZdnDXlEF9nsrf0e9PLuwi 9byC4xS1AL12cRVbTjniJQaGPlJUjfwaQKfTulAULVIjkp+CNET3TaCznua4Enky l6cfG3O6shgwDSI8aFMFNUkK9TtxI57R004G+fZbts5gYlFxpxLUTeWljMa0zt2d PSUQF3umzJ3M1Eqadcy0p8jWJ4IkJtXSgBSbq13lURvFMRESOekUV/M+mDYszApS 0C+YJLY/9ZtumaeyPwi/h+jSwgtxcX4RTl2voZ4a8IwOV8r6yaQ/p3r2tL+Ba8o= =6iwg -END PGP SIGNATURE- -- This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Backing up over WAN connection
On 01/27 10:08 , Sean Gleason wrote: I am currently running BackupPC to a number of remote offices and have been recently asked to add another office that has way more data than the bandwidth could support during the initial FULL what I ended up doing when I ran into this issue, was breaking the backup into parts (via rsync excludes), then gradually making the amount of data excluded smaller and smaller. It did end up taking months to go through the whole process tho; since they didn't want backups running during the day. This was to back up 80GB or so, moving it over a T1. more data, smaller pipes, or greater urgency will of course make this much less attractive. -- Carl Soderstrom Systems Administrator Real-Time Enterprises www.real-time.com --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnkkid=103432bid=230486dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
