On Mar 1, 2013 3:42 PM, "Mark Campbell" <mcampb...@emediatrade.com> wrote:
>
> Lars,
>
> Thanks for the interesting idea! I confess I haven't played with ZFS
much (though I've been wanting to for some time), maybe this is the excuse
I need ;). Question, taking your model here, and applying it to my
situation, how well would this work:
>
> BackupPC server, with a RAID1 zpool, with the third member being my
external fireproof drive. Rather than the rotation you described, just
leave it as is as it does its daily routine. Then, should the day come
where I need to grab the drive and go, plugging the drive into a system
with ZFSonLinux & BackupPC installed, could I mount this drive by itself?
>
> I really like your idea of zfs send/receive for the remote copy. Do you
have any tips/pointers/docs on the best way to run it in this scenario?
>
> Thanks,
>
> --Mark
>
>
> -----Original Message-----
> From: Lars Tobias Skjong-Børsting [mailto:li...@snota.no]
> Sent: Friday, March 01, 2013 4:18 AM
> To: backuppc-users@lists.sourceforge.net
> Subject: Re: [BackupPC-users] BackupPC Pool synchronization?
>
> Hi,
>
> On 3/1/13 12:34 AM, Les Mikesell wrote:
> > On Thu, Feb 28, 2013 at 3:10 PM, Mark Campbell <
mcampb...@emediatrade.com> wrote:
> >
> >> So I'm trying to get a BackupPC pool synced on a daily basis from a
> >> 1TB MD
> >> RAID1 array to an external Fireproof drive (with plans to also sync
> >> to a remote server at our collo).
> >
> > I'm not sure anyone has come up with a really good way to do this.
> > One approach is to use a 3-member raid1 where you periodically remove
> > a drive and resync a new one. If you have reasonable remote
> > bandwidth and enough of a backup window, it is much easier to just run
> > another instance of backuppc hitting the same targets independently.
>
> I have come up with a IMHO good way to do this using ZFS (ZFSonLinux).
>
> Description:
> * uses 3 disks.
> * at all times, keep 1 mirrored disk in a fire safe.
> * periodically swap the safe disk with mirror in server.
>
> 1. create a zpool with three mirrored members.
> 2. create a filesystem on it and mount at /var/lib/backuppc.
> 3. do some backups.
> 4. detach one disk and put in safe.
> 5. do more backups.
> 6. detach one disk and swap with the other disk in the safe.
> 7. attach and online the disk from the safe.
> 8. watch it sync up.
>
> I am currently using 2TB disks, and swap period of 1 month. Because of
ZFS it doesn't need to sync all the blocks, but only the changed blocks
since 1 month ago. For example, with 10GB changed it will sync in less than
25 minutes (approx. 7 MB/s speed). That's a lot faster than anything I got
with mdraid which syncs every block.
>
> ZFS also comes with benefits of checksumming and error correction of file
content and file metadata. BackupPC also supports error correction through
par2, and this gives an extra layer of data protection.
>
> Backing up large numbers of files can take a very long time because of
harddisk seeking. This can be alleviated by using a SSD cache drive for
ZFS. This support for read (ZFS L2ARC) and write (ZFS ZIL) caching on a
small SSD (30 GB) cuts incremental time down to half for some shares.
>
> As for remote sync, you can use "zfs send" on the backup server and "zfs
receive" on the offsite server. This will only send the differences since
last sync (like rsync), and will be probably be significantly faster than
rsync that in addition has to resolve all the hardlinks.
>
> --
> Best regards,
> Lars Tobias
>
>
------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics Download AppDynamics Lite for
free today:
> http://p.sf.net/sfu/appdyn_d2d_feb
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
>
------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
+1 for ZFS as a means to replicate the pool without lots of rsyncing.
However the checksumming in ZFS only takes place on RAIDZ sets. ZFS
mirroring (RAID 1) does not do checksum verification. You would have to
use RAIDZ1 (RAID 5) , RAIDZ2 (RAID 6) or RAIDZ3 (triple parity) to benefit
from checksum verification.
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
