>>>>> "r" == Ross <[EMAIL PROTECTED]> writes:
r> I don't remember anyone saying they couldn't be stored, just
r> that if they are stored it's not ZFS' fault if they go corrupt
r> as it's outside of its control.
they can't be stored because they ``go corrupt'' in a transactional
all-or-nothing way that other checksummed storable things like zpools,
tarballs, zipfiles, do not. The software's written with the
assumption it'll be used in a pipe, and I think fails appropriately
for that use. But it's utterly without grace if you accidentally
separate a stored 'zfs send' from your opportunity to re-send it.
r> I'm actually planning to store the zfs send dump on external
r> USB devices myself, but since the USB device will be running
r> on ZFS I expect to be able to find out if there are any
r> corruption problems,
yeah I still think it's not a good idea. Our best practices are based
on a feel for the statistics of bit-flips and their consequences
If you flip one bit:
in tar:
you get the file back, with the flipped bit, and a warning from tar.
in UFS:
you get the file back with a bit flipped.
in ZFS with redundancy:
you get the file back with the bit unflipped.
in ZFS without redundancy:
you are warned about the flipped bit by losing the whole file
in 'zfs send':
you lose the ENTIRE FILESYSTEM because of one bit.
'zfs recv' will do a great job of warning you about corruption
problems, whether it has ZFS underneath it or not. I don't think
anyone could mistake the signal it sends for affection or
negotiability. That's not the issue. It's the probability you'll be
able to successfully restore your backup <n> years from now.
We have a feel, not as good as we should but some, for this
probability given 'tar' on tape, DVDR, disk, stick. I think 'zfs
send' reduces this probability enough compared to other kinds of
backup that you'd be crazy to use it.
I also think this observation isn't adequately communicated to the
average sysadmin by the disclaimery-sounding IT warning ``*BRRK*
*BRRK* 'zfs send' is not a Backup Soloooshun!!'' It sounds like
someone wants you to spend more money on CYAware and not, as it
should, like they are begging you to use tar instead.
There are other problems besides fragileness with storing 'zfs send'
that are solved by both 'tar' and piping immediately to 'zfs recv':
* stream format has not been as forward-portable across zfs versions
and endiness as the pool structure itself. You could easily find
yourself with a stream that won't restore, with no user-readable
marker as to what it requires to restore itself.
The choices are unstable Nevada builds you may not even be able to
obtain two years from now much less get to boot on the hardware for
sale at the time, and machine endynesses(!)---much harder to
iteratively test than versions of GNU tar, if tar had this same
problem which it doesn't. It may even demand to be restored ONTO a
specific version of zpool, so your desperate restore attempt
strategy is to reinstall an older Nevada, destroy and recreate a
blank zpool, 'zfs recv', repeat.
It's a bad situation. Store a zpool instead and you won't be in it
because there's a commitment to forward-portability and
endyness-independence.
* no equivalent of 'zpool scrub' or 'tar t'. The only way to verify
a 'zfs send' dump is to restore it.
I think it's a best-practice when making backups to read back what
you wrote. This catches medium errors, broken network pipes,
filesystems you didn't quite sync before ejecting, whatever. You
don't need to do silly Windows things like compare the backup to
the files still on the disk and then scream OMG OMG something
Changed!, but you ought to at least read it back now if you expect
to restore it later.
Is something missing from the toolbox? hell yes! Even beyond these
inadequacies I'd like a way to restore a ZFS snapshot tree onto an
LVM2 or Netapp snapshot tree. so I rather think it'll be some
stream-storing rsync or deduplicating GNUtar instead of something
coming from the ZFS project. (AIUI GNUtar incrementals include whole
changed files, which is great for portability but useless for VM disk
images)
There are interesting questions like, would it drastically improve the
efficiency of incremental backups if ZFS exposed some interface to its
snapshot tree besides two copies of the file? Maybe the 'zfs send'
stream itself is sufficient exposure, or maybe it needs to be some
file-level rather than filesystem-level map. But this should probably
come after we have userspace tools that do the job correctly but
inefficiently given two copies of the file.
And how do we design a storable incremental format that will take
bit-flips gracefully, without seg-faulting and without killing the
entire snapshot tree because one block is lost forever?
Thirdly think it'd be good to write backups so one ``tape'' can be
lost. ex:
day 1: full backup
day 3: incremental based on day 1
day 5: incremental based on day 1
day 6: full backup
day 7: incremental based on day 1
day 8: incremental based on day 6
day 9: incremental based on day 1
day 10: incremental based on day 6
day 11: full backup
day 12: incremental based on day 6
day 13: incremental based on day 11
day 14: incremental based on day 6
day 15: incremental based on day 11
Maybe it's a bit quaint given how people do backups now, I don't know.
The point of it: you can combine backps on a single ``tape'' (stick)
if there's enough space, but you can never combine something written
on an odd day with something written on an even day.
if you insist on foolishly using 'zfs send' maybe a two-stick strategy
of that sort can boost your odds a bit. I still think they're not
good enough.
pgpjVmTpKuHYK.pgp
Description: PGP signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
