Hi,
Christian Völker wrote on 2011-07-12 08:43:51 +0200 [Re: [BackupPC-users] My
Solution for "Off-Site"]:
> On 12/07/2011 01:05, Holger Parplies wrote:
> >
> > so, you're saying that you don't trust your file system, but you trust LVM
> > to
> > keep 4 snapshots accurate for up to four weeks? I think I understand Les'
> > point (if he's making it) that a hardware-based "don't do anything" approach
> > is more reliable than a software-based "accumulate the information needed to
> > undo all my changes". But I also understand your point of "as long as it
> > works, it gives me three previous states to go back to".
> Take it as you like. I never said I don't trust my filesystem. At least
> you have to trust *something* or you'll end up in endless layers of
> security.
>
> We both have the possibility to roll-back to a point some weeks ago. If
> LVM doesn't work as expected *or* Less disks getting broken during the
> swap- it's just the same.
I think you missed my point. An offline disk is conceptually a passive element.
It doesn't need to do anything to stay consistent. In fact, as long as it
*doesn't* do anything, it stays consistent.
An LVM snapshot, on the other hand, is an active element. For each write
access to your volume, copy-on-write logic needs to correctly update your
snapshot. For an extended period of time. Under potentially high load.
Through power failures!? That might work fine, but it simply is *not* "just
the same" as a passive element. Additionally, your snapshots are not truely
independant. Les might drop one disk, but that has absolutely no effect on
the other two. Your three copies share both physical medium and software
system, and the older snapshots probably even depend on the newer ones,
meaning corruption to your latest snapshot (due to SW or HW error) probably
automatically corrupts the older ones, too.
As I said, as long as it works, it provides you with "just the same"
advantages as Les' offline disks - with the additional comfort of being easily
accessible ("online", so to say - with all dangers that involves). It simply
appears less robust to me.
Don't get me wrong - I'm not saying your solution has no value. It's a good
idea, and in any case it's your decision, which threats you want to or can
protect against.
> > I'm just wondering whether you're unmounting the pool FS before the
> > snapshot,
> > or if you're relying on it to be in a consistent state by itself. How much
> > testing have you done?
> You can perform tests multiple times- every time they are fine but in
> case of emergency something else fails you haven't thought of
> previously. Meaning: There's no point in testing if a not properly
> closed filesystem is able to recover as you can't forsee it in any case.
> I'm using ext3 with data=journal, so it should work fine even without
> proper unmounting.
Again, you are missing my point. You have several layers stacked which may
make things behave differently from a "normal" FS-on-physical-partition
scenario. Does a sync on a drbd device behave *exactly* like it does on a
physical partition? Can it? Does that depend on network speed/availability?
I'm not saying testing will ensure it works. I'm saying testing might uncover
that it does *not* work. Simply assuming that ext3 with data=journal behaves
over drbd and in conjunction with LVM snapshots in the same way as it does with
local storage seems dangerous.
Well, it's your data, so it's your choice. But other people reading this should
be made aware of the implicit assumptions they would be making.
> >> The only thing I have to evaluate is to have the proper size of the
> >> snapshot.
> [...]
> I have to estimate how much data changes on the volume for a weeks time,
> yes. Then I take a snapshot. And another week. So it's a one week
> estimate.
Do I understand it correctly that only the latest snapshot grows, while the
previous snapshot is made to be relative to the new snapshot? That would seem
to be the only sensible way to implement multiple snapshots in an efficient
way (else you'd have four copies on each write).
If so, you could lvreduce the (previous) snapshot to just above the space it
uses and then reserve all free space for the new snapshot. That would
effectively eliminate any issue, as long as you have enough space for all four
snapshots (and you could even dynamically keep more or less snapshots,
depending on how much space they use).
> And why should this be an issue? The secondary it a 2TB disk
> while the original is around 1TB. So the amount of data changing within
> a four weeks time frame can be 100%.
It's probably not important, but that is only correct if none of the changes
contained in one snapshot are changed again in another (which is obviously not
the case - think FS metadata). You have four (additional) states of your file
system available. If, just as an example, you changed the same 25% of your
file system each week, that would necessarily consume [slightly more than]
100% of your available space. Of course, you *could* say that this requires
100% of your storage capacity of "new data", so that would effectively be 100%
of change.
In any case, it is unlikely to be a problem with BackupPC, as data under
$TopDir is never changed (except for log files and backups files and such -
minor amounts of data). Files are created and removed again, when the
corresponding backups expire. (Well, yes, there's checksum caching, but that
is probably also a minor amount of data.)
So, if you've effectively got as much space for snapshots as the original
volume uses, I would agree that you should be fine. And if you *do* run low,
you can always remove the oldest snapshot ahead of time. If your newest
snapshot runs out of space, the older ones would probably be invalid anyway,
wouldn't they?
Regards,
Holger
------------------------------------------------------------------------------
Storage Efficiency Calculator
This modeling tool is based on patent-pending intellectual property that
has been used successfully in hundreds of IBM storage optimization engage-
ments, worldwide. Store less, Store more with what you own, Move data to
the right place. Try It Now! http://www.accelacomm.com/jaw/sfnl/114/51427378/
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
