On Wed, Aug 31, 2016 at 10:33 AM, Michael Mol <mike...@gmail.com> wrote: > On Wednesday, August 31, 2016 12:12:15 AM Volker Armin Hemmann wrote: >> Am 30.08.2016 um 23:59 schrieb Rich Freeman: >> > >> > That depends on the mode of operation. In journal=data I believe >> > everything gets written twice, which should make it fairly immune to >> > most forms of corruption. >> >> nope. Crash at the wrong time, data gone. FS hopefully sane. > > In data=journal mode, the contents of files pass through the journal as well, > ensuring that, at least as far as the filesystem's responsibility is > concerned, > the data will be intact in the event of a crash. >
Correct. As with any other sane filesystem if you're using data=journal mode with ext4 then your filesystem will always reflect the state of data and metadata on a transaction boundary. If you write something to disk and pull the power after fsck the disk will either contain the contents of your files before you did the write, or after the write was completed, and never anything in-between. This is barring silent corruption, which ext4 does not protect against. > Now, I can still think of ways you can lose data in data=journal mode: Agree, though all of those concerns apply to any filesystem. If you unplug an unmounted device, or pull the power when writes are pending, or never hit save, or whatever, then your data won't end up on disk. Now, a good filesystem should ensure that the data which is on disk is completely consistent. That is, you won't get half of a write, just all or nothing. >> >> If you want an fs that cares about your data: zfs. >> > >> > I won't argue that the COW filesystems have better data security >> > features. It will be nice when they're stable in the main kernel. >> >> it is not so much about cow, but integrity checks all the way from the >> moment the cpu spends some cycles on it. What COW does get you is the security of data=journal without the additional cost of writing it twice. Since data is not overwritten in place you ensure that on an fsck the system can either roll the write completely forwards or backwards. With data=ordered on ext4 there is always the risk of a half-overwritten file if you are overwriting in place. But I agree that many of the zfs/btrfs data integrity features could be implemented on a non-cow filesystem. Maybe ext5 will have some of them, though I'm not sure how much work is going into that vs just fixing btrfs, or begging Oracle to re-license zfs. >> Caught some silent file >> corruptions that way. Switched to ECC ram and never saw them again. > > In-memory corruption of a data is a universal hazard. ECC should be the norm, > not the exception, honestly. > Couldn't agree more here. The hardware vendors aren't helping here though, in their quest to try to make more money from those sensitive to such things. I believe Intel disables ECC on anything less than an i7. As I understand it most of the mainline AMD offerings support it (basically anything over $80 or so), but it isn't clear to me what motherboard support is required and the vendors almost never make a mention of it on anything reasonably-priced. If your RAM gets hosed than any filesystem is going to store bad data or metadata for a multitude of reasons. The typical x86+ arch wasn't designed to handle hardware failures around anything associated with cpu/ram. The ZFS folks tend to make a really big deal out of ECC, but as far as I'm aware it isn't any more important for ZFS than anything else. I think ZFS just tends to draw people really concerned with data integrity, and once you've controlled everything that happens after the data gets sent to the hard drive you tend to start thinking about what happens to it beforehand. I had to completely reinstall a windows system not long ago due to memory failure and drive corruption. Wasn't that big a deal since I don't keep anything on a windows box that isn't disposable, or backed up to something else. -- Rich
