Re: [DNG] btrfs
...on Wed, Feb 21, 2018 at 09:08:15AM -0500, Hendrik Boom wrote: > Laast I heard here about btrfs is that it's recommended for use only bu > those who "know where the bodies are buried". I've been running btrfs for years now, both on distribution kernels and on newer ones I built myself. The more recent the kernel, the better though - there's been constant improvements (which sometimes also need updated versions of btrfs-tools). Some newer features limit backwards compatibility, and older kernels might throw warnings when trying to mount the volume. I'm not using a lot of features - snapshots, and mirroring mostly. Not had any problems with either, though replacing a disk (and having the system boot from a degraded mirror) has been somewhat fiddly. Most of the questions I had were answered by the btrfs wiki (https://btrfs.wiki.kernel.org/index.php/Main_Page). Alex. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] btrfs
Martin Steigerwald [2018-02-21 16:26]: > My short recommendation: Do not use BTRFS with Linux 3.10. Or more likely, don't use 3.10 at all. Support ended in November last year. -- Hilsen Harald ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] btrfs
Hello Hendrik. Hendrik Boom - 21.02.18, 15:08: > Now I have the fortune or misfortune (I don't know which yet) to have > a GnuBee 2, waiting in its box to be assembled. The online page > https://lwn.net/Articles/743609/ tells me that it works with kernel > "Linux 3.10.14 with lots of changes", and that many of the changes have > *not* made it into the mainline kernel. There is also a "4.4.87-based > kernel", reported as not reliable. > > So I'd really like to know how much of btrfs is reliable now, and how > much was reliable back in the days of the 3.10.14 kernel. My short recommendation: Do not use BTRFS with Linux 3.10. Use BTRFS with a lot more recent kernel. My personal recommendation is at least 4.6. Will it fail with 3.10? Probably not, probably yes. Especially in the area of free space management (the long time out of space issues) BTRFS developers implemented tons of significant improvements since 3.10. Thanks, -- Martin ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
[DNG] btrfs
Laast I heard here about btrfs is that it's recommended for use only bu those who "know where the bodies are buried". I am planning a migration of my files to a new server, and I'd ike to know where the bodies are buried. I surmise that this means the thing is very reliable if you only she certain features -- presumably its equivalents of RAID0 and RAID1. RAID1 is what I need. The advantage I would gain from btrfs over software RAID + ext4 is its checksumming. I plan to keep my filesystem safe from bitrot for a decade or two. (After that it will fall to my heirs to preserve or abandon the files. I will be beyond caring.) Now I have the fortune or misfortune (I don't know which yet) to have a GnuBee 2, waiting in its box to be assembled. The online page https://lwn.net/Articles/743609/ tells me that it works with kernel "Linux 3.10.14 with lots of changes", and that many of the changes have *not* made it into the mainline kernel. There is also a "4.4.87-based kernel", reported as not reliable. So I'd really like to know how much of btrfs is reliable now, and how much was reliable back in the days of the 3.10.14 kernel. (As yet, of course, I don't even know whether btrfs is in any of gnubee's kernels.) -- hendrik ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] btrfs
On Sat, Aug 13, 2016 at 02:26:13PM -0400, Hendrik Boom wrote:
> > For anything that can break your system, and for running unstable, btrfs is
> > awesome. You can make snapshots at any point (most people have at least a
> > daily cronjob), and then restore or mount live when you want. And when you
> > make it unbootable, you append subvol=backups/@2016-08-12 (or whatever
> > you named it) in grub, and there you go.
>
> i seem to remember that the btrfs developers still are developing it,
> and have sid it isn't really ready for a production release yet. I
> figured that might be dated information, butt then I read the headline,
> only a few days ago:
>
>Btrfs RAID 5/6 Code Completely Hosed
>
> ( https://soylentnews.org/article.pl?sid=16/08/06/2118237 )
>
> Maybe it's not time to use btrfs yet.
Heh, trusting Phoronix as a news source is not exactly the best idea :p
That's the worst tabloid in the Linux world.
Indeed, btrfs raid 5/6 code has some nasty bugs. That's natural for an
experimental feature (although it's somewhat embarassing it takes so long to
stabilize). If it happened in parts of the filesystem declared stable,
that'd be a reason to worry.
Phoronix picked an exaggerated post by a list member. You wouldn't fully
trust a random DNG regular either, would you? That person is not a btrfs
developer and has not a single commit in either the kernel nor btrfs-progs.
He's not a troll, though, and the concern is real: DON'T USE BTRFS RAID5 OR
6 FOR PRODUCTION!
On the other hand, most parts of btrfs are considered fit for use. Some
mainstream distributions like Suse already have it as default. I for one
use btrfs for a variety of tasks too: backups, vservers, a ceph cluster, an
ARM box doing Debian archive rebuilds, etc, for years.
There are two points to consider:
* btrfs's code is moving quite fast, thus less stable
* btrfs has a number of data safety features
Thus, you need to balance trusting the filesystem itself vs it protecting
you from PEBKAC, hardware failures and bugs in _other_ software.
Thus, it's a natural thing to hesitate before running it on stable.
Unstable, on the other hand, breaks regularly: any time your daily
dist-upgrade may give you broken X drivers, someone pulling runit-init from
under you, etc. That's why I recommend running btrfs on such systems for
its snapshot functionality.
So let's list some of such features:
* snapshots. Most people do it daily, or to clone a chroot, but if you want
to go extreme and snapshot every hour, go ahead!
* reflinks (CoW files/extents): copy a big file (like a VM image) and change
just a small part: it takes only as much space as your changes. Makes it
easy to keep many versions in case you make a mistake.
* deduplication: cram more backups on one disk. Can save your butt if you
realize you made a mistake months ago.
* data and metadata checksums. On single device, lets you know your data
went kaputt so you can restore from backups. On RAID, allows recovery
even if the disk claims the read went ok (in this case, md and hw raid
can't tell which copy is good, thus failing both).
* scrub: cheap test of validity of all extents on a mounted filesystem
* all writes are CoWed: on hardware failure, it's likely a recent earlier
generation of a file you've written to is still there. Been there, done
that, saved my ass.
* if you mount with flushoncommit (some versions have it as default, some
don't), all files on the entire filesystem are consistent in the event of
a crash. This is like data=journal on ext[34], except that there's only
a minor performance cost (unlike less than half of speed as on ext) and
the consistency guarantee also applies _between_ files (ie, no reorders).
* send/receive: O(delta) rsync. If your filesystem takes 30 minutes to stat
all files, btrfs send can do so instantly, knowing what has been changed.
Want to do hourly backups?
* likewise, O(delta) enumeration of new writes
* fancy RAID1/10 modes on differently sized disks
* mounting with -ocompress gets you more disk space, on uncompressible files
compression attempt is quickly aborted so you don't need to micromanage
this. Especially with -ocompress=lzo compression can speed up I/O if your
CPU is better than your disk.
On the other hand, downsides:
* stability: less mature code base
* the only such bug I've personally met on stable kernels was one occurence
of ENOSPC on a filesystem only ~80% full ("btrfs balance" to fix this)
* fragmentation: random writes to a file (like, a database, especially
sqlite as in Firefox's history db) on HDD can make the file ridiculously
slow. Mounting with -oautodedrag helps somewhat.
* when performance sucks, it sucks more than when it shines. Good:
unpacking a tarball, compression on slow disks. Bad: random writes.
Extreme good cases (compiling on eMMC on Odroid-U2) can be 2x ext4, extreme
bad cases (fsyncs in postgres on ext4 in a VM stored on btrfs) can
Re: [Dng] btrfs repair works fine, Lennart has no idea what he is talking about - was OT - It may be only one file, but it does point to the bigger problem!
On 02/22/2015 07:28 PM, Jim Murphy wrote: [...] Part of the discussion: btrfs checksumming theoretically allows you to transparently recover after media corruption if filesystem has redundancy (more than one copy of data). Journald checksum will probably detect corruption, but can it repair it? No it cannot. But btrfs checksumming cannot fix things for you either if you lose non-trivial amounts of data. It might be able to fix a few bits of errors, but not non-trivial amounts. I mean, that's a simple property of error correction codes: the more you want to be able to correct the longer must your checksum be. Neither btrfs' nor journald's are substantial enough to correct even a sector... Lennart This is pure ignorance. It does not require the redundancy provided by the CRC algorithm to recover the data; it uses the checksum just to find out if the copy is good, and uses redundancy provided by raid to repair it. (which is simply what Lennart's victim already said by adding context with if filesystem has redundancy and more than one copy of data, which is not the CRC). The checksum doesn't need to be longer to repair it, only to prevent collision. The chance of a collision is something like one in 2^32 = 4 billion. ( 1 in 512 :P) Test this out simply by making a raid1, filling it with data, then run 2 things in infinite loops. One to repeat scrubs, and one to write random data to the disks, not just a few bits. Here's 30 minutes of the test script (kernel 3.18.x, btrfs tools version 3.18.2): Konsole output Konsole output WARNING: errors detected during scrubbing, corrected. scrub status for af936534-6c3f-4136-809a-740a32a65591 scrub started at Fri Feb 27 15:07:34 2015 and finished after 159 seconds total bytes scrubbed: 13.20GiB with 120 errors error details: csum=120 corrected errors: 120, uncorrectable errors: 0, unverified errors: 0 scrub started on /mnt/test, fsid af936534-6c3f-4136-809a-740a32a65591 (pid=14152) WARNING: errors detected during scrubbing, corrected. scrub status for af936534-6c3f-4136-809a-740a32a65591 scrub started at Fri Feb 27 15:10:14 2015 and finished after 144 seconds total bytes scrubbed: 13.20GiB with 14 errors error details: csum=14 corrected errors: 14, uncorrectable errors: 0, unverified errors: 0 scrub started on /mnt/test, fsid af936534-6c3f-4136-809a-740a32a65591 (pid=14275) WARNING: errors detected during scrubbing, corrected. scrub status for af936534-6c3f-4136-809a-740a32a65591 scrub started at Fri Feb 27 15:12:44 2015 and finished after 139 seconds total bytes scrubbed: 13.20GiB with 80 errors error details: csum=80 corrected errors: 80, uncorrectable errors: 0, unverified errors: 0 scrub started on /mnt/test, fsid af936534-6c3f-4136-809a-740a32a65591 (pid=14377) WARNING: errors detected during scrubbing, corrected. scrub status for af936534-6c3f-4136-809a-740a32a65591 scrub started at Fri Feb 27 15:15:04 2015 and finished after 168 seconds total bytes scrubbed: 13.20GiB with 14 errors error details: csum=14 corrected errors: 14, uncorrectable errors: 0, unverified errors: 0 scrub started on /mnt/test, fsid af936534-6c3f-4136-809a-740a32a65591 (pid=14505) WARNING: errors detected during scrubbing, corrected. scrub status for af936534-6c3f-4136-809a-740a32a65591 scrub started at Fri Feb 27 15:17:54 2015 and finished after 163 seconds total bytes scrubbed: 13.20GiB with 110 errors error details: csum=110 corrected errors: 110, uncorrectable errors: 0, unverified errors: 0 scrub started on /mnt/test, fsid af936534-6c3f-4136-809a-740a32a65591 (pid=14595) WARNING: errors detected during scrubbing, corrected. scrub status for af936534-6c3f-4136-809a-740a32a65591 scrub started at Fri Feb 27 15:20:44 2015 and finished after 173 seconds total bytes scrubbed: 13.20GiB with 53 errors error details: csum=53 corrected errors: 53, uncorrectable errors: 0, unverified errors: 0 scrub started on /mnt/test, fsid af936534-6c3f-4136-809a-740a32a65591 (pid=14737) Obviously there is a chance for both copies to be destroyed at the same time... but it isn't all that likely in 20 minutes, even with such high destruction rate. But clearly this disproves Lennart's unfounded statement, saying a single sector cannot be repaired. Here's 391 blocks so far, which I assume is more than 391 sectors. Clearing cache and then doing a diff on the test files compared to the original copy shows that they are undamaged. (this means you can cp the files away without any loss, but maybe there are bugs that will make btrfs die later :P it's not exactly fully production ready) So change theoretically in the above email to in practice. And the test script: # variables used in many parts of the script disk1=/dev/data/btrfs1 disk2=/dev/data/btrfs2 testuser=peter #
Re: [Dng] btrfs repair works fine, Lennart has no idea what he is talking about - was OT - It may be only one file, but it does point to the bigger problem!
Quite frankly, I would not overly concern myself with Lennart's Poettering's opinion. He has been quoted: Open Source community is full of assholes, and I probably more than most others am one of their most favourite targets. That's probably true in many ways. I've seen plenty of them, however I believe it is his attitude that created the problem, not the software he has written. When I say that, it is not because I dislike him or that he is unintelligent. I've never met him, and every indicator says that he is far from stupid. My response is based on the fact that he has worked in a number of areas: pulseaudio, avahi, and systemd. Being something of a generalist, he is probably a master of none of them - and his attitude toward others is less than personable. This would include BTFS and any opinions thereupon. t.j. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
