Re: Home storage with btrfs

[Luca Citi]

I have had some troubles recently using btrfs with kernels >=4.8 when 
using the cfq scheduler.

Please see my post: http://www.spinics.net/lists/linux-btrfs/msg63599.html
I have no problem with kernels < 4.8 or when using the deadline 
scheduler (or none).


It is possible that the problem I see is a result of a major redesign 
that happened in March 2016.


So, contrary to most people advising to use very recent kernels, I 
suggest to try and strike a balance between a kernel that is recent 
enough to have known bugs fixed but old enough that any recently 
introduced change has been around for a while and tested.


Regards,
Luca

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Home storage with btrfs

[Kai Krakow]

Am Wed, 15 Mar 2017 23:26:32 +0100
schrieb Kai Krakow :

> Well, bugs can hit you with every filesystem. Nothing as complex as a

Meh... I fooled myself. Find the mistake... ;-)

SPOILER:

"Nothing" should be "something".

-- 
Regards,
Kai

Replies to list-only preferred.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Home storage with btrfs

[Kai Krakow]

Am Wed, 15 Mar 2017 23:41:41 +0100
schrieb Kai Krakow :

> Am Wed, 15 Mar 2017 23:26:32 +0100
> schrieb Kai Krakow :
> 
> > Well, bugs can hit you with every filesystem. Nothing as complex as
> > a  
> 
> Meh... I fooled myself. Find the mistake... ;-)
> 
> SPOILER:
> 
> "Nothing" should be "something".

*doublefacepalm*

Please forget what I wrote. The original sentence is correct.

I should get some coffee or go to bed. :-\

-- 
Regards,
Kai

Replies to list-only preferred.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Home storage with btrfs

[Kai Krakow]

Am Wed, 15 Mar 2017 07:55:51 + (UTC)
schrieb Duncan <1i5t5.dun...@cox.net>:

> Hérikz Nawarro posted on Mon, 13 Mar 2017 08:29:32 -0300 as excerpted:
> 
> > Today is safe to use btrfs for home storage? No raid, just secure
> > storage for some files and create snapshots from it.  
> 
> 
> I'll echo the others... but with emphasis on a few caveats the others 
> mentioned but didn't give the emphasis I thought they deserved:
> 
> 1) Btrfs is, as I repeatedly put it in post after post, "stabilizing,
> but not yet fully stable and mature."  In general, that means it's
> likely to work quite or even very well for you (as it has done for
> us) if you don't try the too unusual or get too cocky, but get too
> close to the edge and you just might find yourself over that edge.
> Don't worry too much, tho, those edges are clearly marked if you're
> paying attention, and just by asking here, you're already paying way
> more attention than too many we see here... /after/ they've found
> themselves over the edge.  That's a _very_ good sign. =:^)

Well, bugs can hit you with every filesystem. Nothing as complex as a
file system can ever be proven bug free (except FAT maybe). But as a
general-purpose-no-fancy-features-needed FS, btrfs should be on par
with other FS these days.

> 2) "Stabilizing, not fully stable and mature", means even more than
> ever, if you value your data more than the time, hassle and resources
> necessary to have backups, you HAVE them, tested and available for
> practical use should it be necessary.

This is totally not dependent on "stabilizing, not fully stable and
mature". If you data matters to you, do backups. It's that simple. If
you don't do backups, your data isn't important - by definition.

> Of course any sysadmin (and that's what you are for at least your own 
> systems if you're making this choice) worth the name will tell you
> the value of the data is really defined by the number of backups it
> has, not by any arbitrary claims to value absent those backups.  No
> backups, you simply didn't value the data enough to have them,
> whatever claims of value you might otherwise try to make.  Backups,
> you /did/ value the data.

Yes. :-)

> And of course the corollary to that first sysadmin's rule of backups
> is that an untested as restorable backup isn't yet a backup, only a 
> potential backup, because the job isn't finished and it can't be
> properly called a backup until you know you can restore from it if
> necessary.

Even more true. :-)

> And lest anyone get the wrong idea, a snapshot is /not/ a backup for 
> purposes of the above rules.  It's on the same filesystem and
> hardware media and if that goes down... you've lost it just the
> same.  And since that filesystem is still stabilizing, you really
> must be even more prepared for it to go down, even if the chances are
> still quite good it won't.

A good backup should follow the 3-2-1 rule: Have 3 different backup
copies, 2 different media, and store at least 1 copy external/off-site.

For customers, we usually deploy a strategy like this for Windows
machines: Do one local backup using Windows Image Backup to a local
NAS to backup from inside the VM, use a different software to do image
backups from outside of the VM to the local NAS, mirror the "outside
image" to a remote location (cloud storage). And keep some backup
history. Overwriting the one existing backup with a new one won't help
you anything. All involved software should be able to do efficient
delta backups, otherwise mirroring offsite may be no fun.

In linux, I'm using borgbackup and rsync to have something similar.
Using borgbackup to a local storage, and syncing it offsite with rsync
gives me the 2-1 rule part. You can get the third rule by using rsync
to also mirror the local FS off the machine. But that's usually
overkill for personal backups. Instead, I only have a third copy of
most valuable data like photos, dev stuff, documents, etc.

BTW: For me, different media also means different FS types. So a bug in
one FS wouldn't easily hit the other.

[snip]

> 4) Keep the number of snapshots per subvolume under tight control as 
> already suggested.  A few hundred, NOT a few thousand.  Easy enough
> if you do those snapshots manually, but easy enough to get thousands
> if you're not paying attention to thin out the old ones and using an 
> automated tool such as snapper.

Borgbackup is so fast and storage efficient that you could run it easily
multiple times per day. That in turn means I don't need to rely on
regular snapshots to undo mistakes. I only use snapshots before doing
some knowingly risky stuff to have fast recovery. But that's all,
nothing else should snapshots before (except you are doing more
advanced stuff like container cloning, VM instance spawning, ...).

Re: Home storage with btrfs

[Duncan]

Hérikz Nawarro posted on Mon, 13 Mar 2017 08:29:32 -0300 as excerpted:

> Today is safe to use btrfs for home storage? No raid, just secure
> storage for some files and create snapshots from it.


I'll echo the others... but with emphasis on a few caveats the others 
mentioned but didn't give the emphasis I thought they deserved:

1) Btrfs is, as I repeatedly put it in post after post, "stabilizing, but 
not yet fully stable and mature."  In general, that means it's likely to 
work quite or even very well for you (as it has done for us) if you don't 
try the too unusual or get too cocky, but get too close to the edge and 
you just might find yourself over that edge.  Don't worry too much, tho, 
those edges are clearly marked if you're paying attention, and just by 
asking here, you're already paying way more attention than too many we 
see here... /after/ they've found themselves over the edge.  That's a 
_very_ good sign. =:^)

2) "Stabilizing, not fully stable and mature", means even more than ever, 
if you value your data more than the time, hassle and resources necessary 
to have backups, you HAVE them, tested and available for practical use 
should it be necessary.

Of course any sysadmin (and that's what you are for at least your own 
systems if you're making this choice) worth the name will tell you the 
value of the data is really defined by the number of backups it has, not 
by any arbitrary claims to value absent those backups.  No backups, you 
simply didn't value the data enough to have them, whatever claims of 
value you might otherwise try to make.  Backups, you /did/ value the data.

And of course the corollary to that first sysadmin's rule of backups is 
that an untested as restorable backup isn't yet a backup, only a 
potential backup, because the job isn't finished and it can't be properly 
called a backup until you know you can restore from it if necessary.

And lest anyone get the wrong idea, a snapshot is /not/ a backup for 
purposes of the above rules.  It's on the same filesystem and hardware 
media and if that goes down... you've lost it just the same.  And since 
that filesystem is still stabilizing, you really must be even more 
prepared for it to go down, even if the chances are still quite good it 
won't.

3) "Stabilizing, not fully stable and mature", also means that since the 
current best-practices code is still a moving target, you better be 
prepared to move with it.  The list-recommended kernels are the latest 
two releases of either the current or (mainline) LTS kernel series.  On 
the current track, 4.10 is out, so 4.10 and 4.9 are supported.  If you're 
still on 4.8 or earlier and can't point to a very specific known reason, 
you're behind.  On the LTS track, 4.9 is the latest LTS kernel as well, 
with 4.4 the one before that.  4.1's the one before that but that's a 
very long time ago in btrfs-development time, and while we'll generally 
still /try/ to help, honestly, our memory and thus our effectiveness at 
trying to help are going to be down dramatically from that of the 
recommended series.

If you prefer longer term "enterprise" or just Debian-stable distro 
support, fine, but honestly, the sort of stability they target doesn't 
have much in common with a still stabilizing btrfs, and the chances are 
/extremely/ high that either one or the other isn't a good match for your 
needs.   Either you want/need a more leading edge aka current distro 
which btrfs as still stabilizing fits in well with, or you want/need the 
stability of those longer term releases, and btrfs as still very actively 
stabilizing sticks out like a sore thumb and you're very likely to be 
better off on something that's actually considered stable, ext4 or xfs, 
perhaps, or my longer term stability favorite, reiserfs (which tends to 
be so stable in part because there's nobody screwing with it and messing 
things up, any more, reference the period when the mainline kernel devs 
switched the otherwise quite stable ext3 to the rather less stable 
data=writeback mode, for instance).

4) Keep the number of snapshots per subvolume under tight control as 
already suggested.  A few hundred, NOT a few thousand.  Easy enough if 
you do those snapshots manually, but easy enough to get thousands if 
you're not paying attention to thin out the old ones and using an 
automated tool such as snapper.

5) Stay away from quotas.  Either you need the feature and thus need a 
more mature filesystem where it's actually stable and does what it says 
on the label, or you don't, in which case you'll save yourself a /lot/ of 
headaches keeping them off.  Maybe someday...

6) Stay away from raid56 mode.  It has known problems ATM and is simply 
not ready.

FWIW, single-device and raid1 mode are the best tested and most reliable 
(within the single-device limitations for it, of course).  But even raid1 
mode has some caveats about rebuilding that it might be wise to 

Re: Home storage with btrfs

[waxhead]

Same here, Have been using BTRFS for a 'scratch' disk since about 2014.
The disk have had quite some abuse and no issues yet.
I don't use compression, snapshots or any fancy features.
I have recently moved all of the root filesystem to BTRFS with 5x SSD 
disks set up in RAID1 and everything is (still) working fine, and I have 
been shuffling large amounts of data on this volume. I bet the SSD's 
will break before BTRFS does, so the real test is yet to come I guess...
I am on Debian GNU/Linux with kernel 4.9.0-2-amd64 (Debian 4.9.13-1) - 
btrfs-progs 4.7.3


However, keep in mind that backups is winning the fight against binary 
related traumas :)


Peter Becker wrote:

I can confirm this. I have also no generell issues since the past 2
years with BTRFS in RAID1 and 6 Disks with different sizes and also no
issues with the DUP profile on a single disk.
Only some performance issues with deduplication and very large files.
But i also recommand to use a newer kernel (4.4 or higher) or better
the newest and build a newer version of btrfs progs form source.
I use Ubuntu 16.04 and kernel 4.9 + btrfs progs 4.9 currently.

2017-03-13 13:02 GMT+01:00 Austin S. Hemmelgarn <ahferro...@gmail.com>:

On 2017-03-13 07:52, Juan Orti Alcaine wrote:

2017-03-13 12:29 GMT+01:00 Hérikz Nawarro <herikz.nawa...@gmail.com>:

Hello everyone,

Today is safe to use btrfs for home storage? No raid, just secure
storage for some files and create snapshots from it.


In my humble opinion, yes. I'm running a RAID1 btrfs at home for 5
years and I feel the most serious bugs have been fixed, because in the
last two years I have not experienced any issue.

In general, I'd agree.  I've not seen any issues resulting from BTRFS itself
for the past 2.5 years (although it's helped me find quite a lot of marginal
or failing hardware over that time), but I've also not used many of the less
stable features (raid56, qgroups, and a handful of other things).

One piece of advice I will give though, try to keep the total number of
snapshots to a reasonably small three digit number (ideally less than 200,
absolutely less than 300), otherwise performance is going to be horrible.


Anyway, keeping your kernel and btrfs-progs updated is a must, and of
course, having good backups. I'm using Fedora and it's fine.

Also agreed, Fedora is one of the best options for a traditional distro
(they're very good about staying up to date and back-porting bug-fixes from
the upstream kernel).  The other two I'd recommend are Arch (they actually
use an almost upstream kernel and are generally the first distro to have new
versions of any arbitrary software) and Gentoo (similar to Arch, but more
maintenance intensive (although also more efficient (usually))).

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Home storage with btrfs

[Austin S. Hemmelgarn]

On 2017-03-13 07:52, Juan Orti Alcaine wrote:

2017-03-13 12:29 GMT+01:00 Hérikz Nawarro <herikz.nawa...@gmail.com>:

Hello everyone,

Today is safe to use btrfs for home storage? No raid, just secure
storage for some files and create snapshots from it.



In my humble opinion, yes. I'm running a RAID1 btrfs at home for 5
years and I feel the most serious bugs have been fixed, because in the
last two years I have not experienced any issue.
In general, I'd agree.  I've not seen any issues resulting from BTRFS 
itself for the past 2.5 years (although it's helped me find quite a lot 
of marginal or failing hardware over that time), but I've also not used 
many of the less stable features (raid56, qgroups, and a handful of 
other things).


One piece of advice I will give though, try to keep the total number of 
snapshots to a reasonably small three digit number (ideally less than 
200, absolutely less than 300), otherwise performance is going to be 
horrible.


Anyway, keeping your kernel and btrfs-progs updated is a must, and of
course, having good backups. I'm using Fedora and it's fine.
Also agreed, Fedora is one of the best options for a traditional distro 
(they're very good about staying up to date and back-porting bug-fixes 
from the upstream kernel).  The other two I'd recommend are Arch (they 
actually use an almost upstream kernel and are generally the first 
distro to have new versions of any arbitrary software) and Gentoo 
(similar to Arch, but more maintenance intensive (although also more 
efficient (usually))).

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Home storage with btrfs

[Juan Orti Alcaine]

2017-03-13 12:29 GMT+01:00 Hérikz Nawarro <herikz.nawa...@gmail.com>:
> Hello everyone,
>
> Today is safe to use btrfs for home storage? No raid, just secure
> storage for some files and create snapshots from it.
>

In my humble opinion, yes. I'm running a RAID1 btrfs at home for 5
years and I feel the most serious bugs have been fixed, because in the
last two years I have not experienced any issue.

Anyway, keeping your kernel and btrfs-progs updated is a must, and of
course, having good backups. I'm using Fedora and it's fine.

Kind regards.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Home storage with btrfs

[Hérikz Nawarro]

Hello everyone,

Today is safe to use btrfs for home storage? No raid, just secure
storage for some files and create snapshots from it.

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html