Re: ENOSPC with mkdir and rename

2014-08-10 Thread Martin Steigerwald
Am Montag, 4. August 2014, 11:09:23 schrieb Peter Waller: > On 4 August 2014 10:39, Chris Samuel wrote: > > On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: > >> All of this is *very* surprising. > > > > Hmm, it shouldn't be, the ENOSPC issues are well known and have been > > discussed here for

Re: ENOSPC with mkdir and rename

2014-08-10 Thread Martin Steigerwald
Am Dienstag, 5. August 2014, 14:58:34 schrieb Clemens Eisserer: > Hi Russel, > > > The Debian installer has BTRFS in a list of filesystems to choose with no > > special notice about it. I'm thinking of filing a Debian bug requesting > > that they put a warning against it. > > As long as it is no

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Nick Krause
On Tue, Aug 5, 2014 at 8:38 PM, ronnie sahlberg wrote: > On Tue, Aug 5, 2014 at 5:20 AM, Russell Coker wrote: > >> >> Based on what I've read on this list it seems that BTRFS is less stable in >> 3.15 than in 3.14. Even 3.14 isn't something I'd recommend to random people >> who want something to

Re: ENOSPC with mkdir and rename

2014-08-05 Thread ronnie sahlberg
On Tue, Aug 5, 2014 at 5:20 AM, Russell Coker wrote: > > Based on what I've read on this list it seems that BTRFS is less stable in > 3.15 than in 3.14. Even 3.14 isn't something I'd recommend to random people > who want something to just work. > > The Debian installer has BTRFS in a list of fil

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Duncan
Russell Coker posted on Tue, 05 Aug 2014 22:20:33 +1000 as excerpted: > The Debian installer has BTRFS in a list of filesystems to choose with > no special notice about it. I'm thinking of filing a Debian bug > requesting that they put a warning against it. > > What do people here think? You al

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Chris Samuel
On Tue, 5 Aug 2014 10:20:33 PM Russell Coker wrote: > The Debian installer has BTRFS in a list of filesystems to choose with no > special notice about it. I'm thinking of filing a Debian bug requesting > that they put a warning against it. I think it's a good plan. People should be aware of t

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Peter Waller
On 5 August 2014 13:58, Clemens Eisserer wrote: > As long as it is not selected as the default filesystem, I think it is fine. > Other distributions have been offering btrfs for some time now, too. How do you warn non-BTRFS-developers in this case that they need to run a regular rebalance or they

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Clemens Eisserer
Hi Russel, > The Debian installer has BTRFS in a list of filesystems to choose with no > special notice about it. I'm thinking of filing a Debian bug requesting that > they put a warning against it. As long as it is not selected as the default filesystem, I think it is fine. Other distributions

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Russell Coker
On Tue, 5 Aug 2014 08:06:12 Duncan wrote: > Which is why I'm not particularly happy with seeing all the "btrfs is > still not stable, use at your own risk" warnings disappearing. With them > there, people who chose to run btrfs /could/ be expected to have done > their research and have btrfs sp

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Russell Coker
On Tue, 5 Aug 2014 16:51:44 Qu Wenruo wrote: > In fact such "defeat"(or whatever) is not really btrfs only problem. > In ext*, there is still similiar behavior: ext* has a up limit on the > number of inode after mkfs. > (When you mkfs.ext*, you are prompt the up limit of inodes) > However other me

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Austin S Hemmelgarn
On 2014-08-05 04:20, Duncan wrote: > Austin S Hemmelgarn posted on Mon, 04 Aug 2014 13:09:23 -0400 as > excerpted: > >> Think of each chunk like a box, and each block as a block, and that you >> have two different types of block (data and metadata) and two different >> types of box (also data and

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Qu Wenruo
Original Message Subject: Re: ENOSPC with mkdir and rename From: Peter Waller To: Qu Wenruo Date: 2014年08月04日 16:14 Thanks for responses. All of this is *very* surprising. I'm not new to BTRFS, I've been using it on my own machines for multiple years. I didn'

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Duncan
Austin S Hemmelgarn posted on Mon, 04 Aug 2014 13:09:23 -0400 as excerpted: > Think of each chunk like a box, and each block as a block, and that you > have two different types of block (data and metadata) and two different > types of box (also data and metadata). The data boxes are four times the

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Duncan
Chris Samuel posted on Mon, 04 Aug 2014 20:24:46 +1000 as excerpted: > On Mon, 4 Aug 2014 11:56:46 AM Clemens Eisserer wrote: > >> Which doesn't protect the *average* user from running into issues like >> this. > > No, but they need to be aware of it. Actually, an ordinary user/admin /should/ h

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
Hi Peter, On Mon, 4 Aug 2014 11:59:19 AM Peter Waller wrote: > On 4 August 2014 11:50, Chris Samuel wrote: > > > To be honest I'm not sure I'd suggest btrfs for production use at all at > > present, it's only recently been unmarked as experimental and to be honest > > I feel that was premature.

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 06:31, Peter Waller wrote: > Thanks Hugo, this is the most informative e-mail yet! (more inline) > > On 4 August 2014 11:22, Hugo Mills wrote: >> >> * btrfs fi show >> - look at the total and used values. If used < total, you're OK. >> If used == total, then you could pot

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Mitch Harder
On Mon, Aug 4, 2014 at 9:47 AM, Russell Coker wrote: > If you regularly run a scrub with options such as "-dusage=50 -musage=10" then > the amount of free space in metadata chunks will tend to be a lot greater than > that in data chunks. > Just to clarify for posterity, I'm pretty sure you meant

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Russell Coker
On Mon, 4 Aug 2014 14:17:02 Peter Waller wrote: > For anyone else having this problem, this article is fairly useful for > understanding disk full problems and rebalance: > > http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-> > Full-Problems.html > > It actually covers

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 10:11, Peter Waller wrote: > On 4 August 2014 15:02, Austin S Hemmelgarn wrote: >> I really disagree with the statement that adding more storage is >> difficult or expensive, all you need to do is plug in a 2G USB flash >> drive, or allocate a ramdisk, and add the device to the files

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 15:02, Austin S Hemmelgarn wrote: > I really disagree with the statement that adding more storage is > difficult or expensive, all you need to do is plug in a 2G USB flash > drive, or allocate a ramdisk, and add the device to the filesystem only > long enough to do a full balance.

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 09:17, Peter Waller wrote: > For anyone else having this problem, this article is fairly useful for > understanding disk full problems and rebalance: > > http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html > > It actually covers the problem

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 02:17:02PM +0100, Peter Waller wrote: > For anyone else having this problem, this article is fairly useful for > understanding disk full problems and rebalance: > > http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html > > It actual

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html It actually covers the problem that I had, which is that a rebalance can't take pla

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 01:04:25PM +0200, Clemens Eisserer wrote: > Hi Hugo, > > >On the 3.15+ kernels, the block reserve is split out of metadata > > and reported separately. This helps with the following process: > > Thanks a lot for pointing this out, I hadn't noticed this change until now

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:48:17AM +0100, Peter Waller wrote: > On 4 August 2014 11:39, Hugo Mills wrote: > >> > * btrfs fi df > >> > - look at metadata used vs total. If these are close to zero (on > >> > 3.15+) or close to 512 MiB (on <3.15), then you are in danger of > >> > ENO

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Hugo, >On the 3.15+ kernels, the block reserve is split out of metadata > and reported separately. This helps with the following process: Thanks a lot for pointing this out, I hadn't noticed this change until now. One thing I didn't find any information about is the overhead introduced by

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 11:50, Chris Samuel wrote: > To be honest I'm not sure I'd suggest btrfs for production use at all at > present, it's only recently been unmarked as experimental and to be honest I > feel that was premature. :-( Thanks for the honest answer. There are very positive signals out t

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 11:09:23 AM Peter Waller wrote: > I accept that. It's all very well if you read the BTRFS list and/or > are a BTRFS developer. But if you're trying to work it out in the heat > of battle, as we have sysadmins who would have to, there is a > combination of things here that makes

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
Thanks Hugo, this is the most informative e-mail yet! (more inline) On 4 August 2014 11:22, Hugo Mills wrote: > > * btrfs fi show > - look at the total and used values. If used < total, you're OK. > If used == total, then you could potentially hit ENOSPC. Another thing which is unclea

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 11:39, Hugo Mills wrote: >> > * btrfs fi df >> > - look at metadata used vs total. If these are close to zero (on >> > 3.15+) or close to 512 MiB (on <3.15), then you are in danger of >> > ENOSPC. >> >> Hmm. It's unfortunate that this could indicate an amount of s

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:31:57AM +0100, Peter Waller wrote: > Thanks Hugo, this is the most informative e-mail yet! (more inline) > > On 4 August 2014 11:22, Hugo Mills wrote: > > > > * btrfs fi show > > - look at the total and used values. If used < total, you're OK. > > If used ==

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 11:56:46 AM Clemens Eisserer wrote: > Which doesn't protect the *average* user from running into issues like this. No, but they need to be aware of it. > Just because it has been discussed, doesn't mean nothing can/should be done > about it Indeed, and a lot of work has been

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:09:23AM +0100, Peter Waller wrote: > On 4 August 2014 10:39, Chris Samuel wrote: > > On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: > >> All of this is *very* surprising. > > > > Hmm, it shouldn't be, the ENOSPC issues are well known and have been > > discussed > >

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 10:39, Chris Samuel wrote: > On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: >> All of this is *very* surprising. > > Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed > here for years. I accept that. It's all very well if you read the BTRFS list a

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Chris, > Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed > here for years. Which doesn't protect the *average* user from running into issues like this. Just because it has been discussed, doesn't mean nothing can/should be done about it ;) However, as I am only

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: > All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC -- To unsubscribe from this list

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Peter, > All of this is *very* surprising. I'm not new to BTRFS, I've been > using it on my own machines for multiple years. I didn't realise there > was an un-holstered footgun on my lap at this point. How can it be > made clear how to avoid the ENOSPC problem to myself and other > sysadmins?

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
just that no-one has gotten around to yet? Thanks, - Peter On 4 August 2014 02:38, Qu Wenruo wrote: > Hi, Peter > > Some explain below inline. > > Original Message > Subject: ENOSPC with mkdir and rename > From: Peter Waller > To: > Date: 2014年08

Re: ENOSPC with mkdir and rename

2014-08-03 Thread Qu Wenruo
Hi, Peter Some explain below inline. Original Message Subject: ENOSPC with mkdir and rename From: Peter Waller To: Date: 2014年08月03日 07:35 Hi All, My TL;DR questions are at the bottom, before the stack trace. I'm running Ubuntu 14.04. I wonder if this problem is relat

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Nick Krause
On Sat, Aug 2, 2014 at 10:39 PM, Russell Coker wrote: > On Sun, 3 Aug 2014 00:35:28 Peter Waller wrote: >> I'm running Ubuntu 14.04. I wonder if this problem is related to the >> thread titled "Machine lockup due to btrfs-transaction on AWS EC2 >> >> Ubuntu 14.04" which I started on the 29th of Ju

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Russell Coker
On Sun, 3 Aug 2014 00:35:28 Peter Waller wrote: > I'm running Ubuntu 14.04. I wonder if this problem is related to the > thread titled "Machine lockup due to btrfs-transaction on AWS EC2 > > Ubuntu 14.04" which I started on the 29th of July: > > http://thread.gmane.org/gmane.comp.file-systems.btrf

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Nick Krause
On Sat, Aug 2, 2014 at 8:28 PM, Mitch Harder wrote: > On Sat, Aug 2, 2014 at 6:35 PM, Peter Waller wrote: >> Hi All, >> >> My TL;DR questions are at the bottom, before the stack trace. >> >> I'm running Ubuntu 14.04. I wonder if this problem is related to the >> thread titled "Machine lockup due

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Mitch Harder
On Sat, Aug 2, 2014 at 6:35 PM, Peter Waller wrote: > Hi All, > > My TL;DR questions are at the bottom, before the stack trace. > > I'm running Ubuntu 14.04. I wonder if this problem is related to the > thread titled "Machine lockup due to btrfs-transaction on AWS EC2 > Ubuntu 14.04" which I start

ENOSPC with mkdir and rename

2014-08-02 Thread Peter Waller
Hi All, My TL;DR questions are at the bottom, before the stack trace. I'm running Ubuntu 14.04. I wonder if this problem is related to the thread titled "Machine lockup due to btrfs-transaction on AWS EC2 Ubuntu 14.04" which I started on the 29th of July: > http://thread.gmane.org/gmane.comp.fil