Re: ENOSPC with mkdir and rename

2014-08-10 Thread Martin Steigerwald
Am Dienstag, 5. August 2014, 14:58:34 schrieb Clemens Eisserer: Hi Russel, The Debian installer has BTRFS in a list of filesystems to choose with no special notice about it. I'm thinking of filing a Debian bug requesting that they put a warning against it. As long as it is not

Re: ENOSPC with mkdir and rename

2014-08-10 Thread Martin Steigerwald
Am Montag, 4. August 2014, 11:09:23 schrieb Peter Waller: On 4 August 2014 10:39, Chris Samuel ch...@csamuel.org wrote: On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Duncan
Chris Samuel posted on Mon, 04 Aug 2014 20:24:46 +1000 as excerpted: On Mon, 4 Aug 2014 11:56:46 AM Clemens Eisserer wrote: Which doesn't protect the *average* user from running into issues like this. No, but they need to be aware of it. Actually, an ordinary user/admin /should/ have no

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Duncan
Austin S Hemmelgarn posted on Mon, 04 Aug 2014 13:09:23 -0400 as excerpted: Think of each chunk like a box, and each block as a block, and that you have two different types of block (data and metadata) and two different types of box (also data and metadata). The data boxes are four times the

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Qu Wenruo
Original Message Subject: Re: ENOSPC with mkdir and rename From: Peter Waller pe...@scraperwiki.com To: Qu Wenruo quwen...@cn.fujitsu.com Date: 2014年08月04日 16:14 Thanks for responses. All of this is *very* surprising. I'm not new to BTRFS, I've been using it on my own

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Austin S Hemmelgarn
On 2014-08-05 04:20, Duncan wrote: Austin S Hemmelgarn posted on Mon, 04 Aug 2014 13:09:23 -0400 as excerpted: Think of each chunk like a box, and each block as a block, and that you have two different types of block (data and metadata) and two different types of box (also data and

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Clemens Eisserer
Hi Russel, The Debian installer has BTRFS in a list of filesystems to choose with no special notice about it. I'm thinking of filing a Debian bug requesting that they put a warning against it. As long as it is not selected as the default filesystem, I think it is fine. Other distributions

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Peter Waller
On 5 August 2014 13:58, Clemens Eisserer linuxhi...@gmail.com wrote: As long as it is not selected as the default filesystem, I think it is fine. Other distributions have been offering btrfs for some time now, too. How do you warn non-BTRFS-developers in this case that they need to run a

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Duncan
Russell Coker posted on Tue, 05 Aug 2014 22:20:33 +1000 as excerpted: The Debian installer has BTRFS in a list of filesystems to choose with no special notice about it. I'm thinking of filing a Debian bug requesting that they put a warning against it. What do people here think? You

Re: ENOSPC with mkdir and rename

2014-08-05 Thread ronnie sahlberg
On Tue, Aug 5, 2014 at 5:20 AM, Russell Coker russ...@coker.com.au wrote: Based on what I've read on this list it seems that BTRFS is less stable in 3.15 than in 3.14. Even 3.14 isn't something I'd recommend to random people who want something to just work. The Debian installer has BTRFS

Re: ENOSPC with mkdir and rename

2014-08-05 Thread Nick Krause
On Tue, Aug 5, 2014 at 8:38 PM, ronnie sahlberg ronniesahlb...@gmail.com wrote: On Tue, Aug 5, 2014 at 5:20 AM, Russell Coker russ...@coker.com.au wrote: Based on what I've read on this list it seems that BTRFS is less stable in 3.15 than in 3.14. Even 3.14 isn't something I'd recommend to

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
, Qu Wenruo quwen...@cn.fujitsu.com wrote: Hi, Peter Some explain below inline. Original Message Subject: ENOSPC with mkdir and rename From: Peter Waller pe...@scraperwiki.com To: linux-btrfs@vger.kernel.org Date: 2014年08月03日 07:35 Hi All, My TL;DR questions

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Peter, All of this is *very* surprising. I'm not new to BTRFS, I've been using it on my own machines for multiple years. I didn't realise there was an un-holstered footgun on my lap at this point. How can it be made clear how to avoid the ENOSPC problem to myself and other sysadmins? Or

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC -- To unsubscribe from this

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Chris, Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. Which doesn't protect the *average* user from running into issues like this. Just because it has been discussed, doesn't mean nothing can/should be done about it ;) However, as I am only a

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 10:39, Chris Samuel ch...@csamuel.org wrote: On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. I accept that. It's all very well if you read the

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:09:23AM +0100, Peter Waller wrote: On 4 August 2014 10:39, Chris Samuel ch...@csamuel.org wrote: On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 11:56:46 AM Clemens Eisserer wrote: Which doesn't protect the *average* user from running into issues like this. No, but they need to be aware of it. Just because it has been discussed, doesn't mean nothing can/should be done about it Indeed, and a lot of work has been

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:31:57AM +0100, Peter Waller wrote: Thanks Hugo, this is the most informative e-mail yet! (more inline) On 4 August 2014 11:22, Hugo Mills h...@carfax.org.uk wrote: * btrfs fi show - look at the total and used values. If used total, you're OK. If

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 11:39, Hugo Mills h...@carfax.org.uk wrote: * btrfs fi df - look at metadata used vs total. If these are close to zero (on 3.15+) or close to 512 MiB (on 3.15), then you are in danger of ENOSPC. Hmm. It's unfortunate that this could indicate an amount of

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
Thanks Hugo, this is the most informative e-mail yet! (more inline) On 4 August 2014 11:22, Hugo Mills h...@carfax.org.uk wrote: * btrfs fi show - look at the total and used values. If used total, you're OK. If used == total, then you could potentially hit ENOSPC. Another thing

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 11:50, Chris Samuel ch...@csamuel.org wrote: To be honest I'm not sure I'd suggest btrfs for production use at all at present, it's only recently been unmarked as experimental and to be honest I feel that was premature. :-( Thanks for the honest answer. There are very

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Hugo, On the 3.15+ kernels, the block reserve is split out of metadata and reported separately. This helps with the following process: Thanks a lot for pointing this out, I hadn't noticed this change until now. One thing I didn't find any information about is the overhead introduced by

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 01:04:25PM +0200, Clemens Eisserer wrote: Hi Hugo, On the 3.15+ kernels, the block reserve is split out of metadata and reported separately. This helps with the following process: Thanks a lot for pointing this out, I hadn't noticed this change until now.

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html It actually covers the problem that I had, which is that a rebalance can't take

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 02:17:02PM +0100, Peter Waller wrote: For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html It actually

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 09:17, Peter Waller wrote: For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html It actually covers the problem that

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 15:02, Austin S Hemmelgarn ahferro...@gmail.com wrote: I really disagree with the statement that adding more storage is difficult or expensive, all you need to do is plug in a 2G USB flash drive, or allocate a ramdisk, and add the device to the filesystem only long enough to

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 10:11, Peter Waller wrote: On 4 August 2014 15:02, Austin S Hemmelgarn ahferro...@gmail.com wrote: I really disagree with the statement that adding more storage is difficult or expensive, all you need to do is plug in a 2G USB flash drive, or allocate a ramdisk, and add the

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Russell Coker
On Mon, 4 Aug 2014 14:17:02 Peter Waller wrote: For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem- Full-Problems.html It actually covers the

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Mitch Harder
On Mon, Aug 4, 2014 at 9:47 AM, Russell Coker russ...@coker.com.au wrote: If you regularly run a scrub with options such as -dusage=50 -musage=10 then the amount of free space in metadata chunks will tend to be a lot greater than that in data chunks. Just to clarify for posterity, I'm pretty

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 06:31, Peter Waller wrote: Thanks Hugo, this is the most informative e-mail yet! (more inline) On 4 August 2014 11:22, Hugo Mills h...@carfax.org.uk wrote: * btrfs fi show - look at the total and used values. If used total, you're OK. If used == total, then you

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
Hi Peter, On Mon, 4 Aug 2014 11:59:19 AM Peter Waller wrote: On 4 August 2014 11:50, Chris Samuel ch...@csamuel.org wrote: To be honest I'm not sure I'd suggest btrfs for production use at all at present, it's only recently been unmarked as experimental and to be honest I feel that was

Re: ENOSPC with mkdir and rename

2014-08-03 Thread Qu Wenruo
Hi, Peter Some explain below inline. Original Message Subject: ENOSPC with mkdir and rename From: Peter Waller pe...@scraperwiki.com To: linux-btrfs@vger.kernel.org Date: 2014年08月03日 07:35 Hi All, My TL;DR questions are at the bottom, before the stack trace. I'm running

ENOSPC with mkdir and rename

2014-08-02 Thread Peter Waller
-systems.btrfs/37224 Kernel: 3.15.7-031507-generic I'm on a single block device system, i.e, no RAID. I was observing ENOSPC from `mkdir` and `rename` on this system, with a good amount of free disk space (df -h reports 62 GB remain). I added enospc_debug (full umount/mount, not just mount -o

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Mitch Harder
which I started on the 29th of July: http://thread.gmane.org/gmane.comp.file-systems.btrfs/37224 Kernel: 3.15.7-031507-generic I'm on a single block device system, i.e, no RAID. I was observing ENOSPC from `mkdir` and `rename` on this system, with a good amount of free disk space (df -h

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Nick Krause
`mkdir` and `rename` on this system, with a good amount of free disk space (df -h reports 62 GB remain). I added enospc_debug (full umount/mount, not just mount -o remount), but this had no apparent effect when receiving ENOSPC from userland. $ sudo btrfs fi df /path/to/volume Data, single: total

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Russell Coker
On Sun, 3 Aug 2014 00:35:28 Peter Waller wrote: I'm running Ubuntu 14.04. I wonder if this problem is related to the thread titled Machine lockup due to btrfs-transaction on AWS EC2 Ubuntu 14.04 which I started on the 29th of July: http://thread.gmane.org/gmane.comp.file-systems.btrfs/37224

Re: ENOSPC with mkdir and rename

2014-08-02 Thread Nick Krause
On Sat, Aug 2, 2014 at 10:39 PM, Russell Coker russ...@coker.com.au wrote: On Sun, 3 Aug 2014 00:35:28 Peter Waller wrote: I'm running Ubuntu 14.04. I wonder if this problem is related to the thread titled Machine lockup due to btrfs-transaction on AWS EC2 Ubuntu 14.04 which I started on the