Unrecoverable errors when the btrfs file system was modified outside the running OS

2014-08-04 Thread rocwhite168
Hello, I just had a very frustrating experience with btrfs, which I was only able to resolve by rolling back to ext4 using the subvol btrfs-convert created. The same type of situation occurred before when I was using the ext file system and the result was far less disastrous. The source of

[PATCH] btrfs-progs: Documentations: Fix a format error in btrfs-replace.txt

2014-08-04 Thread Qu Wenruo
Missing '+'s cause '-B' option not displayed correctly, add it to fix. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com --- Documentation/btrfs-replace.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/btrfs-replace.txt b/Documentation/btrfs-replace.txt index

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
Thanks for responses. All of this is *very* surprising. I'm not new to BTRFS, I've been using it on my own machines for multiple years. I didn't realise there was an un-holstered footgun on my lap at this point. How can it be made clear how to avoid the ENOSPC problem to myself and other

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Peter, All of this is *very* surprising. I'm not new to BTRFS, I've been using it on my own machines for multiple years. I didn't realise there was an un-holstered footgun on my lap at this point. How can it be made clear how to avoid the ENOSPC problem to myself and other sysadmins? Or

Re: Scan not being performed properly on boot

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 01:31:42 PM Russell Coker wrote: Is BTRFS supported in that version of Ubuntu? Out of the box a fresh 14.04 install onto btrfs worked fine for me on two different sets of hardware. 13.10 the same on a third piece of hardware. cheers, Chris -- Chris Samuel :

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC -- To unsubscribe from this

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Chris, Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. Which doesn't protect the *average* user from running into issues like this. Just because it has been discussed, doesn't mean nothing can/should be done about it ;) However, as I am only a

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 10:39, Chris Samuel ch...@csamuel.org wrote: On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been discussed here for years. I accept that. It's all very well if you read the

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:09:23AM +0100, Peter Waller wrote: On 4 August 2014 10:39, Chris Samuel ch...@csamuel.org wrote: On Mon, 4 Aug 2014 09:14:19 AM Peter Waller wrote: All of this is *very* surprising. Hmm, it shouldn't be, the ENOSPC issues are well known and have been

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
On Mon, 4 Aug 2014 11:56:46 AM Clemens Eisserer wrote: Which doesn't protect the *average* user from running into issues like this. No, but they need to be aware of it. Just because it has been discussed, doesn't mean nothing can/should be done about it Indeed, and a lot of work has been

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 11:31:57AM +0100, Peter Waller wrote: Thanks Hugo, this is the most informative e-mail yet! (more inline) On 4 August 2014 11:22, Hugo Mills h...@carfax.org.uk wrote: * btrfs fi show - look at the total and used values. If used total, you're OK. If

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 11:39, Hugo Mills h...@carfax.org.uk wrote: * btrfs fi df - look at metadata used vs total. If these are close to zero (on 3.15+) or close to 512 MiB (on 3.15), then you are in danger of ENOSPC. Hmm. It's unfortunate that this could indicate an amount of

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
Thanks Hugo, this is the most informative e-mail yet! (more inline) On 4 August 2014 11:22, Hugo Mills h...@carfax.org.uk wrote: * btrfs fi show - look at the total and used values. If used total, you're OK. If used == total, then you could potentially hit ENOSPC. Another thing

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 11:50, Chris Samuel ch...@csamuel.org wrote: To be honest I'm not sure I'd suggest btrfs for production use at all at present, it's only recently been unmarked as experimental and to be honest I feel that was premature. :-( Thanks for the honest answer. There are very

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Clemens Eisserer
Hi Hugo, On the 3.15+ kernels, the block reserve is split out of metadata and reported separately. This helps with the following process: Thanks a lot for pointing this out, I hadn't noticed this change until now. One thing I didn't find any information about is the overhead introduced by

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 01:04:25PM +0200, Clemens Eisserer wrote: Hi Hugo, On the 3.15+ kernels, the block reserve is split out of metadata and reported separately. This helps with the following process: Thanks a lot for pointing this out, I hadn't noticed this change until now.

Re: [PATCH] Btrfs: fix compressed write corruption on enospc

2014-08-04 Thread Martin Steigerwald
Am Freitag, 25. Juli 2014, 11:54:37 schrieb Martin Steigerwald: Am Donnerstag, 24. Juli 2014, 22:48:05 schrieben Sie: When failing to allocate space for the whole compressed extent, we'll fallback to uncompressed IO, but we've forgotten to redirty the pages which belong to this compressed

Re: [PATCH] Btrfs: fix compressed write corruption on enospc

2014-08-04 Thread Martin Steigerwald
Am Montag, 4. August 2014, 14:50:29 schrieb Martin Steigerwald: Am Freitag, 25. Juli 2014, 11:54:37 schrieb Martin Steigerwald: Am Donnerstag, 24. Juli 2014, 22:48:05 schrieben Sie: When failing to allocate space for the whole compressed extent, we'll fallback to uncompressed IO, but

Re: btrfs on bcache

2014-08-04 Thread Fábio Pfeifer
After completely loosing my filesystem twice because of this bug, I gave up using btrfs on top of bcache (also writeback). In my case, I used to have some subvolumes and some snapshot of these subvolumes, but not many of them. The btrfs mantra backup, bakcup and backup saved me. Best regards,

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html It actually covers the problem that I had, which is that a rebalance can't take

Re: Work Queue for btrfs compression writes

2014-08-04 Thread Chris Mason
On 07/29/2014 11:54 PM, Nick Krause wrote: Hey Guys , I am new to reading and writing kernel code.I got interested in writing code for btrfs as it seems to need more work then other file systems and this seems other then drivers, a good use of time on my part. I interested in helping

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Hugo Mills
On Mon, Aug 04, 2014 at 02:17:02PM +0100, Peter Waller wrote: For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html It actually

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 09:17, Peter Waller wrote: For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html It actually covers the problem that

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Peter Waller
On 4 August 2014 15:02, Austin S Hemmelgarn ahferro...@gmail.com wrote: I really disagree with the statement that adding more storage is difficult or expensive, all you need to do is plug in a 2G USB flash drive, or allocate a ramdisk, and add the device to the filesystem only long enough to

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 10:11, Peter Waller wrote: On 4 August 2014 15:02, Austin S Hemmelgarn ahferro...@gmail.com wrote: I really disagree with the statement that adding more storage is difficult or expensive, all you need to do is plug in a 2G USB flash drive, or allocate a ramdisk, and add the

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Russell Coker
On Mon, 4 Aug 2014 14:17:02 Peter Waller wrote: For anyone else having this problem, this article is fairly useful for understanding disk full problems and rebalance: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem- Full-Problems.html It actually covers the

Re: Scan not being performed properly on boot

2014-08-04 Thread Peter Roberts
On 04/08/2014 04:31, Russell Coker wrote: What is GRUB (or your boot loader) giving as parameters to the kernel? What error messages appear on screen? Sometimes it's helpful to photograph the screen and put the picture on a web server to help people diagnose the problem. Here a screenshot

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Mitch Harder
On Mon, Aug 4, 2014 at 9:47 AM, Russell Coker russ...@coker.com.au wrote: If you regularly run a scrub with options such as -dusage=50 -musage=10 then the amount of free space in metadata chunks will tend to be a lot greater than that in data chunks. Just to clarify for posterity, I'm pretty

Re: Unrecoverable errors when the btrfs file system was modified outside the running OS

2014-08-04 Thread Chris Murphy
On Aug 4, 2014, at 12:29 AM, rocwhite168 rocwhite...@163.com wrote: Hello, I just had a very frustrating experience with btrfs, which I was only able to resolve by rolling back to ext4 using the subvol btrfs-convert created. The same type of situation occurred before when I was using the

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Austin S Hemmelgarn
On 2014-08-04 06:31, Peter Waller wrote: Thanks Hugo, this is the most informative e-mail yet! (more inline) On 4 August 2014 11:22, Hugo Mills h...@carfax.org.uk wrote: * btrfs fi show - look at the total and used values. If used total, you're OK. If used == total, then you

[PATCH] Btrfs: make btrfs_search_forward return with nodes unlocked

2014-08-04 Thread Filipe Manana
None of the uses of btrfs_search_forward() need to have the path nodes (level = 1) read locked, only the leaf needs to be locked while the caller processes it. Therefore make it return a path with all nodes unlocked, except for the leaf. This change is motivated by the observation that during a

Re: [PATCH 1/1] Btrfs: fix sparse warning

2014-08-04 Thread Zach Brown
On Sat, Aug 02, 2014 at 02:24:49PM +0200, Fabian Frederick wrote: On Thu, 17 Jul 2014 12:01:52 -0700 Zach Brown z...@zabbo.net wrote: @@ -515,7 +515,8 @@ static int write_buf(struct file *filp, const void *buf, u32 len, loff_t *off) Though this probably wants to be

Re: [PATCH 03/12] btrfs: handle errors from reading the quota tree root

2014-08-04 Thread Zach Brown
On Fri, Aug 01, 2014 at 06:12:37PM -0500, Eric Sandeen wrote: Reading the quota tree root may fail with ENOENT if there is no quota, which is fine, but the code was ignoring every other error as well, which is not fine. Kinda makes you want to write a test that would have caught this. Kinda.

Re: [PATCH 03/12] btrfs: handle errors from reading the quota tree root

2014-08-04 Thread Eric Sandeen
On 8/4/14, 1:35 PM, Zach Brown wrote: On Fri, Aug 01, 2014 at 06:12:37PM -0500, Eric Sandeen wrote: Reading the quota tree root may fail with ENOENT if there is no quota, which is fine, but the code was ignoring every other error as well, which is not fine. Kinda makes you want to write a

Re: [PATCH 03/12] btrfs: handle errors from reading the quota tree root

2014-08-04 Thread Zach Brown
On Mon, Aug 04, 2014 at 01:42:23PM -0500, Eric Sandeen wrote: On 8/4/14, 1:35 PM, Zach Brown wrote: On Fri, Aug 01, 2014 at 06:12:37PM -0500, Eric Sandeen wrote: Reading the quota tree root may fail with ENOENT if there is no quota, which is fine, but the code was ignoring every other

Re: [PATCH 03/12] btrfs: handle errors from reading the quota tree root

2014-08-04 Thread Eric Sandeen
On 8/4/14, 1:51 PM, Zach Brown wrote: On Mon, Aug 04, 2014 at 01:42:23PM -0500, Eric Sandeen wrote: On 8/4/14, 1:35 PM, Zach Brown wrote: On Fri, Aug 01, 2014 at 06:12:37PM -0500, Eric Sandeen wrote: Reading the quota tree root may fail with ENOENT if there is no quota, which is fine, but the

btrfs sub list output

2014-08-04 Thread Hugo Mills
The output options of btrfs sub list seem a bit... arbitrary? awkward? unhelpful? Here's my problem: Given a path at some arbitrary point into a mounted btrfs (sub)volume, find all subvolumes visible under that point, and identify their absolute path names. My test btrfs filesystem

Re: ENOSPC with mkdir and rename

2014-08-04 Thread Chris Samuel
Hi Peter, On Mon, 4 Aug 2014 11:59:19 AM Peter Waller wrote: On 4 August 2014 11:50, Chris Samuel ch...@csamuel.org wrote: To be honest I'm not sure I'd suggest btrfs for production use at all at present, it's only recently been unmarked as experimental and to be honest I feel that was

btrfs and iostat - how do I measure the live performance of my btrsf filesystems?

2014-08-04 Thread G. Richard Bellamy
I apologize if this has already been addressed, regardless of whether or not it's a n00b question, I'm stumped and am looking for some guidance. Is iostat a viable measure of btrfs performance? I've got the following environment:

Re: integration tree updated

2014-08-04 Thread Chris Mason
On 08/01/2014 05:05 AM, Anand Jain wrote: Hi Chris, Looks like you missed Miao update (on 17 Jul) to back out the bad patch (below), and your integration branch (published on 25 Jul) still contains the same. [PATCH 2/2] Btrfs: fix wrong total device counter after removing a seed