Re: non-atomic xattr replacement in btrfs = rsync random errors

2014-11-07 Thread Holger Hoffstätte
On Thu, 06 Nov 2014 23:39:02 -0200, Alexandre Oliva wrote: [dropping rs...@lists.samba.org, it rejects posts from non-subscribers; refer to https://bugzilla.samba.org/show_bug.cgi?id=10925 instead] On Nov 6, 2014, Alexandre Oliva ol...@gnu.org wrote: What makes the problem visible is

Re: non-atomic xattr replacement in btrfs = rsync random errors

2014-11-07 Thread Filipe David Manana
On Fri, Nov 7, 2014 at 12:37 AM, Alexandre Oliva ol...@gnu.org wrote: A few days ago, I started using rsync batches to archive old copies of ceph OSD snapshots for certain kinds of disaster recovery. This seems to exercise an unexpected race condition in rsync, which happens to expose what

[PATCH] Btrfs: avoid crash when overflowing a dir_item

2014-11-07 Thread Filipe Manana
When attempting to insert a new dir_item, we were calling btrfs_extent_item() without checking if the leaf has enough space to extend the item. This made btrfs_extent_item() crash through a BUG() call. Therefore do the check and return ENOSPC if the leaf doesn't have enough space. Signed-off-by:

Re: [PATCH] Btrfs: avoid crash when overflowing a dir_item

2014-11-07 Thread Filipe David Manana
On Fri, Nov 7, 2014 at 12:15 PM, Filipe Manana fdman...@suse.com wrote: When attempting to insert a new dir_item, we were calling btrfs_extent_item() without checking if the leaf has enough space to extend the item. This made btrfs_extent_item() crash through a BUG() call. Therefore do the

corruption, bad block, input/output errors - do i run --repair?

2014-11-07 Thread Matt McKinnon
Hi All, I'm running into some corruption and I wanted to seek out advice on whether or not to run btrfs check --repair, or if I should fall back to my backup file server, or both. The system is mountable, and usable. # uname -a Linux cbmm-fs 3.17.2-custom #1 SMP Thu Oct 30 14:09:57 EDT 2014

[PATCH 0/4] More mount-time checks

2014-11-07 Thread David Sterba
This series adds more superblock checks that turned out not to be enough after btrfs: add more superblock checks. Would be good to get them into 3.18 as I'd like to submit them to stable kernels as well. This is not easily possible right now as it depends on a patch that's not merged yet

[PATCH 1/4] btrfs: more superblock checks, lower bounds on devices and sectorsize/nodesize

2014-11-07 Thread David Sterba
I received a few crafted images from Jiri, all got through the recently added superblock checks. The lower bounds checks for num_devices and sector/node -sizes were missing and caused a crash during mount. Tools for symbolic code execution were used to prepare the images contents. Reported-by:

[PATCH 3/4] btrfs: cleanup, rename a few variables in btrfs_read_sys_array

2014-11-07 Thread David Sterba
There's a pointer to buffer, integer offset and offset passed as pointer, try to find matching names for them. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/volumes.c | 31 --- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git

[PATCH 4/4] btrfs: add more checks to btrfs_read_sys_array

2014-11-07 Thread David Sterba
Verify that the sys_array has enough bytes to read the next item. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/volumes.c | 26 +++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index

[PATCH 2/4] btrfs: add checks for sys_chunk_array sizes

2014-11-07 Thread David Sterba
Verify that possible minimum and maximum size is set, validity of contents is checked in btrfs_read_sys_array. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/disk-io.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index

[PATCH 0/5] Btrfs-progs updates to build

2014-11-07 Thread David Sterba
We've managed to break build of snapper two times now, a new make rule 'test-build' should help us to check the build. Static build is not required to pass due to dependency on static libs that may not be widely available. David Sterba (5): btrfs-progs: build, add basic build test for library

[PATCH 3/5] btrfs-progs: build, gather utilities that are not built by default

2014-11-07 Thread David Sterba
Add them to build test and to clean rules. Signed-off-by: David Sterba dste...@suse.cz --- Makefile | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 126dc593ddfb..b597ad8e9b89 100644 --- a/Makefile +++ b/Makefile @@ -50,6 +50,9 @@ progs =

[PATCH 5/5] btrfs-progs: update .gitignore

2014-11-07 Thread David Sterba
Signed-off-by: David Sterba dste...@suse.cz --- .gitignore | 4 1 file changed, 4 insertions(+) diff --git a/.gitignore b/.gitignore index fc8c07a22f7f..e637b170714b 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ *.o *.static.o *.o.d +tags .cc-defines.h version.h version @@

[PATCH 2/5] btrfs-progs: build, add make target for a build test

2014-11-07 Thread David Sterba
A quick check that everything still builds. Signed-off-by: David Sterba dste...@suse.cz --- Makefile | 7 +++ 1 file changed, 7 insertions(+) diff --git a/Makefile b/Makefile index 99b03658a406..126dc593ddfb 100644 --- a/Makefile +++ b/Makefile @@ -248,6 +248,13 @@ library-test.static:

[PATCH 4/5] btrfs-progs: build, update the clean rule

2014-11-07 Thread David Sterba
Remove duplicate files, add extra progs, add library test binaries. Signed-off-by: David Sterba dste...@suse.cz --- Makefile | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index b597ad8e9b89..4cae30c179b5 100644 --- a/Makefile +++ b/Makefile @@

[PATCH 1/5] btrfs-progs: build, add basic build test for library

2014-11-07 Thread David Sterba
Basic test based on snapper code that uses the send stream API. Signed-off-by: David Sterba dste...@suse.cz --- Makefile | 8 +++ library-test.c | 69 ++ 2 files changed, 77 insertions(+) create mode 100644 library-test.c diff

Re: [PATCH] btrfs-progs: skip fs with no seed when build seed/sprout mapping for fi show

2014-11-07 Thread David Sterba
On Fri, Nov 07, 2014 at 10:07:41AM +0800, Gui Hecheng wrote: +static int no_seed_devices(struct btrfs_fs_devices *fs_devices) Please reverse the meaning of the function, ie. something like 'has_seed_devices'. It's more natural to deal with negation operators when they're explicit rather than

Re: [PATCH 2/2] btrfs-progs: fix wrong num_devices for btrfs fi show with seed devices

2014-11-07 Thread David Sterba
On Fri, Nov 07, 2014 at 10:07:43AM +0800, Gui Hecheng wrote: The @fi_args-num_devices in @get_fs_info() does not include seed devices. We could just correct it by searching the chunk tree and count how many dev_items there are in total which includes seed devices. Signed-off-by: Gui Hecheng

Re: Quota question

2014-11-07 Thread Cyril Scetbon
Hi guys, I've made some tests with a newer version of btrfs, and I still have issues :( http://pastebin.com/cycu08LG I have 2 directories with quotas and a parent qgroup. All have quotas limits but something goes wrong. When the parent limit is exceeded I have a Disk quota exceeded which is

[PATCH v5] Move BTRFS RCU string to common library

2014-11-07 Thread Omar Sandoval
The RCU-friendly string API used internally by BTRFS is generic enough for common use. This doesn't add any new functionality, but instead just moves the code and documents the existing API. Reviewed-by: Josh Triplett j...@joshtriplett.org Acked-by: Paul E. McKenney paul...@linux.vnet.ibm.com

Re: Quota question

2014-11-07 Thread Christoph Hellwig
On Wed, Nov 05, 2014 at 09:52:18PM +0100, Cyril Scetbon wrote: oh cool to know ! It's weird that the man page says limits are never enforced on the superuser (nor are they enforced for group and project ID zero) http://manpages.ubuntu.com/manpages/trusty/en/man8/xfs_quota.8.html This

Re: [PATCH v5] Move BTRFS RCU string to common library

2014-11-07 Thread Chris Mason
On Fri, Nov 7, 2014 at 3:17 PM, Omar Sandoval osan...@osandov.com wrote: The RCU-friendly string API used internally by BTRFS is generic enough for common use. This doesn't add any new functionality, but instead just moves the code and documents the existing API. Reviewed-by: Josh Triplett

[PATCH] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Filipe Manana
Replacing a xattr consists of doing a lookup for its existing value, delete the current value from the respective leaf, release the search path and then finally insert the new value. This leaves a time window where readers (getxattr, listxattrs) won't see any value for the xattr. Xattrs are used

[PATCH] fstests: add generic test to verify xattr replace operations are atomic

2014-11-07 Thread Filipe Manana
This test verifies that replacing a xattr's value is an atomic operation. This is motivated by an issue in btrfs where replacing a xattr's value wasn't an atomic operation, it consisted of removing the old value and then inserting the new value in a btree. This made readers (getxattr and

Re: [PATCH] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Chris Mason
On Fri, Nov 7, 2014 at 3:39 PM, Filipe Manana fdman...@suse.com wrote: Replacing a xattr consists of doing a lookup for its existing value, delete the current value from the respective leaf, release the search path and then finally insert the new value. This leaves a time window where readers

Re: [PATCH] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Filipe David Manana
On Fri, Nov 7, 2014 at 8:54 PM, Chris Mason c...@fb.com wrote: On Fri, Nov 7, 2014 at 3:39 PM, Filipe Manana fdman...@suse.com wrote: Replacing a xattr consists of doing a lookup for its existing value, delete the current value from the respective leaf, release the search path and then

Re: [PATCH v5] Move BTRFS RCU string to common library

2014-11-07 Thread Joe Perches
On Fri, 2014-11-07 at 15:21 -0500, Chris Mason wrote: On Fri, Nov 7, 2014 at 3:17 PM, Omar Sandoval osan...@osandov.com wrote: The RCU-friendly string API used internally by BTRFS is generic enough for common use. This doesn't add any new functionality, but instead just moves the

Re: [PATCH 2/2] btrfs-progs: fix wrong num_devices for btrfs fi show with seed devices

2014-11-07 Thread anand jain
really nice fix. Thanks Gui. Anand On 08/11/2014 02:16, David Sterba wrote: On Fri, Nov 07, 2014 at 10:07:43AM +0800, Gui Hecheng wrote: The @fi_args-num_devices in @get_fs_info() does not include seed devices. We could just correct it by searching the chunk tree and count how many dev_items

[PATCH v2] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Filipe Manana
Replacing a xattr consists of doing a lookup for its existing value, delete the current value from the respective leaf, release the search path and then finally insert the new value. This leaves a time window where readers (getxattr, listxattrs) won't see any value for the xattr. Xattrs are used

Re: [PATCH v2] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Robert White
On 11/07/2014 05:01 PM, Filipe Manana wrote: V2: Added missing btrfs_mark_buffer_dirty() call for the case where a replace happens for an item with no other xattrs and the new xattr value size is the same as the old xattr value size; Made btrfs_search_slot not release the path if

Re: Quota question

2014-11-07 Thread Duncan
Cyril Scetbon posted on Fri, 07 Nov 2014 20:06:40 +0100 as excerpted: I've made some tests with a newer version of btrfs, and I still have issues :( http://pastebin.com/cycu08LG I have 2 directories with quotas and a parent qgroup. All have quotas limits but something goes wrong. When

Cannot mount volume after device replace locked up

2014-11-07 Thread James Hogarth
Hi, I have a F20 system with BTRFS on a 4 disk RAID1 profile One of the disks failed the other day and when I was replacing it today I think a scheduled snapshot was attempted - the following appeared in the logs and any btrfs commands locked up. I don't know if the snapshot was relate dor not

Re: Cannot mount volume after device replace locked up

2014-11-07 Thread James Hogarth
On 8 November 2014 02:55, James Hogarth james.hoga...@gmail.com wrote: Hi, I have a F20 system with BTRFS on a 4 disk RAID1 profile One of the disks failed the other day and when I was replacing it today I think a scheduled snapshot was attempted - the following appeared in the logs and any

Re: corruption, bad block, input/output errors - do i run --repair?

2014-11-07 Thread Duncan
Matt McKinnon posted on Fri, 07 Nov 2014 09:33:44 -0500 as excerpted: I'm running into some corruption and I wanted to seek out advice on whether or not to run btrfs check --repair, or if I should fall back to my backup file server, or both. The system is mountable, and usable. # uname

Re: [PATCH v2] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Filipe David Manana
On Sat, Nov 8, 2014 at 1:26 AM, Robert White rwh...@pobox.com wrote: On 11/07/2014 05:01 PM, Filipe Manana wrote: V2: Added missing btrfs_mark_buffer_dirty() call for the case where a replace happens for an item with no other xattrs and the new xattr value size is the same as the