Re: [PATCH v2] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Filipe David Manana
On Sat, Nov 8, 2014 at 1:26 AM, Robert White wrote: > On 11/07/2014 05:01 PM, Filipe Manana wrote: >> >> V2: Added missing btrfs_mark_buffer_dirty() call for the case where a >> replace >> happens for an item with no other xattrs and the new xattr value size >> is >> the same as the old

Re: corruption, bad block, input/output errors - do i run --repair?

2014-11-07 Thread Duncan
Matt McKinnon posted on Fri, 07 Nov 2014 09:33:44 -0500 as excerpted: > I'm running into some corruption and I wanted to seek out advice on > whether or not to run btrfs check --repair, or if I should fall back to > my backup file server, or both. > > The system is mountable, and usable. > > # u

Re: Cannot mount volume after device replace locked up

2014-11-07 Thread James Hogarth
On 8 November 2014 02:55, James Hogarth wrote: > Hi, > > I have a F20 system with BTRFS on a 4 disk RAID1 profile > > One of the disks failed the other day and when I was replacing it > today I think a scheduled snapshot was attempted - the following > appeared in the logs and any btrfs commands l

Cannot mount volume after device replace locked up

2014-11-07 Thread James Hogarth
Hi, I have a F20 system with BTRFS on a 4 disk RAID1 profile One of the disks failed the other day and when I was replacing it today I think a scheduled snapshot was attempted - the following appeared in the logs and any btrfs commands locked up. I don't know if the snapshot was relate dor not b

Re: Quota question

2014-11-07 Thread Duncan
Cyril Scetbon posted on Fri, 07 Nov 2014 20:06:40 +0100 as excerpted: > I've made some tests with a newer version of btrfs, and I still have > issues :( > > http://pastebin.com/cycu08LG > > I have 2 directories with quotas and a parent qgroup. All have quotas > limits but something goes wrong. W

Re: [PATCH v2] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Robert White
On 11/07/2014 05:01 PM, Filipe Manana wrote: V2: Added missing btrfs_mark_buffer_dirty() call for the case where a replace happens for an item with no other xattrs and the new xattr value size is the same as the old xattr value size; Made btrfs_search_slot not release the path if i

[PATCH v2] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Filipe Manana
Replacing a xattr consists of doing a lookup for its existing value, delete the current value from the respective leaf, release the search path and then finally insert the new value. This leaves a time window where readers (getxattr, listxattrs) won't see any value for the xattr. Xattrs are used to

Re: [PATCH 2/2] btrfs-progs: fix wrong num_devices for btrfs fi show with seed devices

2014-11-07 Thread anand jain
really nice fix. Thanks Gui. Anand On 08/11/2014 02:16, David Sterba wrote: On Fri, Nov 07, 2014 at 10:07:43AM +0800, Gui Hecheng wrote: The @fi_args->num_devices in @get_fs_info() does not include seed devices. We could just correct it by searching the chunk tree and count how many dev_items

Re: [PATCH v5] Move BTRFS RCU string to common library

2014-11-07 Thread Joe Perches
On Fri, 2014-11-07 at 15:21 -0500, Chris Mason wrote: > > On Fri, Nov 7, 2014 at 3:17 PM, Omar Sandoval > wrote: > > The RCU-friendly string API used internally by BTRFS is generic > > enough for > > common use. This doesn't add any new functionality, but instead just > > moves the > > code an

Re: [PATCH] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Filipe David Manana
On Fri, Nov 7, 2014 at 8:54 PM, Chris Mason wrote: > > > On Fri, Nov 7, 2014 at 3:39 PM, Filipe Manana wrote: >> >> Replacing a xattr consists of doing a lookup for its existing value, >> delete >> the current value from the respective leaf, release the search path and >> then >> finally insert t

Re: [PATCH] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Chris Mason
On Fri, Nov 7, 2014 at 3:39 PM, Filipe Manana wrote: Replacing a xattr consists of doing a lookup for its existing value, delete the current value from the respective leaf, release the search path and then finally insert the new value. This leaves a time window where readers (getxattr, listx

[PATCH] fstests: add generic test to verify xattr replace operations are atomic

2014-11-07 Thread Filipe Manana
This test verifies that replacing a xattr's value is an atomic operation. This is motivated by an issue in btrfs where replacing a xattr's value wasn't an atomic operation, it consisted of removing the old value and then inserting the new value in a btree. This made readers (getxattr and listxattrs

[PATCH] Btrfs: make xattr replace operations atomic

2014-11-07 Thread Filipe Manana
Replacing a xattr consists of doing a lookup for its existing value, delete the current value from the respective leaf, release the search path and then finally insert the new value. This leaves a time window where readers (getxattr, listxattrs) won't see any value for the xattr. Xattrs are used to

Re: [PATCH v5] Move BTRFS RCU string to common library

2014-11-07 Thread Chris Mason
On Fri, Nov 7, 2014 at 3:17 PM, Omar Sandoval wrote: The RCU-friendly string API used internally by BTRFS is generic enough for common use. This doesn't add any new functionality, but instead just moves the code and documents the existing API. Reviewed-by: Josh Triplett Acked-by: Paul E.

Re: Quota question

2014-11-07 Thread Christoph Hellwig
On Wed, Nov 05, 2014 at 09:52:18PM +0100, Cyril Scetbon wrote: > oh cool to know ! > > It's weird that the man page says "limits are never enforced on the superuser > (nor are they enforced for group and project ID zero)" > > http://manpages.ubuntu.com/manpages/trusty/en/man8/xfs_quota.8.html T

[PATCH v5] Move BTRFS RCU string to common library

2014-11-07 Thread Omar Sandoval
The RCU-friendly string API used internally by BTRFS is generic enough for common use. This doesn't add any new functionality, but instead just moves the code and documents the existing API. Reviewed-by: Josh Triplett Acked-by: Paul E. McKenney Signed-off-by: Omar Sandoval --- Chris, could you

Re: Quota question

2014-11-07 Thread Cyril Scetbon
Hi guys, I've made some tests with a newer version of btrfs, and I still have issues :( http://pastebin.com/cycu08LG I have 2 directories with quotas and a parent qgroup. All have quotas limits but something goes wrong. When the parent limit is exceeded I have a "Disk quota exceeded" which is

Re: [PATCH 2/2] btrfs-progs: fix wrong num_devices for btrfs fi show with seed devices

2014-11-07 Thread David Sterba
On Fri, Nov 07, 2014 at 10:07:43AM +0800, Gui Hecheng wrote: > The @fi_args->num_devices in @get_fs_info() does not include seed devices. > We could just correct it by searching the chunk tree and count how > many dev_items there are in total which includes seed devices. > > Signed-off-by: Gui Hec

Re: [PATCH] btrfs-progs: skip fs with no seed when build seed/sprout mapping for fi show

2014-11-07 Thread David Sterba
On Fri, Nov 07, 2014 at 10:07:41AM +0800, Gui Hecheng wrote: > +static int no_seed_devices(struct btrfs_fs_devices *fs_devices) Please reverse the meaning of the function, ie. something like 'has_seed_devices'. It's more natural to deal with negation operators when they're explicit rather than hid

[PATCH 2/5] btrfs-progs: build, add make target for a build test

2014-11-07 Thread David Sterba
A quick check that everything still builds. Signed-off-by: David Sterba --- Makefile | 7 +++ 1 file changed, 7 insertions(+) diff --git a/Makefile b/Makefile index 99b03658a406..126dc593ddfb 100644 --- a/Makefile +++ b/Makefile @@ -248,6 +248,13 @@ library-test.static: $(libs_static) libra

[PATCH 4/5] btrfs-progs: build, update the clean rule

2014-11-07 Thread David Sterba
Remove duplicate files, add extra progs, add library test binaries. Signed-off-by: David Sterba --- Makefile | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index b597ad8e9b89..4cae30c179b5 100644 --- a/Makefile +++ b/Makefile @@ -266,12 +266,12 @@

[PATCH 1/5] btrfs-progs: build, add basic build test for library

2014-11-07 Thread David Sterba
Basic test based on snapper code that uses the send stream API. Signed-off-by: David Sterba --- Makefile | 8 +++ library-test.c | 69 ++ 2 files changed, 77 insertions(+) create mode 100644 library-test.c diff --git a/Makefile

[PATCH 3/5] btrfs-progs: build, gather utilities that are not built by default

2014-11-07 Thread David Sterba
Add them to build test and to clean rules. Signed-off-by: David Sterba --- Makefile | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 126dc593ddfb..b597ad8e9b89 100644 --- a/Makefile +++ b/Makefile @@ -50,6 +50,9 @@ progs = mkfs.btrfs btrfs-debug

[PATCH 5/5] btrfs-progs: update .gitignore

2014-11-07 Thread David Sterba
Signed-off-by: David Sterba --- .gitignore | 4 1 file changed, 4 insertions(+) diff --git a/.gitignore b/.gitignore index fc8c07a22f7f..e637b170714b 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ *.o *.static.o *.o.d +tags .cc-defines.h version.h version @@ -30,8 +31,11 @@

[PATCH 0/5] Btrfs-progs updates to build

2014-11-07 Thread David Sterba
We've managed to break build of snapper two times now, a new make rule 'test-build' should help us to check the build. Static build is not required to pass due to dependency on static libs that may not be widely available. David Sterba (5): btrfs-progs: build, add basic build test for library

[PATCH 3/4] btrfs: cleanup, rename a few variables in btrfs_read_sys_array

2014-11-07 Thread David Sterba
There's a pointer to buffer, integer offset and offset passed as pointer, try to find matching names for them. Signed-off-by: David Sterba --- fs/btrfs/volumes.c | 31 --- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/vo

[PATCH 2/4] btrfs: add checks for sys_chunk_array sizes

2014-11-07 Thread David Sterba
Verify that possible minimum and maximum size is set, validity of contents is checked in btrfs_read_sys_array. Signed-off-by: David Sterba --- fs/btrfs/disk-io.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index e97fa11488f1..7

[PATCH 1/4] btrfs: more superblock checks, lower bounds on devices and sectorsize/nodesize

2014-11-07 Thread David Sterba
I received a few crafted images from Jiri, all got through the recently added superblock checks. The lower bounds checks for num_devices and sector/node -sizes were missing and caused a crash during mount. Tools for symbolic code execution were used to prepare the images contents. Reported-by: Ji

[PATCH 4/4] btrfs: add more checks to btrfs_read_sys_array

2014-11-07 Thread David Sterba
Verify that the sys_array has enough bytes to read the next item. Signed-off-by: David Sterba --- fs/btrfs/volumes.c | 26 +++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index fbdfed2e0ba8..92f4f011882e 100644 ---

[PATCH 0/4] More mount-time checks

2014-11-07 Thread David Sterba
This series adds more superblock checks that turned out not to be enough after "btrfs: add more superblock checks". Would be good to get them into 3.18 as I'd like to submit them to stable kernels as well. This is not easily possible right now as it depends on a patch that's not merged yet (mention

corruption, bad block, input/output errors - do i run --repair?

2014-11-07 Thread Matt McKinnon
Hi All, I'm running into some corruption and I wanted to seek out advice on whether or not to run btrfs check --repair, or if I should fall back to my backup file server, or both. The system is mountable, and usable. # uname -a Linux cbmm-fs 3.17.2-custom #1 SMP Thu Oct 30 14:09:57 EDT 2014

Re: [PATCH] Btrfs: avoid crash when overflowing a dir_item

2014-11-07 Thread Filipe David Manana
On Fri, Nov 7, 2014 at 12:15 PM, Filipe Manana wrote: > When attempting to insert a new dir_item, we were calling btrfs_extent_item() > without checking if the leaf has enough space to extend the item. This made > btrfs_extent_item() crash through a BUG() call. > Therefore do the check and return

[PATCH] Btrfs: avoid crash when overflowing a dir_item

2014-11-07 Thread Filipe Manana
When attempting to insert a new dir_item, we were calling btrfs_extent_item() without checking if the leaf has enough space to extend the item. This made btrfs_extent_item() crash through a BUG() call. Therefore do the check and return ENOSPC if the leaf doesn't have enough space. Signed-off-by: F

Re: non-atomic xattr replacement in btrfs => rsync random errors

2014-11-07 Thread Filipe David Manana
On Fri, Nov 7, 2014 at 12:37 AM, Alexandre Oliva wrote: > A few days ago, I started using rsync batches to archive old copies of > ceph OSD snapshots for certain kinds of disaster recovery. This seems > to exercise an unexpected race condition in rsync, which happens to > expose what appears to b

Re: non-atomic xattr replacement in btrfs => rsync random errors

2014-11-07 Thread Holger Hoffstätte
On Thu, 06 Nov 2014 23:39:02 -0200, Alexandre Oliva wrote: > [dropping rs...@lists.samba.org, it rejects posts from non-subscribers; > refer to https://bugzilla.samba.org/show_bug.cgi?id=10925 instead] > > On Nov 6, 2014, Alexandre Oliva wrote: > >> What makes the problem visible is that btrf