[PATCH] xfstests: btrfs/276 - stop all fsstress before exiting

2013-04-25 Thread Eric Sandeen
Tests after 276 were failing because the background fsstress hadn't quit prior to exit, devices couldn't be unmounted, etc. Just use the same trick as generic/068 does, and use a tmpfile to control whether the background loop keeps running. Also, no need to umount scratch at cleanup time, the scr

[PATCH v2] btrfs-progs: mkfs seg fault for wrong free

2013-04-25 Thread Anand Jain
With commit 87c09f7 Btrfs-progs: fix memory leaks on cleanup mkfs on multiple dev is ending with segfault at close_all_devices() during kfree(device->name) because mkfs calls btrfs_add_to_fsid, which does not initialize name when dev is added to the list. Signed-off-by: Anand Jain ---

Re: [BULK] Re: [PATCH] xfstests 311: test fsync with dm flakey V2

2013-04-25 Thread Dave Chinner
On Thu, Apr 25, 2013 at 09:32:37PM -0400, Josef Bacik wrote: > On Thu, Apr 25, 2013 at 07:08:29PM -0600, Dave Chinner wrote: > > On Thu, Apr 25, 2013 at 08:24:04PM -0400, Josef Bacik wrote: > > > On Thu, Apr 25, 2013 at 04:45:56PM -0600, Dave Chinner wrote: > > > > On Thu, Apr 25, 2013 at 10:12:56A

Re: [PATCH v4 2/3] Btrfs: rescan for qgroups

2013-04-25 Thread Wang Shilong
> If qgroup tracking is out of sync, a rescan operation can be started. It > iterates the complete extent tree and recalculates all qgroup tracking data. > This is an expensive operation and should not be used unless required. > > A filesystem under rescan can still be umounted. The rescan continu

Re: [BULK] Re: [PATCH] xfstests 311: test fsync with dm flakey V2

2013-04-25 Thread Josef Bacik
On Thu, Apr 25, 2013 at 07:08:29PM -0600, Dave Chinner wrote: > On Thu, Apr 25, 2013 at 08:24:04PM -0400, Josef Bacik wrote: > > On Thu, Apr 25, 2013 at 04:45:56PM -0600, Dave Chinner wrote: > > > On Thu, Apr 25, 2013 at 10:12:56AM -0400, Josef Bacik wrote: > . > > > > + $here/src/fsync-t

Re: [PATCH v4 1/3] Btrfs: split btrfs_qgroup_account_ref into four functions

2013-04-25 Thread Wang Shilong
Hello Jan, > The function is separated into a preparation part and the three accounting > steps mentioned in the qgroups documentation. The goal is to make steps two > and three usable by the rescan functionality. A side effect is that the > function is restructured into readable subunits. > In

Re: [BULK] Re: [PATCH] xfstests 311: test fsync with dm flakey V2

2013-04-25 Thread Dave Chinner
On Thu, Apr 25, 2013 at 08:24:04PM -0400, Josef Bacik wrote: > On Thu, Apr 25, 2013 at 04:45:56PM -0600, Dave Chinner wrote: > > On Thu, Apr 25, 2013 at 10:12:56AM -0400, Josef Bacik wrote: . > > > + $here/src/fsync-tester -s $SEED -r -t $test_num $extra $testfile > > > + if [ $? -ne 0 ]; then

Re: [BULK] Re: [PATCH] xfstests 311: test fsync with dm flakey V2

2013-04-25 Thread Josef Bacik
On Thu, Apr 25, 2013 at 04:45:56PM -0600, Dave Chinner wrote: > On Thu, Apr 25, 2013 at 10:12:56AM -0400, Josef Bacik wrote: > > This test sets up a dm flakey target and then runs my fsync tester I've been > > using to verify btrfs's fsync() is working properly. It will create a dm > > flakey > >

Re: [PATCH] xfstests 311: test fsync with dm flakey V2

2013-04-25 Thread Dave Chinner
On Thu, Apr 25, 2013 at 10:12:56AM -0400, Josef Bacik wrote: > This test sets up a dm flakey target and then runs my fsync tester I've been > using to verify btrfs's fsync() is working properly. It will create a dm > flakey > device, mount it, run my test, make the flakey device start dropping wr

Re: [PATCH] btrfs: handle errors returned from get_tree_block_key

2013-04-25 Thread Zach Brown
On Wed, Apr 24, 2013 at 06:51:34PM +0200, David Sterba wrote: > Signed-off-by: David Sterba > --- > fs/btrfs/relocation.c | 7 +-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c > index c22ccfe..e85be23 100644 > --- a/fs/btrfs

Re: One random read streaming is fast (~1200MB/s), but two or more are slower (~750MB/s)?

2013-04-25 Thread Josef Bacik
On Thu, Apr 25, 2013 at 03:01:18PM -0600, Matt Pursley wrote: > Ok, awesome, let me know how it goes.. I don't have the raid > formatted to btrfs right now, but I could probably do that in about 30 > minutes or so. > Huh so I'm getting the full bandwidth, 120 mb/s with one thread and 60 mb/s wit

Re: One random read streaming is fast (~1200MB/s), but two or more are slower (~750MB/s)?

2013-04-25 Thread Matt Pursley
Ok, awesome, let me know how it goes.. I don't have the raid formatted to btrfs right now, but I could probably do that in about 30 minutes or so. Thanks Josef, Matt On Thu, Apr 25, 2013 at 1:39 PM, Josef Bacik wrote: > On Thu, Apr 25, 2013 at 01:52:44PM -0600, Matt Pursley wrote: >> Hey Jo

[PATCH V2] btrfs: make static code static & remove dead code

2013-04-25 Thread Eric Sandeen
Big patch, but all it does is add statics to functions which are in fact static, then remove the associated dead-code fallout. removed functions: btrfs_iref_to_path() __btrfs_lookup_delayed_deletion_item() __btrfs_search_delayed_insertion_item() __btrfs_search_delayed_deletion_item() find_eb_for_

Re: One random read streaming is fast (~1200MB/s), but two or more are slower (~750MB/s)?

2013-04-25 Thread Josef Bacik
On Thu, Apr 25, 2013 at 01:52:44PM -0600, Matt Pursley wrote: > Hey Josef, > > Were you able to look into this any further? > It's still pretty reproducible on my machine... > Nope I've been tracking down random problems, I'll try it now. Thanks, Josef -- To unsubscribe from this list: send th

[PATCH] Btrfs: remove almost all of the BUG()'s from tree-log.c

2013-04-25 Thread Josef Bacik
There were a whole bunch and I was doing it for other things. I haven't tested these error paths but at the very least this is better than panicing. I've only left 2 BUG_ON()'s since they are logic errors and I want to replace them with a ASSERT framework that we can compile out for production us

[PATCH] Btrfs: deal with free space cache errors while replaying log

2013-04-25 Thread Josef Bacik
So everybody who got hit by my fsync bug will still continue to hit this BUG_ON() in the free space cache, which is pretty heavy handed. So I took a file system that had this bug and fixed up all the BUG_ON()'s and leaks that popped up when I tried to mount a broken file system like this. With th

Re: One random read streaming is fast (~1200MB/s), but two or more are slower (~750MB/s)?

2013-04-25 Thread Matt Pursley
Hey Josef, Were you able to look into this any further? It's still pretty reproducible on my machine... Thanks, Matt On Thu, Apr 18, 2013 at 2:58 PM, Josef Bacik wrote: > This is strange, and I can't see any reason why this would happen. I'll try > and > reproduce next week when I'm bac

Re: [PATCH] Btrfs: improve the loop of scrub_stripe

2013-04-25 Thread Josef Bacik
On Thu, Apr 18, 2013 at 10:54:55AM -0600, Liu Bo wrote: > 1) Right now scrub_stripe() is looping in some unnecessary cases: > * when the found extent item's objectid has been out of the dev extent's range > but we haven't finish scanning all the range within the dev extent > * when all the items

[PATCH] Btrfs: various abort cleanups

2013-04-25 Thread Josef Bacik
I have a broken file system that when it aborts leaves all sorts of accounting things wrong and gives you lots of WARN_ON()'s other than the abort. This is because we're not cleaning up various parts of the file system when we abort. The first chunks are specific to mount failures, we weren't clea

Re: [bug] 3.9-rc7+next: NULL deref in btrfs_next_old_leaf/btrfs_search_slot

2013-04-25 Thread David Sterba
On Thu, Apr 18, 2013 at 04:42:18PM +0200, David Sterba wrote: > [64394.422743] BUG: unable to handle kernel NULL pointer dereference at > 0078 > [64394.426716] RIP: 0010:[] [] > btrfs_search_slot+0xbf/0x9e0 [btrfs] > [64394.426716] [] btrfs_next_old_leaf+0x247/0x4e0 [btrfs] > [64394

[PATCH v4 2/3] Btrfs: rescan for qgroups

2013-04-25 Thread Jan Schmidt
If qgroup tracking is out of sync, a rescan operation can be started. It iterates the complete extent tree and recalculates all qgroup tracking data. This is an expensive operation and should not be used unless required. A filesystem under rescan can still be umounted. The rescan continues on the

[PATCH v4 1/3] Btrfs: split btrfs_qgroup_account_ref into four functions

2013-04-25 Thread Jan Schmidt
The function is separated into a preparation part and the three accounting steps mentioned in the qgroups documentation. The goal is to make steps two and three usable by the rescan functionality. A side effect is that the function is restructured into readable subunits. Signed-off-by: Jan Schmidt

[PATCH v4 3/3] Btrfs: automatic rescan after "quota enable" command

2013-04-25 Thread Jan Schmidt
When qgroup tracking is enabled, we do an automatic cycle of the new rescan mechanism. Signed-off-by: Jan Schmidt --- fs/btrfs/qgroup.c | 11 +++ 1 files changed, 11 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index 664d457..1df4db5 100644 --- a/fs

[PATCH v4 0/3] Btrfs: quota rescan for 3.10

2013-04-25 Thread Jan Schmidt
The kernel side for rescan, which is needed if you want to enable qgroup tracking on a non-empty volume. The first patch splits btrfs_qgroup_account_ref into readable ans reusable units. The second patch adds the rescan implementation (refer to its commit message for a description of the algorithm)

Re: [PATCH v3 2/3] Btrfs: rescan for qgroups

2013-04-25 Thread Jan Schmidt
On Thu, April 25, 2013 at 04:16 (+0200), Wang Shilong wrote: > I just have an example in my mind, considering the following example: > > qgroup(1/1) > / \ >/\ > subv(257) snapsho

[PATCH] xfstests 311: test fsync with dm flakey V2

2013-04-25 Thread Josef Bacik
This test sets up a dm flakey target and then runs my fsync tester I've been using to verify btrfs's fsync() is working properly. It will create a dm flakey device, mount it, run my test, make the flakey device start dropping writes, and then unmount the fs. Then we mount it back up and make sure

Re: [PATCH] Btrfs: only exclude supers in the range of our block group V2

2013-04-25 Thread David Sterba
On Thu, Apr 25, 2013 at 09:04:13AM -0400, Josef Bacik wrote: > > Sorry, I still didn't get how this happens... > > > > I'll try to create new btrfs with raid0, raid1, raid10, raid5, raid6... > > > > Could you please show me the testcase or something so that I can persuade > > myself? > > > > Ok

Re: [PATCH] Btrfs: only exclude supers in the range of our block group V2

2013-04-25 Thread Josef Bacik
On Wed, Apr 24, 2013 at 09:48:27PM -0600, Liu Bo wrote: > On Wed, Apr 24, 2013 at 10:48:09AM -0400, Josef Bacik wrote: > > On Wed, Apr 24, 2013 at 08:43:21AM -0600, Liu Bo wrote: > > > On Wed, Apr 24, 2013 at 09:00:16AM -0400, Josef Bacik wrote: > > > > On Wed, Apr 24, 2013 at 02:57:40AM -0600, Liu

Re: [PATCH] btrfs-progs: mkfs seg fault for wrong free

2013-04-25 Thread David Sterba
On Thu, Apr 25, 2013 at 08:15:42PM +0800, Anand Jain wrote: > --- a/disk-io.c > +++ b/disk-io.c > @@ -1290,8 +1290,10 @@ static int close_all_devices(struct btrfs_fs_info > *fs_info) > fprintf(stderr, "Warning, could not drop > caches\n"); > } >

[PATCH] btrfs-progs: mkfs seg fault for wrong free

2013-04-25 Thread Anand Jain
With commit 87c09f7 Btrfs-progs: fix memory leaks on cleanup mkfs on multiple dev is ending with segfault at close_all_devices() during kfree(device->name) because mkfs calls btrfs_add_to_fsid, which does not initialize name when dev is added to the list. Signed-off-by: Anand Jain ---

Re: [PATCH 2/2] Btrfs: introduce noextiref mount option

2013-04-25 Thread Miao Xie
On Mon, 15 Apr 2013 19:20:51 +0200, David Sterba wrote: > On Fri, Apr 12, 2013 at 12:01:19PM -0500, Eric Sandeen wrote: >> On 4/11/13 5:35 AM, Miao Xie wrote: >>> Now, we set incompat flag EXTEND_IREF when we actually need insert a >>> extend inode reference, not when making a fs. But some users ma

[PATCH] Btrfs: allocate new chunks if the space is not enough for global rsv

2013-04-25 Thread Miao Xie
When running the 208th of xfstests, the fs returned the enospc error when there was lots of free space in the disk. By bisect debug, we found it was introduced by commit 96f1bb5777. This commit makes the space check for the global reservation in can_overcommit() be inconsistent with should_alloc_c