question:replace

2013-08-29 Thread lilofile
The replace operation depends on scrub and makes use of the scrub code. And scrub does not yet support RAID5/6. Therefore 'btrfs replace start' fails with EINVAL on RAID5/6 filesystems. like linux md raid5 support rescontruct ,btrfs raid5 has plan to support resconstructing? -- To unsubscribe

[PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed.

2013-08-29 Thread Alex Lyakas
caching_thread()s do all their work under read access to extent_commit_sem. They give up on this read access only when need_resched() tells them, or when they exit. As a result, somebody that wants a WRITE access to this sem, might wait for a long time. Especially this is problematic in

Re: bug

2013-08-29 Thread David Sterba
On Thu, Aug 29, 2013 at 10:34:00AM +0800, lilofile wrote: I made a btrfs on five disks using RAID5 (-d raid5 for mount option). When a power failure occurs, I can not remount btrfs after my system reboots. Dmesg for remount is presented as following: Please note that current implementation

Re: [PATCH] Btrfs: allocate the free space by the existed max extent size when ENOSPC

2013-08-29 Thread David Sterba
On Thu, Aug 29, 2013 at 01:47:38PM +0800, Miao Xie wrote: By the current code, if the requested size is very large, and all the extents in the free space cache are small, we will waste lots of the cpu time to cut the requested size in half and search the cache again and again until it gets

[PATCH] Btrfs: optimize key searches in btrfs_search_slot

2013-08-29 Thread Filipe David Borba Manana
When the binary search returns 0 (exact match), the target key will necessarily be at slot 0 of all nodes below the current one, so in this case the binary search is not needed because it will always return 0, and we waste time doing it, holding node locks for longer than necessary, etc. Below

Re: [PATCH] xfstests: btrfs/003: regression test for subvol delete V2

2013-08-29 Thread David Sterba
On Tue, Aug 13, 2013 at 02:38:05PM -0500, Eric Sandeen wrote: +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch + +_scratch_mkfs /dev/null 21 +_scratch_mount + +# This will be set to subvolid 256.

[PATCH v2] Btrfs: optimize key searches in btrfs_search_slot

2013-08-29 Thread Filipe David Borba Manana
When the binary search returns 0 (exact match), the target key will necessarily be at slot 0 of all nodes below the current one, so in this case the binary search is not needed because it will always return 0, and we waste time doing it, holding node locks for longer than necessary, etc. Below

Re: [PATCH] Btrfs: optimize key searches in btrfs_search_slot

2013-08-29 Thread Josef Bacik
On Thu, Aug 29, 2013 at 01:44:13PM +0100, Filipe David Borba Manana wrote: When the binary search returns 0 (exact match), the target key will necessarily be at slot 0 of all nodes below the current one, so in this case the binary search is not needed because it will always return 0, and we

Re: [PATCH] Btrfs: optimize key searches in btrfs_search_slot

2013-08-29 Thread Filipe David Manana
On Thu, Aug 29, 2013 at 2:49 PM, Josef Bacik jba...@fusionio.com wrote: On Thu, Aug 29, 2013 at 01:44:13PM +0100, Filipe David Borba Manana wrote: When the binary search returns 0 (exact match), the target key will necessarily be at slot 0 of all nodes below the current one, so in this case

[PATCH v3] Btrfs: optimize key searches in btrfs_search_slot

2013-08-29 Thread Filipe David Borba Manana
When the binary search returns 0 (exact match), the target key will necessarily be at slot 0 of all nodes below the current one, so in this case the binary search is not needed because it will always return 0, and we waste time doing it, holding node locks for longer than necessary, etc. Below

Re: [PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed.

2013-08-29 Thread Josef Bacik
On Thu, Aug 29, 2013 at 01:31:05PM +0300, Alex Lyakas wrote: caching_thread()s do all their work under read access to extent_commit_sem. They give up on this read access only when need_resched() tells them, or when they exit. As a result, somebody that wants a WRITE access to this sem, might

Re: [PATCH 0/2] btrfs-progs: prevent mkfs from aborting with small volume

2013-08-29 Thread Eric Sandeen
On 8/28/13 12:01 AM, Hidetoshi Seto wrote: (2013/08/26 23:23), Eric Sandeen wrote: Thanks for looking into this - how small of a device did you test? I tried a 2MB device w/ these 2 patches and still got: [btrfs-progs]# truncate --size=2m testfile [btrfs-progs]# ./mkfs.btrfs testfile

Re: [PATCH] xfstests: btrfs/003: regression test for subvol delete V2

2013-08-29 Thread Eric Sandeen
On 8/29/13 8:21 AM, David Sterba wrote: On Tue, Aug 13, 2013 at 02:38:05PM -0500, Eric Sandeen wrote: +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs btrfs +_supported_os Linux +_require_scratch + +_scratch_mkfs /dev/null 21 +_scratch_mount + +# This

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

2013-08-29 Thread Zach Brown
If those fail, then look in dmesg for errors relating to the log tree -- if that's corrupt and can't be read (or causes a crash), use btrfs-zero-log. In a bit of a tangent: btrfs-zero-log throws away data that fsync/sync could have previously claimed was stable on disk. Given how often

[PATCH] Btrfs: allow partial ordered extent completion

2013-08-29 Thread Josef Bacik
We currently have this problem where you can truncate pages that have not yet been written for an ordered extent. We do this because the truncate will be coming behind to clean us up anyway so what's the harm right? Well if truncate fails for whatever reason we leave an orphan item around for

[PATCH] Btrfs: add support for asserts V2

2013-08-29 Thread Josef Bacik
One of the complaints we get a lot is how many BUG_ON()'s we have. So to help with this I'm introducing a kconfig option to enable/disable a new ASSERT() mechanism much like what XFS does. This will allow us developers to still get our nice panics but allow users/distros to compile them out.

btrfs device delete missing - why does it write on healthy device?

2013-08-29 Thread Tomasz Chmielewski
So I've removed a missing device, which took some time: # time btrfs device delete missing /home real1512m33.763s user0m0.000s sys 121m37.740s OK, it needs time, fine. And shifted quite large amounts of data: Device:tpsMB_read/sMB_wrtn/sMB_readMB_wrtn

Re: [PATCH v3] Btrfs: optimize key searches in btrfs_search_slot

2013-08-29 Thread Zach Brown
+ if (level == 0) + offset = offsetof(struct btrfs_leaf, items); + else + offset = offsetof(struct btrfs_node, ptrs); (+10 fragility points for assuming that the key starts each struct instead of using [0].key) Ok. I just copied that from

[PATCH v4] Btrfs: optimize key searches in btrfs_search_slot

2013-08-29 Thread Filipe David Borba Manana
When the binary search returns 0 (exact match), the target key will necessarily be at slot 0 of all nodes below the current one, so in this case the binary search is not needed because it will always return 0, and we waste time doing it, holding node locks for longer than necessary, etc. Below

[PATCH] Btrfs: separate out tests into their own directory V2

2013-08-29 Thread Josef Bacik
The plan is to have a bunch of unit tests that run when btrfs is loaded when you build with the appropriate config option. My ultimate goal is to have a test for every non-static function we have, but at first I'm going to focus on the things that cause us the most problems. To start out with

Re: [PATCH] Btrfs: allocate the free space by the existed max extent size when ENOSPC

2013-08-29 Thread Josef Bacik
On Thu, Aug 29, 2013 at 01:47:38PM +0800, Miao Xie wrote: By the current code, if the requested size is very large, and all the extents in the free space cache are small, we will waste lots of the cpu time to cut the requested size in half and search the cache again and again until it gets

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

2013-08-29 Thread Chris Murphy
On Aug 29, 2013, at 11:35 AM, Zach Brown z...@redhat.com wrote: If those fail, then look in dmesg for errors relating to the log tree -- if that's corrupt and can't be read (or causes a crash), use btrfs-zero-log. In a bit of a tangent: btrfs-zero-log throws away data that fsync/sync

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

2013-08-29 Thread Hugo Mills
On Thu, Aug 29, 2013 at 01:37:51PM -0600, Chris Murphy wrote: On Aug 29, 2013, at 11:35 AM, Zach Brown z...@redhat.com wrote: If those fail, then look in dmesg for errors relating to the log tree -- if that's corrupt and can't be read (or causes a crash), use btrfs-zero-log. In a

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

2013-08-29 Thread Chris Murphy
On Aug 29, 2013, at 1:40 PM, Hugo Mills h...@carfax.org.uk wrote: On Thu, Aug 29, 2013 at 01:37:51PM -0600, Chris Murphy wrote: Proceeding will roll back the file system to a previous state, and may cause the loss of successfully written data. Proceed? (Y/N) ... the loss of up to the

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

2013-08-29 Thread Hugo Mills
On Thu, Aug 29, 2013 at 01:44:54PM -0600, Chris Murphy wrote: On Aug 29, 2013, at 1:40 PM, Hugo Mills h...@carfax.org.uk wrote: On Thu, Aug 29, 2013 at 01:37:51PM -0600, Chris Murphy wrote: Proceeding will roll back the file system to a previous state, and may cause the loss of

Re: [PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed.

2013-08-29 Thread Josef Bacik
On Thu, Aug 29, 2013 at 10:09:29PM +0300, Alex Lyakas wrote: Hi Josef, On Thu, Aug 29, 2013 at 5:38 PM, Josef Bacik jba...@fusionio.com wrote: On Thu, Aug 29, 2013 at 01:31:05PM +0300, Alex Lyakas wrote: caching_thread()s do all their work under read access to extent_commit_sem. They

Re: btrfs:async-thread: atomic_start_pending=1 is set, but it's too late

2013-08-29 Thread Josef Bacik
On Mon, Aug 26, 2013 at 05:16:42PM +0300, Alex Lyakas wrote: Greetings all, I see a following issue with spawning new threads for btrfs_workers that have atomic_worker_start set: # I have BTRFS that has 24Gb of total metadata, out of which extent tree takes 11Gb; space_cache option is not

Re: btrfs:async-thread: atomic_start_pending=1 is set, but it's too late

2013-08-29 Thread Chris Mason
Quoting Josef Bacik (2013-08-29 16:03:06) On Mon, Aug 26, 2013 at 05:16:42PM +0300, Alex Lyakas wrote: Greetings all, I see a following issue with spawning new threads for btrfs_workers that have atomic_worker_start set: # I have BTRFS that has 24Gb of total metadata, out of which

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

2013-08-29 Thread Chris Murphy
On Aug 29, 2013, at 1:53 PM, Hugo Mills h...@carfax.org.uk wrote: On Thu, Aug 29, 2013 at 01:44:54PM -0600, Chris Murphy wrote: Certainly, if known for sure it won't be more than 30 seconds? Mmm... it'll depend on the setting of the commit period, which up until a couple of weeks ago

Re: Question: How can I recover this partition? (unable to find logical $hugenum len 4096)

2013-08-29 Thread Chris Murphy
On Aug 29, 2013, at 2:19 PM, Chris Murphy li...@colorremedies.com wrote: On Aug 29, 2013, at 1:53 PM, Hugo Mills h...@carfax.org.uk wrote: On Thu, Aug 29, 2013 at 01:44:54PM -0600, Chris Murphy wrote: Certainly, if known for sure it won't be more than 30 seconds? Mmm... it'll depend

[PATCH] Btrfs: only update disk_i_size as we remove extents

2013-08-29 Thread Josef Bacik
This fixes a problem where if we fail a truncate we will leave the i_size set where we wanted to truncate to instead of where we were able to truncate to. Fix this by making btrfs_truncate_inode_items do the disk_i_size update as it removes extents, that way it will always be consistent with where

[PATCH] Btrfs: don't use an async starter for most of our workers

2013-08-29 Thread Josef Bacik
We only need an async starter if we can't make a GFP_NOFS allocation in our current path. This is the case for the endio stuff since it happens in IRQ context, but things like the caching thread workers and the delalloc flushers we can easily make this allocation and start threads right away.

Re: [PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed.

2013-08-29 Thread Alex Lyakas
On Thu, Aug 29, 2013 at 10:55 PM, Josef Bacik jba...@fusionio.com wrote: On Thu, Aug 29, 2013 at 10:09:29PM +0300, Alex Lyakas wrote: Hi Josef, On Thu, Aug 29, 2013 at 5:38 PM, Josef Bacik jba...@fusionio.com wrote: On Thu, Aug 29, 2013 at 01:31:05PM +0300, Alex Lyakas wrote:

Re: btrfs:async-thread: atomic_start_pending=1 is set, but it's too late

2013-08-29 Thread Alex Lyakas
Thanks, Chris, Josef, for confirming! On Thu, Aug 29, 2013 at 11:08 PM, Chris Mason clma...@fusionio.com wrote: Quoting Josef Bacik (2013-08-29 16:03:06) On Mon, Aug 26, 2013 at 05:16:42PM +0300, Alex Lyakas wrote: Greetings all, I see a following issue with spawning new threads for

Re: [PATCH] Btrfs: separate out tests into their own directory V2

2013-08-29 Thread Wang Shilong
On 08/30/2013 03:29 AM, Josef Bacik wrote: The plan is to have a bunch of unit tests that run when btrfs is loaded when you build with the appropriate config option. My ultimate goal is to have a test for every non-static function we have, but at first I'm going to focus on the things that

Re: [PATCH 1/3] btrfs-progs: move out print in cmd_df to another function

2013-08-29 Thread Wang Shilong
On 08/16/2013 08:48 PM, Anand Jain wrote: Hello Anand, We'd appreciate you use checkpatch.pl to check coding style before sending patches. For this patch: ERROR: foo * bar should be foo *bar #35: FILE: cmds-filesystem.c:47: +static char * group_type_str(u64 flag) ERROR: foo * bar should be

Re: [PATCH 2/3] btrfs-progs: read from the kernel when disk is mounted

2013-08-29 Thread Wang Shilong
On 08/16/2013 08:48 PM, Anand Jain wrote: ERROR: spaces required around that '?' (ctx:VxV) #63: FILE: cmds-filesystem.c:279: +strlen(label)?label:none, uuidbuf, path); ^ ERROR: spaces required around that ':' (ctx:VxV) #63: FILE: cmds-filesystem.c:279: +

Re: [PATCH] Btrfs: fix printing of non NULL terminated string

2013-08-29 Thread Wang Shilong
On 08/21/2013 12:51 AM, Filipe David Borba Manana wrote: please use checkpatch.pl to check coding styles before sending patch ERROR: code indent should use tabs where possible #37: FILE: fs/btrfs/delayed-inode.c:1477: +^I^Iname_len, name,$ total: 1 errors, 0 warnings, 13 lines

Re: [PATCH] Btrfs: fix send to deal with sparse files properly V2

2013-08-29 Thread Wang Shilong
On 08/22/2013 11:47 PM, Josef Bacik wrote: please use checkpatch.pl to check coding styles before sending patches: WARNING: line over 80 characters #86: FILE: fs/btrfs/send.c:4051: +if (btrfs_file_extent_disk_bytenr(path-nodes[0], ei) == 0) { total: 0 errors, 1 warnings, 59 lines

Re: [PATCH 1/3] btrfs-progs: move out print in cmd_df to another function

2013-08-29 Thread Anand Jain
Hi Wang, apologies it didn't pass checkpatch.pl. will fix them. Thanks, Anand On 08/30/2013 10:01 AM, Wang Shilong wrote: On 08/16/2013 08:48 PM, Anand Jain wrote: Hello Anand, We'd appreciate you use checkpatch.pl to check coding style before sending patches. For this patch: ERROR: