Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread youagree
This is most probably related to the same regression seen after 2.6.38, my blocked comment on 3 August included an indication to that the behavior was present in my distro 2.6.38 kernel too, it just was appearing after a considerably longer uptime (on my desktop system using btrfs as rootfs on an

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Chris Samuel
On 18/08/11 00:29, Michael Cronenworth wrote: I'm running kernel 3.0 (Fedora 15's 2.6.40) on two boxes and I have not seen slow downs or hangs. I use Firefox. I've got btrfs on an external USB drive with the 3.0.1 kernel and I see that sync seems to take an age, according to iotop it seems

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread youagree
Are these processes principally btrfs-submit and btrfs-transacti in particular? Then it may be related to my very similar issue reported earlier. On 08/18/2011 08:47 AM, Chris Samuel wrote: On 18/08/11 00:29, Michael Cronenworth wrote: I'm running kernel 3.0 (Fedora 15's 2.6.40) on two

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Andrew Guertin
On 08/18/2011 02:44 AM, youagree wrote: Also, a patch by Josef Bacik was an attempt for fixing this, but no one reported about testing it on an affected system, it did not eliminate the slowdowns for me: PLEASE TEST: Everybody who is seeing weird and long hangs

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Andrew Guertin
On 08/17/2011 10:41 PM, Anand Jain wrote: Dave, good to have a test case on the 3.0 kernel. do you have btrfs as root fs ? and can you show how are you using the btrfs mainly I would need 'btrfs fi show' let me try if I can reproduce. Thanks, Anand Personally, I find that large compiles are

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread youagree
On 08/18/2011 09:29 AM, Andrew Guertin wrote: * Many processes occasionally hang for a short time * When this happens, my cpu monitor shows a short burst of cpu activity (100% of 1 core) followed by a longer period of IO * When this happens, iotop shows [btrfs-submit-0] and [btrfs-transacti]

[PATCH 1/3] Btrfs: fix wrong nbytes information of the inode

2011-08-18 Thread Miao Xie
If we write some data into the data hole of the file(no preallocation for this hole), Btrfs will allocate some disk space, and update nbytes of the inode, but the other element--disk_i_size needn't be updated. At this condition, we must update inode metadata though disk_i_size is not

[PATCH 2/3] Btrfs: fix the file extent gap when doing direct IO

2011-08-18 Thread Miao Xie
When we write some data to the place that is beyond the end of the file in direct I/O mode, a data hole will be created. And Btrfs should insert a file extent item that point to this hole into the fs tree. But unfortunately Btrfs forgets doing it. The following is a simple way to reproduce it: #

[PATCH 3/3] Btrfs: fix unclosed transaction handle in btrfs_cont_expand()

2011-08-18 Thread Miao Xie
The function - btrfs_cont_expand() forgot to close the transaction handle before it jump out the while loop. Fix it. Signed-off-by: Miao Xie mi...@cn.fujitsu.com --- fs/btrfs/inode.c |8 ++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c

Re: [PATCH 1/3] Btrfs: fix wrong nbytes information of the inode

2011-08-18 Thread Miao Xie
On thu, 18 Aug 2011 16:11:31 +0800, Miao Xie wrote: If we write some data into the data hole of the file(no preallocation for this hole), Btrfs will allocate some disk space, and update nbytes of the inode, but the other element--disk_i_size needn't be updated. At this condition, we must

Re: [RFC] btrfs auto snapshot

2011-08-18 Thread Anand Jain
Thanks MgE. snapper is cool, does most the stuff required here. however the challenging part will be to keep the number of tools (to manage btrfs) at a limit 1 or 2 max. (too many tools to manage btrfs is most likely to confuse). Cheers, Anand On 08/17/2011 09:31 PM, Matthias G.

Re: [RFC] btrfs auto snapshot

2011-08-18 Thread Anand Jain
David, I think that you need to be careful not to impose your idea of when to take snapshots and how long to keep them onto the design. For example why take snapshots every 15 minutes? Why not every 10 or every hour? crontab is anyways changeable by the admin, I think we can have that

Re: snapshot ctime // Re: [RFC] btrfs auto snapshot

2011-08-18 Thread Anand Jain
On 08/17/2011 11:56 PM, Jérôme Poulin wrote: On Wed, Aug 17, 2011 at 11:13 AM, Roman Mamedovr...@romanrm.ru wrote: So until someone cares about snapshot ctime enough to fix this, btrfs will not be a convenient FS to work with timed snapshotting/cleanup. Isn't the ctime the creation date

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Andrew Guertin
On 08/18/2011 03:29 AM, Andrew Guertin wrote: I have not seen slowdowns on 2.6.38. More specifically, I observe the following behaviors after commit 4e69b59: * Many processes occasionally hang for a short time * When this happens, my cpu monitor shows a short burst of cpu activity (100% of

[PATCH 5/5] btrfs: use fs netlink interface for ENOSPC conditions

2011-08-18 Thread Lukas Czerner
Register fs netlink interface and send proper warning if ENOSPC is encountered. Note that we differentiate between enospc for metadata and enospc for data. Signed-off-by: Lukas Czerner lczer...@redhat.com Cc: Chris Mason chris.ma...@oracle.com Cc: linux-btrfs@vger.kernel.org ---

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Chris Mason
Excerpts from Andrew Guertin's message of 2011-08-11 21:13:18 -0400: On 08/09/2011 05:29 PM, Andrew Guertin wrote: I have not tried 3.1-rc1, but plan to soon. I've tested now, this does still occur in 3.1-rc1. Ok, I had high hopes that the btrfs changes in rc1 would fix this. Could you

Re: processes stuck in llseek

2011-08-18 Thread Mitch Harder
On Tue, Aug 16, 2011 at 8:23 PM, Li Zefan l...@cn.fujitsu.com wrote: Dan Merillat wrote: On Tue, Aug 16, 2011 at 8:51 AM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Dan Merillat's message of 2011-08-15 23:59:50 -0400: Dan Carpenter sent a patch for this, I'll get it queued up for

Re: processes stuck in llseek

2011-08-18 Thread Chris Mason
Excerpts from Mitch Harder's message of 2011-08-18 11:40:17 -0400: On Tue, Aug 16, 2011 at 8:23 PM, Li Zefan l...@cn.fujitsu.com wrote: Dan Merillat wrote: On Tue, Aug 16, 2011 at 8:51 AM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Dan Merillat's message of 2011-08-15

Re: processes stuck in llseek

2011-08-18 Thread Mitch Harder
On Thu, Aug 18, 2011 at 11:00 AM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Mitch Harder's message of 2011-08-18 11:40:17 -0400: On Tue, Aug 16, 2011 at 8:23 PM, Li Zefan l...@cn.fujitsu.com wrote: Dan Merillat wrote: On Tue, Aug 16, 2011 at 8:51 AM, Chris Mason

Re: processes stuck in llseek

2011-08-18 Thread Chris Mason
Excerpts from Mitch Harder's message of 2011-08-18 12:53:33 -0400: On Thu, Aug 18, 2011 at 11:00 AM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Mitch Harder's message of 2011-08-18 11:40:17 -0400: On Tue, Aug 16, 2011 at 8:23 PM, Li Zefan l...@cn.fujitsu.com wrote: Dan

Re: Filesystem corrupt after renaming snapshots.

2011-08-18 Thread David Sterba
Hi, On Wed, Aug 10, 2011 at 08:38:59PM +1200, Ralph Loader wrote: Hi, Recently I suffered from a badly corrupted btrfs filesystem. I had several snapshots in /snap that I moved into / (using /bin/mv). After that, attempting to access the ls the snapshot resulted in the ls process

Re: [PATCH 5/5] btrfs: use fs netlink interface for ENOSPC conditions

2011-08-18 Thread David Sterba
Hi, I see you are mixing a cleanup while adding a new feature. On Thu, Aug 18, 2011 at 02:18:26PM +0200, Lukas Czerner wrote: Register fs netlink interface and send proper warning if ENOSPC is encountered. Note that we differentiate between enospc for metadata and enospc for data.

Re: Filesystem corrupt after renaming snapshots.

2011-08-18 Thread Josef Bacik
On Thu, Aug 18, 2011 at 07:53:46PM +0200, David Sterba wrote: Hi, On Wed, Aug 10, 2011 at 08:38:59PM +1200, Ralph Loader wrote: Hi, Recently I suffered from a badly corrupted btrfs filesystem. I had several snapshots in /snap that I moved into / (using /bin/mv). After that,

Re: [PATCH 5/5] btrfs: use fs netlink interface for ENOSPC conditions

2011-08-18 Thread Lukas Czerner
On Thu, 18 Aug 2011, David Sterba wrote: Hi, I see you are mixing a cleanup while adding a new feature. On Thu, Aug 18, 2011 at 02:18:26PM +0200, Lukas Czerner wrote: Register fs netlink interface and send proper warning if ENOSPC is encountered. Note that we differentiate between

Re: Honest timeline for btrfsck

2011-08-18 Thread Chris Mason
Excerpts from Yalonda Gishtaka's message of 2011-08-17 21:09:37 -0400: Chris Mason chris.mason at oracle.com writes: Aside from making sure the kernel code is stable, btrfsck is all I'm working on right now. I do expect a release in the next two weeks that can recover your data (and

Re: Honest timeline for btrfsck

2011-08-18 Thread Hugo Mills
On Thu, Aug 18, 2011 at 04:50:08PM -0400, Chris Mason wrote: I've been working non-stop on this. Currently fsck has four parts: This all looks like great stuff. Can't wait to try it out... One thing strikes me for purposes of automated testing and regression testing: Do you have tools or

Re: [GIT PULL] Btrfs updates

2011-08-18 Thread Sage Weil
Hi Chris, Josef, Can some form of the clone ioctl transaction start reservation fix go in soon as well? That hits a BUG_ON every time. Thanks! sage On Thu, 18 Aug 2011, Chris Mason wrote: Hi everyone, The for-linus branch of the btrfs-unstable tree:

[PATCH 0/8] btrfs: Error handling fixes

2011-08-18 Thread Mark Fasheh
Hi, The following are assorted fixes to error handling from all parts of the Btrfs code. Every patch in this series stands on it's own, with the exception of the last patch which relies on the one before it (so patches 7 and 8 can be considered a pair). I also included in this series an

[PATCH 1/8] btrfs: Don't BUG_ON errors from btrfs_create_subvol_root()

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh mfas...@suse.com This is called from only one place - create_subvol() which passes errors safely back out to it's caller, btrfs_mksubvol where they are handled. Additionally, btrfs_create_subvol_root() itself bug's needlessly from error return of btrfs_update_inode(). Since

[PATCH 2/8] btrfs: Don't BUG_ON() errors in update_ref_for_cow()

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh mfas...@suse.com The only caller of update_ref_for_cow() is __btrfs_cow_block() which was originally ignoring any return values. update_ref_for_cow() however doesn't look like a candidate to become a void function - there are a few places where errors can occur. So instead I

[PATCH 3/8] btrfs: Don't BUG_ON kzalloc error in btrfs_lookup_csums_range()

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh mfas...@suse.com Unfortunately it isn't enough to just exit here - the kzalloc() happens in a loop and the allocated items are added to a linked list whose head is passed in from the caller. To fix the BUG_ON() and also provide the semantic that the list passed in is only

[PATCH 4/8] btrfs: make insert_ptr() void

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh mfas...@suse.com insert_ptr() always returns zero, so all the exta error handling can go away. This makes it trivial to also make copy_for_split() a void function as it's only return was from insert_ptr(). Finally, this all makes the BUG_ON(ret) in split_leaf() meaningless so I

[PATCH 6/8] btrfs: fix error check of btrfs_lookup_dentry()

2011-08-18 Thread Mark Fasheh
From: Tsutomu Itoh t-i...@jp.fujitsu.com Clean up btrfs_lookup_dentry() to never return NULL, but PTR_ERR(-ENOENT) instead. This keeps the return value convention consistent. Callers who pass to d_instatiate() require a trivial update. create_snapshot() in particular looks like it can also lose

[PATCH 7/8] btrfs: make fixup_low_keys() void

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh mfas...@suse.com This is trivial - fixup_low_keys always returns zero so we can make it void. As a result, we can then make setup_items_for_insert() void too which lets us cut out a couple of BUG_ON(ret) lines. Signed-off-by: Mark Fasheh mfas...@suse.de --- fs/btrfs/ctree.c

[PATCH 5/8] btrfs: Don't BUG_ON errors in __finish_chunk_alloc()

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh mfas...@suse.com All callers of __finish_chunk_alloc() BUG_ON() return value, so it's trivial for us to always bubble up any errors caught in __finish_chunk_alloc() to be caught there. Signed-off-by: Mark Fasheh mfas...@suse.de --- fs/btrfs/volumes.c |7 ++- 1 files

[PATCH 8/8] btrfs: make del_ptr() and btrfs_del_leaf() void

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh mfas...@suse.com Since fixup_low_keys() has been made void, del_ptr() always returns zero. We can then make it void as well. This allows us in turn to make btrfs_del_leaf() void as the only return value it was previously catching was from del_ptr(). This winds up removing a

Re: kernel oops - no mount (different problem than others)

2011-08-18 Thread cwillu
You might try mounting it -o ro as a stopgap to regain readonly access. Judging from the bootlog, the error itself appears to be enospc. In which case there's no already-available quick fix; I expect a developer to chime in any second now :p From the logs it is listing a transid error but NOT

Re: [PATCH] Btrfs: check if there is enough space for balancing smarter

2011-08-18 Thread David Sterba
Hi, too late, already pulled On Wed, Aug 03, 2011 at 06:15:25PM +0800, Liu Bo wrote: --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -6682,6 +6682,10 @@ int btrfs_can_relocate(struct btrfs_root *root, u64 bytenr) struct btrfs_space_info *space_info; struct