Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Andrew Guertin
On 08/18/2011 02:44 AM, youagree wrote: Also, a patch by Josef Bacik was an attempt for fixing this, but no one reported about testing it on an affected system, it did not eliminate the slowdowns for me: PLEASE TEST: Everybody who is seeing weird and long hangs news://news.gmane.org:119/4e36c47e

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Andrew Guertin
On 08/17/2011 10:41 PM, Anand Jain wrote: Dave, good to have a test case on the 3.0 kernel. do you have btrfs as root fs ? and can you show how are you using the btrfs mainly I would need 'btrfs fi show' let me try if I can reproduce. Thanks, Anand Personally, I find that large compiles are v

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread youagree
On 08/18/2011 09:29 AM, Andrew Guertin wrote: > * Many processes occasionally hang for a short time > * When this happens, my cpu monitor shows a short burst of cpu activity > (100% of 1 core) followed by a longer period of IO > * When this happens, iotop shows [btrfs-submit-0] and [btrfs-transacti

[PATCH 1/3] Btrfs: fix wrong nbytes information of the inode

2011-08-18 Thread Miao Xie
If we write some data into the data hole of the file(no preallocation for this hole), Btrfs will allocate some disk space, and update nbytes of the inode, but the other element--disk_i_size needn't be updated. At this condition, we must update inode metadata though disk_i_size is not changed(btrfs

[PATCH 2/3] Btrfs: fix the file extent gap when doing direct IO

2011-08-18 Thread Miao Xie
When we write some data to the place that is beyond the end of the file in direct I/O mode, a data hole will be created. And Btrfs should insert a file extent item that point to this hole into the fs tree. But unfortunately Btrfs forgets doing it. The following is a simple way to reproduce it: #

[PATCH 3/3] Btrfs: fix unclosed transaction handle in btrfs_cont_expand()

2011-08-18 Thread Miao Xie
The function - btrfs_cont_expand() forgot to close the transaction handle before it jump out the while loop. Fix it. Signed-off-by: Miao Xie --- fs/btrfs/inode.c |8 ++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 634dd797..ee

Re: [PATCH 1/3] Btrfs: fix wrong nbytes information of the inode

2011-08-18 Thread Miao Xie
On thu, 18 Aug 2011 16:11:31 +0800, Miao Xie wrote: > If we write some data into the data hole of the file(no preallocation for this > hole), Btrfs will allocate some disk space, and update nbytes of the inode, > but > the other element--disk_i_size needn't be updated. At this condition, we must >

Re: [RFC] btrfs auto snapshot

2011-08-18 Thread Anand Jain
Thanks MgE. snapper is cool, does most the stuff required here. however the challenging part will be to keep the number of tools (to manage btrfs) at a limit 1 or 2 max. (too many tools to manage btrfs is most likely to confuse). Cheers, Anand On 08/17/2011 09:31 PM, Matthias G. Eckermann

Re: [RFC] btrfs auto snapshot

2011-08-18 Thread Anand Jain
David, I think that you need to be careful not to impose your idea of when to take snapshots and how long to keep them onto the design. For example why take snapshots every 15 minutes? Why not every 10 or every hour? crontab is anyways changeable by the admin, I think we can have that flex

Re: snapshot ctime // Re: [RFC] btrfs auto snapshot

2011-08-18 Thread Anand Jain
On 08/17/2011 11:56 PM, Jérôme Poulin wrote: On Wed, Aug 17, 2011 at 11:13 AM, Roman Mamedov wrote: So until someone cares about snapshot ctime enough to fix this, btrfs will not be a convenient FS to work with timed snapshotting/cleanup. Isn't the ctime the creation date of the original f

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Andrew Guertin
On 08/18/2011 03:29 AM, Andrew Guertin wrote: > I have not seen slowdowns on 2.6.38. More specifically, I observe the > following behaviors after commit 4e69b59: > > * Many processes occasionally hang for a short time > * When this happens, my cpu monitor shows a short burst of cpu activity > (100

[PATCH 5/5] btrfs: use fs netlink interface for ENOSPC conditions

2011-08-18 Thread Lukas Czerner
Register fs netlink interface and send proper warning if ENOSPC is encountered. Note that we differentiate between enospc for metadata and enospc for data. Signed-off-by: Lukas Czerner Cc: Chris Mason Cc: linux-btrfs@vger.kernel.org --- fs/btrfs/extent-tree.c | 13 ++--- fs/btrfs/supe

Re: Applications using fsync cause hangs for several seconds every few minutes

2011-08-18 Thread Chris Mason
Excerpts from Andrew Guertin's message of 2011-08-11 21:13:18 -0400: > On 08/09/2011 05:29 PM, Andrew Guertin wrote: > > I have not tried 3.1-rc1, but plan to soon. > > I've tested now, this does still occur in 3.1-rc1. Ok, I had high hopes that the btrfs changes in rc1 would fix this. Could you

Re: processes stuck in llseek

2011-08-18 Thread Mitch Harder
On Tue, Aug 16, 2011 at 8:23 PM, Li Zefan wrote: > Dan Merillat wrote: >> On Tue, Aug 16, 2011 at 8:51 AM, Chris Mason wrote: >>> Excerpts from Dan Merillat's message of 2011-08-15 23:59:50 -0400: >> >>> Dan Carpenter sent a patch for this, I'll get it queued up for rc3. >> >> Can you send it?  I

Re: processes stuck in llseek

2011-08-18 Thread Chris Mason
Excerpts from Mitch Harder's message of 2011-08-18 11:40:17 -0400: > On Tue, Aug 16, 2011 at 8:23 PM, Li Zefan wrote: > > Dan Merillat wrote: > >> On Tue, Aug 16, 2011 at 8:51 AM, Chris Mason > >> wrote: > >>> Excerpts from Dan Merillat's message of 2011-08-15 23:59:50 -0400: > >> > >>> Dan Carp

Re: processes stuck in llseek

2011-08-18 Thread Mitch Harder
On Thu, Aug 18, 2011 at 11:00 AM, Chris Mason wrote: > Excerpts from Mitch Harder's message of 2011-08-18 11:40:17 -0400: >> On Tue, Aug 16, 2011 at 8:23 PM, Li Zefan wrote: >> > Dan Merillat wrote: >> >> On Tue, Aug 16, 2011 at 8:51 AM, Chris Mason >> >> wrote: >> >>> Excerpts from Dan Merilla

Re: processes stuck in llseek

2011-08-18 Thread Chris Mason
Excerpts from Mitch Harder's message of 2011-08-18 12:53:33 -0400: > On Thu, Aug 18, 2011 at 11:00 AM, Chris Mason wrote: > > Excerpts from Mitch Harder's message of 2011-08-18 11:40:17 -0400: > >> On Tue, Aug 16, 2011 at 8:23 PM, Li Zefan wrote: > >> > Dan Merillat wrote: > >> >> On Tue, Aug 16,

Re: Filesystem corrupt after renaming snapshots.

2011-08-18 Thread David Sterba
Hi, On Wed, Aug 10, 2011 at 08:38:59PM +1200, Ralph Loader wrote: > Hi, > > Recently I suffered from a badly corrupted btrfs filesystem. > > I had several snapshots in /snap that I moved into / (using /bin/mv). > After that, attempting to access the ls the snapshot resulted in the > ls process h

[GIT PULL] Btrfs updates

2011-08-18 Thread Chris Mason
Hi everyone, The for-linus branch of the btrfs-unstable tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable.git for-linus Has our current pull request for 3.1-rc. The for-linus branch includes 3.1-rc2 because it two fixes from Dan Carpenter and Jeff Mahoney that only apply

Re: [PATCH 5/5] btrfs: use fs netlink interface for ENOSPC conditions

2011-08-18 Thread David Sterba
Hi, I see you are mixing a cleanup while adding a new feature. On Thu, Aug 18, 2011 at 02:18:26PM +0200, Lukas Czerner wrote: > Register fs netlink interface and send proper warning if ENOSPC is > encountered. Note that we differentiate between enospc for metadata and > enospc for data. > > Sign

Re: Filesystem corrupt after renaming snapshots.

2011-08-18 Thread Josef Bacik
On Thu, Aug 18, 2011 at 07:53:46PM +0200, David Sterba wrote: > Hi, > > On Wed, Aug 10, 2011 at 08:38:59PM +1200, Ralph Loader wrote: > > Hi, > > > > Recently I suffered from a badly corrupted btrfs filesystem. > > > > I had several snapshots in /snap that I moved into / (using /bin/mv). > > Aft

Re: [PATCH 5/5] btrfs: use fs netlink interface for ENOSPC conditions

2011-08-18 Thread Lukas Czerner
On Thu, 18 Aug 2011, David Sterba wrote: > Hi, > > I see you are mixing a cleanup while adding a new feature. > > On Thu, Aug 18, 2011 at 02:18:26PM +0200, Lukas Czerner wrote: > > Register fs netlink interface and send proper warning if ENOSPC is > > encountered. Note that we differentiate betw

Re: Honest timeline for btrfsck

2011-08-18 Thread Chris Mason
Excerpts from Yalonda Gishtaka's message of 2011-08-17 21:09:37 -0400: > Chris Mason oracle.com> writes: > > > > > Aside from making sure the kernel code is stable, btrfsck is all I'm > > working on right now. I do expect a release in the next two weeks that > > can recover your data (and many

Re: Honest timeline for btrfsck

2011-08-18 Thread Hugo Mills
On Thu, Aug 18, 2011 at 04:50:08PM -0400, Chris Mason wrote: > I've been working non-stop on this. Currently fsck has four parts: This all looks like great stuff. Can't wait to try it out... One thing strikes me for purposes of automated testing and regression testing: Do you have tools or

Re: [GIT PULL] Btrfs updates

2011-08-18 Thread Sage Weil
Hi Chris, Josef, Can some form of the clone ioctl transaction start reservation fix go in soon as well? That hits a BUG_ON every time. Thanks! sage On Thu, 18 Aug 2011, Chris Mason wrote: > Hi everyone, > > The for-linus branch of the btrfs-unstable tree: > > git://git.kernel.org/pub/scm/l

[PATCH 0/8] btrfs: Error handling fixes

2011-08-18 Thread Mark Fasheh
Hi, The following are assorted fixes to error handling from all parts of the Btrfs code. Every patch in this series stands on it's own, with the exception of the last patch which relies on the one before it (so patches 7 and 8 can be considered a pair). I also included in this series an uncommit

[PATCH 1/8] btrfs: Don't BUG_ON errors from btrfs_create_subvol_root()

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh This is called from only one place - create_subvol() which passes errors safely back out to it's caller, btrfs_mksubvol where they are handled. Additionally, btrfs_create_subvol_root() itself bug's needlessly from error return of btrfs_update_inode(). Since create_subvol() was

[PATCH 2/8] btrfs: Don't BUG_ON() errors in update_ref_for_cow()

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh The only caller of update_ref_for_cow() is __btrfs_cow_block() which was originally ignoring any return values. update_ref_for_cow() however doesn't look like a candidate to become a void function - there are a few places where errors can occur. So instead I changed update_ref_

[PATCH 3/8] btrfs: Don't BUG_ON kzalloc error in btrfs_lookup_csums_range()

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh Unfortunately it isn't enough to just exit here - the kzalloc() happens in a loop and the allocated items are added to a linked list whose head is passed in from the caller. To fix the BUG_ON() and also provide the semantic that the list passed in is only modified on success, I

[PATCH 4/8] btrfs: make insert_ptr() void

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh insert_ptr() always returns zero, so all the exta error handling can go away. This makes it trivial to also make copy_for_split() a void function as it's only return was from insert_ptr(). Finally, this all makes the BUG_ON(ret) in split_leaf() meaningless so I removed that. S

[PATCH 6/8] btrfs: fix error check of btrfs_lookup_dentry()

2011-08-18 Thread Mark Fasheh
From: Tsutomu Itoh Clean up btrfs_lookup_dentry() to never return NULL, but PTR_ERR(-ENOENT) instead. This keeps the return value convention consistent. Callers who pass to d_instatiate() require a trivial update. create_snapshot() in particular looks like it can also lose a BUG_ON(!inode) whic

[PATCH 7/8] btrfs: make fixup_low_keys() void

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh This is trivial - fixup_low_keys always returns zero so we can make it void. As a result, we can then make setup_items_for_insert() void too which lets us cut out a couple of BUG_ON(ret) lines. Signed-off-by: Mark Fasheh --- fs/btrfs/ctree.c | 59 ++-

[PATCH 5/8] btrfs: Don't BUG_ON errors in __finish_chunk_alloc()

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh All callers of __finish_chunk_alloc() BUG_ON() return value, so it's trivial for us to always bubble up any errors caught in __finish_chunk_alloc() to be caught there. Signed-off-by: Mark Fasheh --- fs/btrfs/volumes.c |7 ++- 1 files changed, 2 insertions(+), 5 deleti

[PATCH 8/8] btrfs: make del_ptr() and btrfs_del_leaf() void

2011-08-18 Thread Mark Fasheh
From: Mark Fasheh Since fixup_low_keys() has been made void, del_ptr() always returns zero. We can then make it void as well. This allows us in turn to make btrfs_del_leaf() void as the only return value it was previously catching was from del_ptr(). This winds up removing a couple of un-needed B

kernel oops -> no mount (different problem than others)

2011-08-18 Thread Liam
Hi. I''ve been using btrfs on an external drive for >1yr but now I cannot mount it. >From the logs it is listing a transid error but NOT that it is expecting a different one, simply device label 1TB devid 1 transid 248472 /dev/sdb I've been having this problem now for more than 1 month and have n

Re: kernel oops -> no mount (different problem than others)

2011-08-18 Thread cwillu
You might try mounting it "-o ro" as a stopgap to regain readonly access. Judging from the bootlog, the error itself appears to be enospc. In which case there's no already-available quick fix; I expect a developer to chime in any second now :p > From the logs it is listing a transid error but NO

Re: [PATCH] Btrfs: check if there is enough space for balancing smarter

2011-08-18 Thread David Sterba
Hi, too late, already pulled On Wed, Aug 03, 2011 at 06:15:25PM +0800, Liu Bo wrote: > --- a/fs/btrfs/extent-tree.c > +++ b/fs/btrfs/extent-tree.c > @@ -6682,6 +6682,10 @@ int btrfs_can_relocate(struct btrfs_root *root, u64 > bytenr) > struct btrfs_space_info *space_info; > struct bt

Re: [PATCH] Btrfs: check if there is enough space for balancing smarter

2011-08-18 Thread liubo
On 08/19/2011 10:46 AM, David Sterba wrote: > Hi, > > too late, already pulled > > On Wed, Aug 03, 2011 at 06:15:25PM +0800, Liu Bo wrote: >> --- a/fs/btrfs/extent-tree.c >> +++ b/fs/btrfs/extent-tree.c >> @@ -6682,6 +6682,10 @@ int btrfs_can_relocate(struct btrfs_root *root, u64 >> bytenr) >>

btrfs and mainline and git

2011-08-18 Thread Anand Jain
Hello, 1. I normally copy btrfs into the mainline and run make, however with the recent btrfs release its failing with the following, any idea. ? - # make scripts/kconfig/conf --silentoldconfig Kconfig fs/btrfs/Kconfig:6: syntax error fs/Kconfig:9