Re: [PATCH 4/8] fs: kill i_alloc_sem

2011-06-30 Thread Joel Becker
On Mon, Jun 20, 2011 at 06:18:57PM -0400, Christoph Hellwig wrote: > On Mon, Jun 20, 2011 at 02:32:03PM -0700, Joel Becker wrote: > > Are we guaranteed that all allocation changes are locked out by > > i_dio_count>0? I don't think we are. The ocfs2 code very strongly > > assumes the state of

Re: btrfs-progs: integration branch updated

2011-06-30 Thread Li Zefan
> Jan Schmidt (5): > commands added > scrub ioctls > added check_mounted_where > scrub userland implementation > scrub added to manpage > > WuBo (1): > Btrfs-progs: Add chunk tree recover tool > I think we should wait for the kernel patch to be accepted before

Re: subvolumes missing from "btrfs subvolume list" output

2011-06-30 Thread Li Zefan
>>> Hugo Mills keeps an integration branch with nearly all patches to >>> btrfs-progs applied. >>> See >>> >>> http://www.spinics.net/lists/linux-btrfs/msg10594.html >>> >>> and for the last update >>> >>> http://www.spinics.net/lists/linux-btrfs/msg10890.html >> [...] >> >> Thanks. >> >> It might

Re: [PATCH v6 2/8] Add --monitor option to btrfs balance progress.

2011-06-30 Thread Andreas Philipp
On 01.05.2011 17:47, Hugo Mills wrote: > For the impatient, this patch introduces the pot-watching --monitor > option, which checks the balance progress at regular intervals, and > updates a single status line with the current progress and an > estimated completion time. > > Signed-off-by: Hugo Mil

Re: [PATCH v6 2/8] Add --monitor option to btrfs balance progress.

2011-06-30 Thread Andreas Philipp
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01.05.2011 17:47, Hugo Mills wrote: > For the impatient, this patch introduces the pot-watching > --monitor option, which checks the balance progress at regular > intervals, and updates a single status line with the current > progress and an estim

Re: btrfs-progs: integration branch updated

2011-06-30 Thread Andreas Philipp
anic.net/repo/btrfs-progs-unstable.git/ > integration-20110630 > > The shortlog of 17 patches in this branch beyond the ones I've > sent to Chris is below. > > Hugo. > > > Andreas Philipp (1): print parent ID in btrfs subvolume list > > Goffredo Baroncelli (1):

Re: btrfs-progs: integration branch updated

2011-06-30 Thread Andreas Philipp
anic.net/repo/btrfs-progs-unstable.git/ > integration-20110630 > > The shortlog of 17 patches in this branch beyond the ones I've > sent to Chris is below. > > Hugo. > > > Andreas Philipp (1): print parent ID in btrfs subvolume list This is still the first version of my pat

btrfs-progs: integration branch updated

2011-06-30 Thread Hugo Mills
After a reorganisation of patches, and sending a bunch of them to Chris, I've also updated the integration branch to match that. It's available from: http://git.darksatanic.net/repo/btrfs-progs-unstable.git/ integration-20110630 The shortlog of 17 patches in this branch beyon

Re: [PATCH] [btrfs-progs integration] incorrect argument checking for "btrfs sub snap -r"

2011-06-30 Thread Hugo Mills
On Thu, Jun 30, 2011 at 10:55:15PM +0200, Andreas Philipp wrote: > On 30.06.2011 14:34, Stephane Chazelas wrote: > > Looks like this was missing in integration-20110626 for the > > readonly snapshot patch: > > > > diff --git a/btrfs.c b/btrfs.c > > index e117172..be6ece5 100644 > > --- a/btrfs.c >

Re: [PATCH] [btrfs-progs integration] incorrect argument checking for "btrfs sub snap -r"

2011-06-30 Thread Andreas Philipp
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30.06.2011 14:34, Stephane Chazelas wrote: > Looks like this was missing in integration-20110626 for the > readonly snapshot patch: > > diff --git a/btrfs.c b/btrfs.c > index e117172..be6ece5 100644 > --- a/btrfs.c > +++ b/btrfs.c > @@ -49,7 +49,7

[GIT PULL] btrfs-progs: first slice of integration branch

2011-06-30 Thread Hugo Mills
Hi, Chris, I've done a more detailed examination of the patches in my integration branch of btrfs-progs, and this is the set of small and generally inoffensive/uncontroversial patches from that stack. I've reviewed them all, and didn't see anything horribly wrong in there. Since I was lo

[PATCH] Btrfs: use a worker thread to do caching

2011-06-30 Thread Josef Bacik
A user reported a deadlock when copying a bunch of files. This is because they were low on memory and kthreadd got hung up trying to migrate pages for an allocation when starting the caching kthread. The page was locked by the person starting the caching kthread. To fix this we just need to use

Re: [btrfs-transacti] & btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-06-30 Thread Proskurin Kirill
On 06/30/2011 09:13 PM, Josef Bacik wrote: On 06/30/2011 10:12 AM, Proskurin Kirill wrote: On 06/29/2011 08:14 PM, Josef Bacik wrote: Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse. Now I see [btrfs-transacti]& btrfs-endio-wri] 80-100% all the time and io performance looks

Re: [btrfs-transacti] & btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-06-30 Thread Proskurin Kirill
On 06/30/2011 09:13 PM, Josef Bacik wrote: On 06/30/2011 10:12 AM, Proskurin Kirill wrote: On 06/29/2011 08:14 PM, Josef Bacik wrote: Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse. Now I see [btrfs-transacti]& btrfs-endio-wri] 80-100% all the time and io performance looks

Re: [btrfs-transacti] & btrfs-endio-wri] - WAS: Re: [btrfs-delalloc-]

2011-06-30 Thread Josef Bacik
On 06/30/2011 10:12 AM, Proskurin Kirill wrote: > On 06/29/2011 08:14 PM, Josef Bacik wrote: >>> Ok - I upgrade to 2.6.39-2 but it is seems to all things get worse. >>> Now I see [btrfs-transacti]& btrfs-endio-wri] 80-100% all the time and >>> io performance looks like lower then before. >>> >>> O

Re: btrfs lock-up on copying files

2011-06-30 Thread Josef Bacik
On 06/30/2011 10:43 AM, Roman Mamedov wrote: > On Thu, 30 Jun 2011 10:37:51 -0400 > Josef Bacik wrote: > >> Can you do sysrq+w when this happens? The caching kthread should still >> be able to make progress, which is what we seem to be waiting on. Does >> it eventually unhang and continue on?

Re: btrfs lock-up on copying files

2011-06-30 Thread Roman Mamedov
On Thu, 30 Jun 2011 10:37:51 -0400 Josef Bacik wrote: > Can you do sysrq+w when this happens? The caching kthread should still > be able to make progress, which is what we seem to be waiting on. Does > it eventually unhang and continue on? Thanks, Hello, Unfortunately I already rebooted that

Re: btrfs lock-up on copying files

2011-06-30 Thread Josef Bacik
On 06/30/2011 02:37 AM, Roman Mamedov wrote: > Hello, > > I was copying a set of files to a btrfs filesystem, and the copy process > locked-up with the following messages in dmesg. > > The kernel version is 2.6.39.1; the filesystem was recently resized from 3 to > 4TB (if that matters). > > It

Re: [PATCH v4 4/7] btrfs: initial readahead code and prototypes

2011-06-30 Thread Arne Jansen
On 30.06.2011 14:49, Josef Bacik wrote: > On 06/30/2011 03:37 AM, Arne Jansen wrote: >> On 29.06.2011 23:49, Josef Bacik wrote: >>> On 06/29/2011 04:10 PM, Arne Jansen wrote: > >> >>> +struct krefrefcnt; +wait_queue_head_twait; +}; + +struct reada_extct

Re: [PATCH] [btrfs-progs integration] incorrect argument checking for "btrfs sub snap -r"

2011-06-30 Thread Hugo Mills
On Thu, Jun 30, 2011 at 01:34:38PM +0100, Stephane Chazelas wrote: > Looks like this was missing in integration-20110626 for the > readonly snapshot patch: > > diff --git a/btrfs.c b/btrfs.c > index e117172..be6ece5 100644 > --- a/btrfs.c > +++ b/btrfs.c > @@ -49,7 +49,7 @@ static struct Command c

Re: [PATCH v4 4/7] btrfs: initial readahead code and prototypes

2011-06-30 Thread Josef Bacik
On 06/30/2011 03:37 AM, Arne Jansen wrote: On 29.06.2011 23:49, Josef Bacik wrote: On 06/29/2011 04:10 PM, Arne Jansen wrote: + struct kref refcnt; + wait_queue_head_t wait; +}; + +struct reada_extctl { + struct list_headlist; + struct rea

[PATCH] [btrfs-progs integration] incorrect argument checking for "btrfs sub snap -r"

2011-06-30 Thread Stephane Chazelas
Looks like this was missing in integration-20110626 for the readonly snapshot patch: diff --git a/btrfs.c b/btrfs.c index e117172..be6ece5 100644 --- a/btrfs.c +++ b/btrfs.c @@ -49,7 +49,7 @@ static struct Command commands[] = { /* avoid short commands different for the cas

Re: subvolumes missing from "btrfs subvolume list" output

2011-06-30 Thread Hugo Mills
On Thu, Jun 30, 2011 at 11:43:40AM +0100, Stephane Chazelas wrote: > 2011-06-30 11:18:42 +0200, Andreas Philipp: > [...] > > >> After that, I posted a patch to fix btrfs-progs, which Chris > > >> aggreed on: > > >> > > >> http://marc.info/?l=linux-btrfs&m=129238454714319&w=2 > > > [...] > > > > > >

Re: subvolumes missing from "btrfs subvolume list" output

2011-06-30 Thread Hugo Mills
On Thu, Jun 30, 2011 at 12:52:59PM +0200, Andreas Philipp wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 30.06.2011 12:43, Stephane Chazelas wrote: > > 2011-06-30 11:18:42 +0200, Andreas Philipp: [...] > After that, I posted a patch to fix btrfs-progs, which Chris >

Re: subvolumes missing from "btrfs subvolume list" output

2011-06-30 Thread Andreas Philipp
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30.06.2011 12:43, Stephane Chazelas wrote: > 2011-06-30 11:18:42 +0200, Andreas Philipp: [...] After that, I posted a patch to fix btrfs-progs, which Chris aggreed on: http://marc.info/?l=linux-btrfs&m=129238454714319&w=2 >>> [

Re: subvolumes missing from "btrfs subvolume list" output

2011-06-30 Thread Stephane Chazelas
2011-06-30 11:18:42 +0200, Andreas Philipp: [...] > >> After that, I posted a patch to fix btrfs-progs, which Chris > >> aggreed on: > >> > >> http://marc.info/?l=linux-btrfs&m=129238454714319&w=2 > > [...] > > > > Great. Thanks a lot > > > > It fixes my problem indeed. > > > > Which brings me to m

[PATCH] Btrfs: initial online fsck support

2011-06-30 Thread Li Zefan
This is an initial version of online fsck. What it does is: - check the dir item and dir index pointing to a file. - check the structure of extents of a file. As furthur work, we should consider: - fix but not only check the structure of a file. - verify the extent allocation tree on the fly. ..

Re: [GIT PULL v4] Btrfs: improve write ahead log with sub transaction

2011-06-30 Thread liubo
On 06/30/2011 03:36 PM, Liu Bo wrote: > I've been working to try to improve the write-ahead log's performance, > and I found that the bottleneck addresses in the checksum items, > especially when we want to make a random write on a large file, e.g a 4G file. > > Then a idea for this suggested by C

Re: subvolumes missing from "btrfs subvolume list" output

2011-06-30 Thread Andreas Philipp
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30.06.2011 10:40, Stephane Chazelas wrote: > 2011-06-30 08:47:38 +0800, Li Zefan: >> Stephane Chazelas wrote: >>> 2011-06-29 15:37:47 +0100, Stephane Chazelas: [...] I found http://thread.gmane.org/gmane.comp.file-systems.btrfs/8123/focu

Re: please review snapshot corruption path with delayed metadata insertion

2011-06-30 Thread Miao Xie
On thu, 30 Jun 2011 16:03:21 +0800, Miao Xie wrote: > Hi, Chris > > I think the snapshot should be the image of the fs tree before it was created, > so the metadata of the snapshot should not exist in the its tree. But now, we > found the directory item and directory name index is in both the snap

Re: subvolumes missing from "btrfs subvolume list" output

2011-06-30 Thread Stephane Chazelas
2011-06-30 08:47:38 +0800, Li Zefan: > Stephane Chazelas wrote: > > 2011-06-29 15:37:47 +0100, Stephane Chazelas: > > [...] > >> I found > >> http://thread.gmane.org/gmane.comp.file-systems.btrfs/8123/focus=8208 > >> > >> which looks like the same issue, with Li Zefan saying he had a > >> fix, but

Re: please review snapshot corruption path with delayed metadata insertion

2011-06-30 Thread Miao Xie
Hi, Chris I think the snapshot should be the image of the fs tree before it was created, so the metadata of the snapshot should not exist in the its tree. But now, we found the directory item and directory name index is in both the snapshot tree and the fs tree. Besides that, it also makes the us

[PATCH 01/12] Btrfs: introduce sub transaction stuff

2011-06-30 Thread Liu Bo
Introduce a new concept "sub transaction", the relation between transaction and sub transaction is transaction A ---> transid = x sub trans a(1) ---> sub_transid = x+1 sub trans a(2) ---> sub_transid = x+2 ... ... sub trans a(n-1) ---> sub_transid = x+n-1 sub trans a(n)

[PATCH 04/12] Btrfs: introduce first sub trans

2011-06-30 Thread Liu Bo
In multi-thread situations, writeback of a file may span across several sub transactions, and we need to introduce first_sub_trans to get sub_transid of teh first sub transaction recorded, so that log code can skip file extents which have been logged or committed onto disk. Signed-off-by: Liu Bo

[PATCH 06/12] Btrfs: improve log with sub transaction

2011-06-30 Thread Liu Bo
When logging an inode _A_, current btrfs will a) clear all items belonged to _A_ in log, b) copy all items belonged to _A_ from fs/file tree to log tree, and this just wastes a lot of time, especially when logging big files. So we want to use a smarter approach, i.e. "find and merge". The amount o

[PATCH 07/12] Btrfs: add checksum check for log

2011-06-30 Thread Liu Bo
If a inode is a BTRFS_INODE_NODATASUM one, it need not to look for csum items any more. Signed-off-by: Liu Bo --- fs/btrfs/tree-log.c |8 +--- 1 files changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index 0ccffb1..5b5c336 100644 --- a/fs/b

[PATCH 08/12] Btrfs: fix a bug of log check

2011-06-30 Thread Liu Bo
The current code uses struct root's last_log_commit to check if an inode has been logged, but the problem is that this root->last_log_commit is shared among files. Say we have N inodes to be logged, after the first inode, root-last_log_commit is updated and the N-1 remains will not be logged. As

[PATCH 12/12] Revert "Btrfs: do not flush csum items of unchanged file data during treelog"

2011-06-30 Thread Liu Bo
This reverts commit 8e531cdfeb75269c6c5aae33651cca39707848da. Signed-off-by: Liu Bo --- fs/btrfs/tree-log.c |3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index b7fca84..3f52182 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/

[PATCH 10/12] Btrfs: use the right generation number to read log_root_tree

2011-06-30 Thread Liu Bo
Currently we use the generation number of the super to read in the log tree root after a crash. This doesn't always match the sub trans id and so it doesn't always match the transid stored in the btree blocks. We can use log_root_transid to record the log_root_tree's generation so that when we re

[PATCH 11/12] Btrfs: do not iput inode when inode is still in log

2011-06-30 Thread Liu Bo
We maintain the inode's logged_trans to avoid reloging it, but if we iput the inode and reread it, we'll get logged_trans to zero. So when an inode is still in log tree, and transaction is not committed yet, we do not iput the inode. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 11 +--

[PATCH 09/12] Btrfs: kick off useless code

2011-06-30 Thread Liu Bo
fsync will wait for writeback till it finishes, and last_trans will get the real transid recorded in writeback, so it does not need an extra +1 to ensure fsync's process on the file. Signed-off-by: Liu Bo --- fs/btrfs/file.c | 13 - 1 files changed, 0 insertions(+), 13 deletions(-)

[PATCH 05/12] Btrfs: still update inode trans stuff when size remains unchanged

2011-06-30 Thread Liu Bo
Due to DIO stuff, commit 1ef30be142d2cc60e2687ef267de864cf31be995 makes btrfs not call btrfs_update_inode when it does not update i_disk_size, but in buffer write case, we need to update btrfs internal inode's trans stuff, so that the log code can find the inode's changes. Signed-off-by: Liu Bo -

[PATCH 03/12] Btrfs: modify btrfs_drop_extents API

2011-06-30 Thread Liu Bo
We want to use btrfs_drop_extent() in log code. Signed-off-by: Liu Bo --- fs/btrfs/ctree.h|3 ++- fs/btrfs/file.c |9 +++-- fs/btrfs/inode.c|6 +++--- fs/btrfs/ioctl.c|4 ++-- fs/btrfs/tree-log.c |2 +- 5 files changed, 15 insertions(+), 9 deletions(-) di

Re: [PATCH v4 4/7] btrfs: initial readahead code and prototypes

2011-06-30 Thread Arne Jansen
On 29.06.2011 23:49, Josef Bacik wrote: > On 06/29/2011 04:10 PM, Arne Jansen wrote: >> This is the implementation for the generic read ahead framework. >> >> To trigger a readahead, btrfs_reada_add must be called. It will start >> a read ahead for the given range [start, end) on tree root. The ret

[GIT PULL v4] Btrfs: improve write ahead log with sub transaction

2011-06-30 Thread Liu Bo
I've been working to try to improve the write-ahead log's performance, and I found that the bottleneck addresses in the checksum items, especially when we want to make a random write on a large file, e.g a 4G file. Then a idea for this suggested by Chris is to use sub transaction ids and just to l

[PATCH 02/12] Btrfs: update block generation if should_cow_block fails

2011-06-30 Thread Liu Bo
Cause we've added sub transaction, if it do not want to cow a block, we also need to get new sub transid recorded. This is used for log code to find the most uptodate file extents. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 34 +- 1 files changed, 33 insertio