[PATCH 2/2] Btrfs: fix space leak when trimming free extents

2011-06-20 Thread Li Zefan
When the end of an extent exceeds the end of the specified range, the extent will be accidentally truncated. Signed-off-by: Li Zefan --- fs/btrfs/free-space-cache.c |9 - 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-

[PATCH 1/2] Btrfs: fix space leak when skipping small extents during trimming

2011-06-20 Thread Li Zefan
We're taking a free space extent out of the free space cache, trimming it and then putting it back into the cache. However for an extent that is smaller than the specified minimum length, it's taken out but won't be put back, which causes space leak. Signed-off-by: Li Zefan --- Unfortunately I

Re: [PATCH 4/8] fs: kill i_alloc_sem

2011-06-20 Thread Dave Chinner
On Mon, Jun 20, 2011 at 04:15:37PM -0400, Christoph Hellwig wrote: > i_alloc_sem is a rather special rw_semaphore. It's the last one that may > be released by a non-owner, and it's write side is always mirrored by > real exclusion. It's intended use it to wait for all pending direct I/O > request

kernel BUG at fs/btrfs/extent-tree.c:5299!

2011-06-20 Thread Andrej Podzimek
Hello, a backtrace from a machine running a Btrfs RAID0 array with two disks is attached. I've never seen this bug before. Please let me know if you need any further info/experiments. Andrej [ 2040.038677] [ cut here ] [ 2040.038696] kernel BUG at fs/btrfs/extent-tree.c

[PATCH] btrfs-progs: Improvement for making btrfs image from source directory.

2011-06-20 Thread Zhong, Xin
* Initialize ret in btrfs_csum_file_block * Do not abort when xattr is not supported in the source directory * Remove size limitation of 256M * Alloc data chunk in a smaller size (8M) to make btrfs image smaller * Let user specify the btrfs image name Depends on below patch from samsung guys: http:

Re: please review snapshot corruption path with delayed metadata insertion

2011-06-20 Thread Tsutomu Itoh
(2011/06/21 9:40), Chris Mason wrote: > Excerpts from David Sterba's message of 2011-06-20 20:24:35 -0400: >> On Mon, Jun 20, 2011 at 08:41:39AM +0900, Tsutomu Itoh wrote: >>> (2011/06/19 13:34), Tsutomu Itoh wrote: > I've fixed this up by moving the delayed metadata run down into the > sna

Re: please review snapshot corruption path with delayed metadata insertion

2011-06-20 Thread Chris Mason
Excerpts from David Sterba's message of 2011-06-20 20:24:35 -0400: > On Mon, Jun 20, 2011 at 08:41:39AM +0900, Tsutomu Itoh wrote: > > (2011/06/19 13:34), Tsutomu Itoh wrote: > > >> I've fixed this up by moving the delayed metadata run down into the > > >> snapshot creation code, please take a look

Re: please review snapshot corruption path with delayed metadata insertion

2011-06-20 Thread David Sterba
On Mon, Jun 20, 2011 at 08:41:39AM +0900, Tsutomu Itoh wrote: > (2011/06/19 13:34), Tsutomu Itoh wrote: > >> I've fixed this up by moving the delayed metadata run down into the > >> snapshot creation code, please take a look. If nobody objects I'll have > >> this in the pull I send to Linus this w

Re: Abysmal Performance

2011-06-20 Thread Josef Bacik
On 06/20/2011 05:51 PM, Henning Rohlfs wrote: Hello, I've migrated my system to btrfs (raid1) a few months ago. Since then the performance has been pretty bad, but recently it's gotten unbearable: a simple sync called while the system is idle can take 20 up to 60 seconds. Creating or deleting fi

Re: kernel BUG at fs/btrfs/delayed-inode.c:1301!

2011-06-20 Thread David Sterba
On Mon, Jun 20, 2011 at 06:12:10PM +0800, Miao Xie wrote: > >From 457f39393b2e3d475fbba029b90b6a4e17b94d43 Mon Sep 17 00:00:00 2001 > From: Miao Xie > Date: Mon, 20 Jun 2011 17:21:51 +0800 > Subject: [PATCH] btrfs: fix inconsonant inode information > > When iputting the inode, We may leave the de

Re: [PATCH] Btrfs: protect the pending_snapshots list with trans_lock

2011-06-20 Thread David Sterba
On Wed, Jun 15, 2011 at 07:41:37AM -0400, Chris Mason wrote: > There is definitely a window where two procs can be inside > create_snapshot() at the same time in the same transaction. I trust you on that. I was trying to follow the callgraph from btrfs_strat_transaction called from create_snapshot

Re: kernel BUG at fs/btrfs/delayed-inode.c:1301!

2011-06-20 Thread Jim Schutt
Hi Miao, Miao Xie wrote: Hi, Jim Could you test the attached patch for me? I have done some quick tests, it worked well. But I'm not sure if it can fix the bug you reported or not, so I need your help! So far I haven't been able to reproduce with your patch applied. I'd like to test for a

Re: [PATCH 6/8] fs: always maintain i_dio_count

2011-06-20 Thread Christoph Hellwig
On Mon, Jun 20, 2011 at 02:29:24PM -0700, Joel Becker wrote: > Oh god you're making the world scary. Are you guaranteeing that > all allocation changes are locked out by the time we get into > file_aio_write() and file_aio_read()? This is not obvious to me. I have no idea how ocfs2's inter

Re: [PATCH 4/8] fs: kill i_alloc_sem

2011-06-20 Thread Christoph Hellwig
On Mon, Jun 20, 2011 at 02:32:03PM -0700, Joel Becker wrote: > Are we guaranteed that all allocation changes are locked out by > i_dio_count>0? I don't think we are. The ocfs2 code very strongly > assumes the state of a file's allocation when it holds i_alloc_sem. I > feel like we lose tha

Abysmal Performance

2011-06-20 Thread Henning Rohlfs
Hello, I've migrated my system to btrfs (raid1) a few months ago. Since then the performance has been pretty bad, but recently it's gotten unbearable: a simple sync called while the system is idle can take 20 up to 60 seconds. Creating or deleting files often has several seconds latency, too.

Re: [PATCH 4/8] fs: kill i_alloc_sem

2011-06-20 Thread Joel Becker
On Mon, Jun 20, 2011 at 04:15:37PM -0400, Christoph Hellwig wrote: > i_alloc_sem is a rather special rw_semaphore. It's the last one that may > be released by a non-owner, and it's write side is always mirrored by > real exclusion. It's intended use it to wait for all pending direct I/O > request

Re: [PATCH 6/8] fs: always maintain i_dio_count

2011-06-20 Thread Joel Becker
On Mon, Jun 20, 2011 at 04:15:39PM -0400, Christoph Hellwig wrote: > Maintain i_dio_count for all filesystems, not just those using DIO_LOCKING. > This these filesystems to also protect truncate against direct I/O requests > by using common code. Right now the only non-DIO_LOCKING filesystem that

Re: [PATCH 0/8] remove i_alloc_sem

2011-06-20 Thread Christoph Hellwig
On Mon, Jun 20, 2011 at 04:15:33PM -0400, Christoph Hellwig wrote: > This series removes it in favour of a simpler counter scheme, thus getting > rid of the rw_semaphore non-owner APIs as requests by Thomas, while at the > same time shrinking the size of struct inode by 160 bytes on 64-bit systems.

[PATCH 1/8] far: remove i_alloc_sem abuse

2011-06-20 Thread Christoph Hellwig
Add a new rw_semaphore to protect bmap against truncate. Previous i_alloc_sem was abused for this, but it's going away in this series. Signed-off-by: Christoph Hellwig Index: linux-2.6/fs/fat/inode.c === --- linux-2.6.orig/fs/fat/i

[PATCH 2/8] ext4: remove i_alloc_sem abuse

2011-06-20 Thread Christoph Hellwig
Add a new rw_semaphore to protect page_mkwrite against truncate. Previous i_alloc_sem was abused for this, but it's going away in this series. Signed-off-by: Christoph Hellwig Index: linux-2.6/fs/ext4/inode.c === --- linux-2.6.orig

[PATCH 6/8] fs: always maintain i_dio_count

2011-06-20 Thread Christoph Hellwig
Maintain i_dio_count for all filesystems, not just those using DIO_LOCKING. This these filesystems to also protect truncate against direct I/O requests by using common code. Right now the only non-DIO_LOCKING filesystem that appears to do so is XFS, which uses an opencoded variant of the i_dio_cou

[PATCH 5/8] fs: move inode_dio_wait calls into ->setattr

2011-06-20 Thread Christoph Hellwig
Let filesystems handle waiting for direct I/O requests themselves instead of doing it beforehand. This means filesystem-specific locks to prevent new dio referenes from appearing can be held. This is important to allow generalizing i_dio_count to non-DIO_LOCKING filesystems. Signed-off-by: Chris

[PATCH 4/8] fs: kill i_alloc_sem

2011-06-20 Thread Christoph Hellwig
i_alloc_sem is a rather special rw_semaphore. It's the last one that may be released by a non-owner, and it's write side is always mirrored by real exclusion. It's intended use it to wait for all pending direct I/O requests to finish before starting a truncate. Replace it with a hand-grown const

[PATCH 3/8] fs: simpler handling of zero sized reads in __blockdev_direct_IO

2011-06-20 Thread Christoph Hellwig
Reject zero sized reads as soon as we know our I/O length, and don't borther with locks or allocations that might have to be cleaned up otherwise. Signed-off-by: Christoph Hellwig Index: linux-2.6/fs/direct-io.c === --- linux-2.6.or

[PATCH 0/8] remove i_alloc_sem

2011-06-20 Thread Christoph Hellwig
i_alloc_sem has always been a bit of an odd "lock". It's the only remaining rw_semaphore that can be released by a different thread than the one that locked it, and it's use case in the core direct I/O code is more like a counter given that the writers already have external serialization. This se

[PATCH 7/8] btrfs: wait for direct I/O requests in truncate

2011-06-20 Thread Christoph Hellwig
Wait for all direct I/O requests to finish before performing a truncate. Signed-off-by: Christoph Hellwig Index: linux-2.6/fs/btrfs/inode.c === --- linux-2.6.orig/fs/btrfs/inode.c 2011-06-11 12:58:46.615017504 +0200 +++ linux-2.

[PATCH 8/8] rw_semaphore: remove up/down_read_non_owner

2011-06-20 Thread Christoph Hellwig
Now that the last users is gone these can be removed. Signed-off-by: Christoph Hellwig Index: linux-2.6/include/linux/rwsem.h === --- linux-2.6.orig/include/linux/rwsem.h2011-06-20 14:58:15.449148809 +0200 +++ linux-2.6/inc

[PATCH] Btrfs: fix how we merge extent states and deal with cached states

2011-06-20 Thread Josef Bacik
First, we can sometimes free the state we're merging, which means anybody who calls merge_state() may have the state it passed in free'ed. This is problematic because we could end up caching the state, which makes caching useless as the state will no longer be part of the tree. So instead of free

[PATCH] Btrfs: save preloaded extent_state's in a percpu cache

2011-06-20 Thread Josef Bacik
When doing DIO tracing I noticed we were doing a ton of allocations, a lot of the time for extent_states. Some of the time we don't even use the prealloc'ed extent_state, it just get's free'd up. So instead create a per-cpu cache like the radix tree stuff. So we will check to see if our per-cpu

Re: compression ratio

2011-06-20 Thread Mitch Harder
On Mon, Jun 20, 2011 at 10:22 AM, Hugo Mills wrote: > On Mon, Jun 20, 2011 at 07:17:22PM +0400, Proskurin Kirill wrote: >> On 06/20/2011 06:34 PM, Helmut Hullen wrote: >> >Du meintest am 20.06.11: >> >>What we have: >> >>SL6 - kernel 2.6.32-131.2.1.el6.x86_64 >> >>mdadm RAID5 with 8 HDD - 27T part

Re: compression ratio

2011-06-20 Thread Hugo Mills
On Mon, Jun 20, 2011 at 07:17:22PM +0400, Proskurin Kirill wrote: > On 06/20/2011 06:34 PM, Helmut Hullen wrote: > >Du meintest am 20.06.11: > >>What we have: > >>SL6 - kernel 2.6.32-131.2.1.el6.x86_64 > >>mdadm RAID5 with 8 HDD - 27T partition. > > > >You should take a newer kernel. On my system I

Re: compression ratio

2011-06-20 Thread Proskurin Kirill
On 06/20/2011 06:34 PM, Helmut Hullen wrote: Hallo, Proskurin, Du meintest am 20.06.11: I`m new to btrfs and do some testing now. What we have: SL6 - kernel 2.6.32-131.2.1.el6.x86_64 mdadm RAID5 with 8 HDD - 27T partition. You should take a newer kernel. On my system I needed 2.6.38.5 and

Re: compression ratio

2011-06-20 Thread Helmut Hullen
Hallo, Proskurin, Du meintest am 20.06.11: > I`m new to btrfs and do some testing now. > What we have: > SL6 - kernel 2.6.32-131.2.1.el6.x86_64 > mdadm RAID5 with 8 HDD - 27T partition. You should take a newer kernel. On my system I needed 2.6.38.5 and newer for btrfs; older kernels lead to u

Re: compression ratio

2011-06-20 Thread David Sterba
Hi, On Mon, Jun 20, 2011 at 03:29:45PM +0400, Proskurin Kirill wrote: > What we have: > SL6 - kernel 2.6.32-131.2.1.el6.x86_64 > mdadm RAID5 with 8 HDD - 27T partition. btw .32 is very old > Mount options is "noatime,noacl,compress-force" > I use scribe daemon to copy log files from 200 hosts to

compression ratio

2011-06-20 Thread Proskurin Kirill
Hello all. I`m new to btrfs and do some testing now. What we have: SL6 - kernel 2.6.32-131.2.1.el6.x86_64 mdadm RAID5 with 8 HDD - 27T partition. Mount options is "noatime,noacl,compress-force" I use scribe daemon to copy log files from 200 hosts to that partition for stress testing. But I f

Re: kernel BUG at fs/btrfs/delayed-inode.c:1301!

2011-06-20 Thread Miao Xie
Hi, Jim Could you test the attached patch for me? I have done some quick tests, it worked well. But I'm not sure if it can fix the bug you reported or not, so I need your help! Thanks Miao On fri, 17 Jun 2011 10:10:31 -0600, Jim Schutt wrote: > Hi, > > I've hit this delayed-inode BUG several t