[PATCH] Btrfs: save us a mutex_lock usage when doing quota rescan

2013-05-06 Thread Wang Shilong
If qgroup_rescan worker is in progress, we should ignore the extent that has not been dealt with qgroup_rescan worker,just let them dealt later otherwise we may get wrong qgroup accounting. However, we have checked this before find_all_roots() without spin_lock. When doing qgroup accounting, we do

[PATCH] Btrfs: fix passing wrong arg gfp_t to decide the correct allocation mode

2013-05-06 Thread Wang Shilong
If you look the code carefully, you will see all the tree_mod_alloc() has to use GFP_ATOMIC. However, the original code pass the wrong arg gfp_t in some places, this dosen't cause any problems, because in the tree_mod_alloc(), it ignores arg gfp_t and just use GFP_ATOMIC directly, this is not good.

Re: [PATCH] Btrfs: add ioctl to wait for qgroup rescan completion

2013-05-06 Thread Jan Schmidt
On Mon, May 06, 2013 at 23:20 (+0200), David Sterba wrote: > On Mon, May 06, 2013 at 09:14:17PM +0200, Jan Schmidt wrote: >> --- a/include/uapi/linux/btrfs.h >> +++ b/include/uapi/linux/btrfs.h >> @@ -530,6 +530,7 @@ struct btrfs_ioctl_send_args { >> struct btrfs_ioctl_q

Re: hard freezes with 3.9.0 during io-intensive loads

2013-05-06 Thread Jan Schmidt
On Mon, May 06, 2013 at 22:29 (+0200), Kai Krakow wrote: > Jan Schmidt schrieb: > >> That one should be fixed in btrfs-next. If you can reliably reproduce the >> bug I'd be glad to get a confirmation - you can probably even save putting >> it on bugzilla then ;-) > > I can reliably reproduce it

Re: [RFC 0/5] BTRFS hot relocation support

2013-05-06 Thread Tomasz Torcz
On Mon, May 06, 2013 at 10:36:03PM +0200, Kai Krakow wrote: > zwu.ker...@gmail.com schrieb: > > > The patchset is trying to introduce hot relocation support > > for BTRFS. In hybrid storage environment, when the data in > > HDD disk get hot, it can be relocated to SSD disk by BTRFS > > hot relo

Re: [PATCH v3] btrfs: clean snapshots one by one

2013-05-06 Thread Chris Mason
Quoting David Sterba (2013-03-12 11:13:28) > Each time pick one dead root from the list and let the caller know if > it's needed to continue. This should improve responsiveness during > umount and balance which at some point waits for cleaning all currently > queued dead roots. > > A new dead root

[PATCH] btrfs-progs: fix typecast when printing csum value

2013-05-06 Thread David Sterba
Only the first byte of the wanted csum is printed: checksum verify failed on 65536 found DA97CF61 wanted 6B checksum verify failed on 65536 found DA97CF61 wanted 6BC3870D Also add leading zeros to the format. Signed-off-by: David Sterba --- disk-io.c | 6 +++--- 1 file changed, 3 insertions(+)

[PATCH] btrfs: don't stop searching after encountering the wrong item

2013-05-06 Thread Gabriel de Perthuis
The search ioctl skips items that are too large for a result buffer, but inline items of a certain size occuring before any search result is found would trigger an overflow and stop the search entirely. Cc: sta...@vger.kernel.org Bug: https://bugzilla.kernel.org/show_bug.cgi?id=57641 Signed-off-b

Re: [PATCH] Btrfs: add ioctl to wait for qgroup rescan completion

2013-05-06 Thread David Sterba
On Mon, May 06, 2013 at 09:14:17PM +0200, Jan Schmidt wrote: > --- a/include/uapi/linux/btrfs.h > +++ b/include/uapi/linux/btrfs.h > @@ -530,6 +530,7 @@ struct btrfs_ioctl_send_args { > struct btrfs_ioctl_quota_rescan_args) > #define BTRFS_IOC_QUOTA_RESCAN_STATUS _IOR(

[PATCH] btrfs-progs: image: handle superblocks correctly on fs with big blocks

2013-05-06 Thread David Sterba
Superblock is always 4k, but metadata blocks may be larger. We have to use the appropriate block size when doing checksums, otherwise they're wrong. Signed-off-by: David Sterba --- btrfs-image.c | 27 +++ 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/btrf

Re: Possible to dedpulicate read-only snapshots for space-efficient backups

2013-05-06 Thread Kai Krakow
james northrup schrieb: > tried a git based backup? sounds spot-on as a compromise prior to > applying btrfs tweaks. snapshotting the git binaries would have the > dedupe characteristics. Git is efficient with space, yes. But if you have a lot of binary files, and a lot of them are big, git be

Re: [RFC 0/5] BTRFS hot relocation support

2013-05-06 Thread Kai Krakow
zwu.ker...@gmail.com schrieb: > The patchset is trying to introduce hot relocation support > for BTRFS. In hybrid storage environment, when the data in > HDD disk get hot, it can be relocated to SSD disk by BTRFS > hot relocation support automatically; also, if SSD disk ratio > exceed its upper

Re: hard freezes with 3.9.0 during io-intensive loads

2013-05-06 Thread Kai Krakow
Jan Schmidt schrieb: > That one should be fixed in btrfs-next. If you can reliably reproduce the > bug I'd be glad to get a confirmation - you can probably even save putting > it on bugzilla then ;-) I can reliably reproduce it from two different approaches. I'd like to only apply the commits f

Re: [PATCH] Btrfs: fix off-by-one in fiemap

2013-05-06 Thread Josef Bacik
On Wed, May 01, 2013 at 10:23:41AM -0600, Liu Bo wrote: > lock_extent/unlock_extent expect an exclusive end. > Can you make an xfstest for this so we can make sure we don't screw this up in the future? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in th

Re: [PATCH v2] btrfs: device delete to get errors from the kernel

2013-05-06 Thread Josef Bacik
On Tue, Apr 30, 2013 at 07:19:40AM -0600, Anand Jain wrote: > v1->v2: > introduce error codes for the device mgmt usage > > v1: > adds a parameter in the ioctl arg struct to carry the error string > > Signed-off-by: Anand Jain > --- I need a proper log for this patch. Thanks, Josef -- To unsu

[PATCH] Btrfs: add ioctl to wait for qgroup rescan completion

2013-05-06 Thread Jan Schmidt
btrfs_qgroup_wait_for_completion waits until the currently running qgroup operation completes. It returns immediately when no rescan process is in progress. This is useful to automate things around the rescan process (e.g. testing). Signed-off-by: Jan Schmidt --- fs/btrfs/ctree.h |

[PATCH 2/2] Btrfs-progs: added "btrfs quota rescan" -w switch (wait)

2013-05-06 Thread Jan Schmidt
With -w one can wait for a rescan operation to finish. It can be used when starting a rescan operation or later to wait for the currently running rescan operation to finish. Waiting is interruptible. Signed-off-by: Jan Schmidt --- cmds-quota.c | 19 +-- ioctl.h |1 + 2

[PATCH 1/2] Btrfs-progs: fixup: add flags to struct btrfs_ioctl_quota_rescan_args

2013-05-06 Thread Jan Schmidt
The patch set previously sent was sent together with the kernel part, but was not updated as I added some reserved bytes to the ioctl struct for future compatibility. This fixes struct btrfs_ioctl_quota_rescan_args. Signed-off-by: Jan Schmidt --- ioctl.h |1 + 1 files changed, 1 insertions(+

Btrfs: wait for quota rescan to complete

2013-05-06 Thread Jan Schmidt
Two small patches, one for the kernel and one for the user mode. Both required to support waiting for quota rescan to complete. Jan Schmidt (1): Btrfs: add ioctl to wait for qgroup rescan completion fs/btrfs/ctree.h |2 ++ fs/btrfs/ioctl.c | 12 fs/btrfs/

Re: [PATCH] btrfs: don't stop searching after encountering the wrong item

2013-05-06 Thread Greg KH
On Mon, May 06, 2013 at 07:40:18PM +0200, Gabriel de Perthuis wrote: > The search ioctl skips items that are too large for a result buffer, but > inline items of a certain size occuring before any search result is > found would trigger an overflow and stop the search entirely. > > Bug: https://bug

[PATCH] btrfs: don't stop searching after encountering the wrong item

2013-05-06 Thread Gabriel de Perthuis
The search ioctl skips items that are too large for a result buffer, but inline items of a certain size occuring before any search result is found would trigger an overflow and stop the search entirely. Bug: https://bugzilla.kernel.org/show_bug.cgi?id=57641 Signed-off-by: Gabriel de Perthuis ---

Re: Possible to dedpulicate read-only snapshots for space-efficient backups

2013-05-06 Thread james northrup
tried a git based backup? sounds spot-on as a compromise prior to applying btrfs tweaks. snapshotting the git binaries would have the dedupe characteristics. On Mon, May 6, 2013 at 12:44 AM, Kai Krakow wrote: > Jan Schmidt schrieb: > >>> I'm using an bash/rsync script[1] to backup my whole syst

Re: [PATCH 3/4] btrfs: Introduce extent_read_full_page_nolock()

2013-05-06 Thread David Sterba
On Tue, Apr 16, 2013 at 03:15:34PM -0700, Mark Fasheh wrote: > @@ -2625,7 +2625,7 @@ static int __extent_read_full_page(struct > extent_io_tree *tree, > } > > end = page_end; > - while (1) { > + while (1 && !parent_locked) { the patch is ok, just this caught my eye :) -- To

Re: [PATCH 4/4] btrfs: offline dedupe

2013-05-06 Thread David Sterba
On Tue, Apr 16, 2013 at 03:15:35PM -0700, Mark Fasheh wrote: > +static void btrfs_double_lock(struct inode *inode1, u64 loff1, > + struct inode *inode2, u64 loff2, u64 len) > +{ > + if (inode1 < inode2) { > + mutex_lock_nested(&inode1->i_mutex, I_MUTEX_PARE

[PATCH] Btrfs: introduce qgroup_ulist to avoid frequently allocating/freeing ulist

2013-05-06 Thread Wang Shilong
When doing qgroup accounting, we call ulist_alloc()/ulist_free() every time when we want to walk qgroup tree. By introducing 'qgroup_ulist', we only need to call ulist_alloc()/ulist_free() once. This reduce some sys time to allocate memory, see the measurements below fsstress -p 4 -n 1 -d $di

[PATCH V3] Btrfs: remove btrfs_sector_sum structure

2013-05-06 Thread Miao Xie
Using the structure btrfs_sector_sum to keep the checksum value is unnecessary, because the extents that btrfs_sector_sum points to are continuous, we can find out the expected checksums by btrfs_ordered_sum's bytenr and the offset, so we can remove btrfs_sector_sum's bytenr. After removing bytenr,

Re: hard freezes with 3.9.0 during io-intensive loads

2013-05-06 Thread Harald Glatt
I have this problem too, and I cannot reproduce it properly... When is that patch in btrfs-next going to be in the mainline kernel? On Mon, May 6, 2013 at 10:55 AM, Jan Schmidt wrote: > On Sun, May 05, 2013 at 18:10 (+0200), Kai Krakow wrote: >> Hello list, >> >> Kai Krakow schrieb: >> >>> I've

Re: hard freezes with 3.9.0 during io-intensive loads

2013-05-06 Thread Jan Schmidt
On Sun, May 05, 2013 at 18:10 (+0200), Kai Krakow wrote: > Hello list, > > Kai Krakow schrieb: > >> I've upgraded to 3.9.0 mainly for the snapshot-aware defragging patches. >> I'm running bedup[1] on a regular basis and it is now the third time that >> I got back to my PC just to find it hard-fr

[RFC 5/5] btrfs: add hot relocation support

2013-05-06 Thread zwu . kernel
From: Zhi Yong Wu Add one new mount option '-o hot_move' for hot relocation support. When hot relocation is enabled, hot tracking will be enabled automatically. Its usage looks like: mount -o hot_move mount -o nouser,hot_move mount -o nouser,hot_move,loop mount -o hot_move,nou

[RFC 4/5] procfs: add three proc interfaces

2013-05-06 Thread zwu . kernel
From: Zhi Yong Wu Add three proc interfaces hot-reloc-interval, hot-reloc-threshold, and hot-reloc-max-items under the dir /proc/sys/fs/ in order to turn HOT_RELOC_INTERVAL, HOT_RELOC_THRESHOLD, and HOT_RELOC_MAX_ITEMS into be tunable. Signed-off-by: Zhi Yong Wu --- fs/btrfs/hot_relocate.c |

[RFC 3/5] btrfs: add one hot relocation kthread

2013-05-06 Thread zwu . kernel
From: Zhi Yong Wu Add one private kthread for hot relocation. It will check if there're some extents which is hotter than the threshold and queue them at first, if no, it will return and wait for its next turn; otherwise, it will check if SSD ratio is beyond beyond its usage threshold, if no,

[RFC 2/5] btrfs: add one new block group

2013-05-06 Thread zwu . kernel
From: Zhi Yong Wu Introduce one new block group BTRFS_BLOCK_GROUP_DATA_SSD, which is used to differentiate if the block space is reserved and allocated from one HDD disk or SSD disk. Signed-off-by: Zhi Yong Wu --- fs/btrfs/Makefile | 3 +- fs/btrfs/ctree.h| 24 ++- f

[RFC 1/5] vfs: add one list_head field

2013-05-06 Thread zwu . kernel
From: Zhi Yong Wu Add one list_head field 'reloc_list' to accommodate hot relocation support. Signed-off-by: Zhi Yong Wu --- fs/hot_tracking.c| 1 + include/linux/hot_tracking.h | 1 + 2 files changed, 2 insertions(+) diff --git a/fs/hot_tracking.c b/fs/hot_tracking.c index 3b00

[RFC 0/5] BTRFS hot relocation support

2013-05-06 Thread zwu . kernel
From: Zhi Yong Wu The patchset as RFC is sent out mainly to see if it goes in the correct development direction. The patchset is trying to introduce hot relocation support for BTRFS. In hybrid storage environment, when the data in HDD disk get hot, it can be relocated to SSD disk by BTRFS ho

Re: hard freezes with 3.9.0 during io-intensive loads

2013-05-06 Thread Kai Krakow
Josef Bacik schrieb: >> I've upgraded to 3.9.0 mainly for the snapshot-aware defragging patches. >> I'm running bedup[1] on a regular basis and it is now the third time that >> I got back to my PC just to find it hard-frozen and I needed to use the >> reset button. >> >> It looks like this happe

Re: Possible to dedpulicate read-only snapshots for space-efficient backups

2013-05-06 Thread Kai Krakow
Jan Schmidt schrieb: >> I'm using an bash/rsync script[1] to backup my whole system on a nightly >> basis to an attached USB3 drive into a scratch area, then take a snapshot >> of this area. I'd like to have these snapshots immutable, so they should >> be read-only. > > Have you considered using