Freeze file system. hung_task_timeout_secs

2012-10-10 Thread Vladislav Vorobiev
Hi, my system freeze every day with (see screenshot) Kernel: 3.5-trunk-amd64 #1 SMP Thu Aug 2 17:16:27 UTC 2012 x86_64 GNU/Linux # btrfs filesystem df / Data: total=401.01GB, used=304.14GB System, DUP: total=8.00MB, used=64.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=106.12GB,

[RFC v3 01/13] btrfs: add one new mount option '-o hot_track'

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Introduce one new mount option '-o hot_track', and add its parsing support. Its usage looks like: mount -o hot_track mount -o nouser,hot_track mount -o nouser,hot_track,loop mount -o hot_track,nouser Signed-off-by: Zhi Yong Wu

[RFC v3 06/13] vfs: add hooks to enable hot data tracking

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Miscellaneous features that implement hot data tracking and generally make the hot data functions a bit more friendly. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/direct-io.c |8 fs/hot_tracking.h |5 +

[RFC v3 08/13] vfs: add aging function for old map info

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c | 57 + fs/hot_tracking.h |6 + 2 files changed, 63 insertions(+), 0 deletions(-) diff --git a/fs/hot_tracking.c

[RFC v3 10/13] vfs: register one memory shrinker

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Register a shrinker to control the amount of memory that is used in tracking hot regions - if we are throwing inodes out of memory due to memory pressure, we most definitely are going to need to reduce the amount of memory the tracking code is using,

[RFC v3 12/13] vfs: add debugfs support

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add a /sys/kernel/debug/hot_track/device_name/ directory for each volume that contains two files. The first, `inode_data', contains the heat information for inodes that have been brought into the hot data map structures. The second, `range_data',

[RFC v3 13/13] vfs: add documentation

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- Documentation/filesystems/00-INDEX |2 + Documentation/filesystems/hot_tracking.txt | 165 2 files changed, 167 insertions(+), 0 deletions(-) create mode

[RFC v3 07/13] vfs: add function for updating map arrays

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c | 153 + fs/hot_tracking.h | 60 + 2 files changed, 213 insertions(+), 0 deletions(-) diff --git

[RFC v3 09/13] vfs: add one wq to update map info periodically

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add a per-superblock workqueue and a work_struct to run periodic work to update map info on each superblock. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c| 94 ++

[RFC v3 04/13] vfs: add function for collecting raw access info

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add some utils helpers to update access frequencies for one file or its range. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c| 190 ++ fs/hot_tracking.h| 12

[RFC v3 02/13] vfs: introduce private radix tree structures

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com One root structure hot_info is defined, is hooked up in super_block, and will be used to hold radix tree root, hash list root and some other information, etc. Adds hot_inode_tree struct to keep track of frequently accessed files, and be keyed by

[RFC v3 03/13] vfs: Initialize and free main data structures

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add initialization function to create some key data structures when hot tracking is enabled; Clean up them when hot tracking is disabled Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/btrfs/super.c |8 +++

[RFC v3 05/13] vfs: add two map arrays

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Adds two map arrays which contains a lot of list and is used to efficiently look up the data temperature of a file or its ranges. In each list of map arrays, the array node will keep track of temperature info. Signed-off-by: Zhi Yong Wu

[RFC v3 00/13] vfs: hot data tracking

2012-10-10 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com NOTE: The patchset is currently post out mainly to make sure it is going in the correct direction and hope to get some helpful comments from other guys. For more infomation, please check hot_tracking.txt in Documentation TODO List: 1.) Need to do

[PATCH] Btrfs: extended inode refs support for send mechanism

2012-10-10 Thread Jan Schmidt
This adds support for the new extended inode refs to btrfs send. Signed-off-by: Jan Schmidt list.bt...@jan-o-sch.net --- fs/btrfs/backref.c | 22 - fs/btrfs/backref.h |4 ++ fs/btrfs/send.c| 129 ++-- 3 files changed, 99

Re: [RFC v3 01/13] btrfs: add one new mount option '-o hot_track'

2012-10-10 Thread Zhi Yong Wu
On Wed, Oct 10, 2012 at 7:59 PM, Lukáš Czerner lczer...@redhat.com wrote: On Wed, 10 Oct 2012, zwu.ker...@gmail.com wrote: Date: Wed, 10 Oct 2012 18:07:23 +0800 From: zwu.ker...@gmail.com To: linux-fsde...@vger.kernel.org Cc: linux-e...@vger.kernel.org, linux-btrfs@vger.kernel.org,

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-10 Thread Chris Mason
On Tue, Oct 09, 2012 at 03:00:12AM -0600, David Sterba wrote: On Tue, Oct 09, 2012 at 08:33:57AM +0100, Richard W.M. Jones wrote: On Tue, Oct 09, 2012 at 08:20:02AM +0100, Richard W.M. Jones wrote: On Mon, Oct 08, 2012 at 08:00:51PM -0400, Chris Mason wrote: Ok, what's a rough idea of

Re: [RFC v3 01/13] btrfs: add one new mount option '-o hot_track'

2012-10-10 Thread Lukáš Czerner
On Wed, 10 Oct 2012, Zhi Yong Wu wrote: Date: Wed, 10 Oct 2012 20:21:48 +0800 From: Zhi Yong Wu zwu.ker...@gmail.com To: Lukáš Czerner lczer...@redhat.com Cc: linux-fsde...@vger.kernel.org, linux-e...@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-ker...@vger.kernel.org,

Re: [RFC v3 01/13] btrfs: add one new mount option '-o hot_track'

2012-10-10 Thread Zhi Yong Wu
On Wed, Oct 10, 2012 at 9:11 PM, Lukáš Czerner lczer...@redhat.com wrote: On Wed, 10 Oct 2012, Zhi Yong Wu wrote: Date: Wed, 10 Oct 2012 20:21:48 +0800 From: Zhi Yong Wu zwu.ker...@gmail.com To: Lukáš Czerner lczer...@redhat.com Cc: linux-fsde...@vger.kernel.org, linux-e...@vger.kernel.org,

[PATCH] Fix a sign bug causing invalid memory access in the ino_paths ioctl.

2012-10-10 Thread Gabriel de Perthuis
To see the problem, create many hardlinks to the same file (120 should do it), then look up paths by inode with: ls -i btrfs inspect inode-resolve -v $ino /mnt/btrfs I noticed the memory layout of the fspath-val data had some irregularities (some unnecessary gaps that stop appearing about

Re: [RFC v3 02/13] vfs: introduce private radix tree structures

2012-10-10 Thread David Sterba
On Wed, Oct 10, 2012 at 06:07:24PM +0800, zwu.ker...@gmail.com wrote: +void hot_track_init(struct super_block *sb) +{ ... +} +void hot_track_exit(struct super_block *sb) +{ + hot_cache_exit(); +} Needs to be exported if btrfs is built as a module, otherwise does not link LDS

Re: [RFC v3 12/13] vfs: add debugfs support

2012-10-10 Thread David Sterba
On Wed, Oct 10, 2012 at 06:07:34PM +0800, zwu.ker...@gmail.com wrote: +static int hot_debugfs_log_init(struct debugfs_vol_data *data) +{ + int err = 0; + struct lstring *debugfs_log = data-debugfs_log; + + spin_lock(data-log_lock); + debugfs_log-str =

Re: [RFC v3 01/13] btrfs: add one new mount option '-o hot_track'

2012-10-10 Thread David Sterba
Hi, On Wed, Oct 10, 2012 at 06:07:23PM +0800, zwu.ker...@gmail.com wrote: --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -1726,6 +1726,7 @@ struct btrfs_ioctl_defrag_range_args { #define BTRFS_MOUNT_CHECK_INTEGRITY (1 20) #define BTRFS_MOUNT_CHECK_INTEGRITY_INCLUDING_EXTENT_DATA (1

Re: [PATCH][BTRFS-PROGS][V3] btrfs filesystem df

2012-10-10 Thread Goffredo Baroncelli
On 10/10/2012 12:15 AM, David Sterba wrote: On Tue, Oct 09, 2012 at 08:07:51PM +0200, Goffredo Baroncelli wrote: [...] * show the byte units At the beginning it was so. But the space required was greater than 80 columns. Moreover the allocation space is in multiply of 4KiB, so showing the

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-10 Thread Richard W.M. Jones
On Wed, Oct 10, 2012 at 08:38:08AM -0400, Chris Mason wrote: On Tue, Oct 09, 2012 at 03:00:12AM -0600, David Sterba wrote: On Tue, Oct 09, 2012 at 08:33:57AM +0100, Richard W.M. Jones wrote: On Tue, Oct 09, 2012 at 08:20:02AM +0100, Richard W.M. Jones wrote: On Mon, Oct 08, 2012 at

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-10 Thread Chris Mason
On Wed, Oct 10, 2012 at 01:38:53PM -0600, Richard W.M. Jones wrote: On Wed, Oct 10, 2012 at 08:38:08AM -0400, Chris Mason wrote: On Tue, Oct 09, 2012 at 03:00:12AM -0600, David Sterba wrote: On Tue, Oct 09, 2012 at 08:33:57AM +0100, Richard W.M. Jones wrote: On Tue, Oct 09, 2012 at

Re: Anyone seeing lots of Check tree block failed and other errors with latest kernel?

2012-10-10 Thread Richard W.M. Jones
On Wed, Oct 10, 2012 at 03:41:13PM -0400, Chris Mason wrote: On Wed, Oct 10, 2012 at 01:38:53PM -0600, Richard W.M. Jones wrote: On Wed, Oct 10, 2012 at 08:38:08AM -0400, Chris Mason wrote: On Tue, Oct 09, 2012 at 03:00:12AM -0600, David Sterba wrote: On Tue, Oct 09, 2012 at 08:33:57AM

Re: block rsv returned -28 during balance

2012-10-10 Thread Mitch Harder
On Mon, Oct 1, 2012 at 1:28 AM, Roman Mamedov r...@romanrm.ru wrote: Hello, On a 3.6.0-rc7 kernel, I launched: # btrfs fi balance start -f -mconvert=single /mnt/tmp/ Current situation: # df -h /mnt/tmp/ Filesystem Size Used Avail Use% Mounted on /dev/mapper/alpha-lv1

Re: btrfs send/receive review by vfs folks

2012-10-10 Thread Alex Lyakas
Hi Jan, On Mon, Oct 8, 2012 at 3:37 PM, Jan Schmidt list.bt...@jan-o-sch.net wrote: On Mon, October 08, 2012 at 13:38 (+0200), Alex Lyakas wrote: I realize this is a big change, and a new IOCTL has to be introduced in order not to break current user-kernel protocol. The pros as I see them: #

Re: [RFC v3 12/13] vfs: add debugfs support

2012-10-10 Thread David Sterba
On Wed, Oct 10, 2012 at 06:07:34PM +0800, zwu.ker...@gmail.com wrote: +static int hot_debugfs_copy(struct debugfs_vol_data *data, char *msg, int len) +{ + struct lstring *debugfs_log = data-debugfs_log; + uint new_log_alloc_size; + char *new_log; + static char err_msg[] =

Re: btrfs receive to subdirectory

2012-10-10 Thread Rory Campbell-Lange
On 09/10/12, Alex Lyakas (alex.bt...@zadarastorage.com) wrote: Hi Rory, Arne, I think the problem is that currently mnt_fd in struct btrfs_receive is used both as mount root and directory in which the subvolume/snapshot needs to be created. Arne, does the following patch make sense? It uses

Re: [PATCH] Btrfs-progs: introduce '-p' option and fullpath into subvolume set-default command

2012-10-10 Thread Chen Yang
On 2012-10-9 22:44, David Sterba wrote: On Mon, Sep 24, 2012 at 02:42:13PM +0800, Chen Yang wrote: In command btrfs subvolume set-default, we used subvolume id and path to set the default subvolume of a filesystem. It's not easy for a common user, What is not easy? How often do you