Re: [RFC v4 03/15] vfs,hot_track: add the function for collecting I/O frequency

2012-10-28 Thread Zheng Liu
Hi Zhiyong, On Thu, Oct 25, 2012 at 11:08:55PM +0800, zwu.ker...@gmail.com wrote: [snip] @@ -199,6 +342,54 @@ err: } /* + * Main function to update access frequency from read/writepage(s) hooks + */ +inline void hot_update_freqs(struct inode *inode, u64 start, +

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Goffredo Baroncelli
On 2012-10-28 00:38, Hugo Mills wrote: On Sun, Oct 28, 2012 at 12:30:44AM +0200, Martin Steigerwald wrote: Am Samstag, 27. Oktober 2012 schrieb Michael Kjörling: On 27 Oct 2012 18:43 +0200, from mar...@lichtvoll.de (Martin Steigerwald): Possibly this could be done tabular as well, like:

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Goffredo Baroncelli
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2012-10-25 21:21, Goffredo Baroncelli wrote: Hi all, this is a new attempt to improve the output of the command btrfs fi df. Below you can see another iteration. I tried to address all the cwillu requests, which to me make sense. I thought a

Re: Naming of (bootable) subvolumes

2012-10-28 Thread Goffredo Baroncelli
On 2012-10-27 19:22, Chris Murphy wrote: On Oct 26, 2012, at 9:03 PM, Fajar A. Nugraha l...@fajar.net wrote: So back to the original question, I'd suggest NOT to use either send/receive or set-default. Instead, setup multiple boot environment (e.g. old version, current version) and let

Re: Naming of (bootable) subvolumes

2012-10-28 Thread Fajar A. Nugraha
On Sun, Oct 28, 2012 at 12:22 AM, Chris Murphy li...@colorremedies.com wrote: On Oct 26, 2012, at 9:03 PM, Fajar A. Nugraha l...@fajar.net wrote: So back to the original question, I'd suggest NOT to use either send/receive or set-default. Instead, setup multiple boot environment (e.g. old

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Martin Steigerwald
Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli: On 2012-10-25 21:21, Goffredo Baroncelli wrote: Hi all, this is a new attempt to improve the output of the command btrfs fi df. Below you can see another iteration. I tried to address all the cwillu requests, which to me make

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Martin Steigerwald
Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli: On 2012-10-28 00:38, Hugo Mills wrote: On Sun, Oct 28, 2012 at 12:30:44AM +0200, Martin Steigerwald wrote: Am Samstag, 27. Oktober 2012 schrieb Michael Kjörling: On 27 Oct 2012 18:43 +0200, from mar...@lichtvoll.de (Martin

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Goffredo Baroncelli
On 2012-10-28 11:33, Martin Steigerwald wrote: Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli: [...] $ sudo ./btrfs fi df /mnt/btrfs1/ Path: /mnt/btrfs1 Summary: Disk_size: 21.00GB Disk_allocated: 1.83GB Disk_unallocated:19.17GB Used:

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Martin Steigerwald
Michael, please keep CCs. Its usual to keep them on kernel related lists. Thanks. Am Sonntag, 28. Oktober 2012 schrieb Michael Kjörling: On 27 Oct 2012 23:38 +0100, from h...@carfax.org.uk (Hugo Mills): Data: RAID 0 System: RAID 1 Unused /dev/vdb 307.25 MB

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Goffredo Baroncelli
On 2012-10-28 11:38, Martin Steigerwald wrote: Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli: On 2012-10-28 00:38, Hugo Mills wrote: On Sun, Oct 28, 2012 at 12:30:44AM +0200, Martin Steigerwald wrote: Am Samstag, 27. Oktober 2012 schrieb Michael Kjörling: On 27 Oct 2012 18:43

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Martin Steigerwald
Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli: What was your reasoning for not using options to btrfs filesystem df? That df doesn´t show more than disk free as well? My feel is that a switch should change a bit a command. In this case there are very different outputs, for

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Michael Kjörling
On 28 Oct 2012 11:59 +0100, from kreij...@gmail.com (Goffredo Baroncelli): On 2012-10-28 11:38, Martin Steigerwald wrote: But still if if can be arbitrarily long due to that per object replication config, a vertical output might and leaving graphical representation to a Qt Quick application

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Michael Kjörling
On 27 Oct 2012 17:35 -0600, from li...@colorremedies.com (Chris Murphy): On Oct 27, 2012, at 1:55 PM, Michael Kjörling mich...@kjorling.se wrote: Data: RAID 0 System: RAID 1 Unused /dev/vdb 307.25 MB-2.23 GB /dev/vdc 307.25 MB 8 MB

Re: btrfs seems to do COW while inode has NODATACOW set

2012-10-28 Thread Alex Lyakas
Hi, it appears that I found why the COW is happening. The code in the kernel that triggers this is: check_committed_ref(): if (btrfs_extent_generation(leaf, ei) = btrfs_root_last_snapshot(root-root_item)) goto out; It appears that both extent_generation and

Re: How does btrfs behave on checksum mismatch?

2012-10-28 Thread Ronnie Collinson
In a raid1 situation, it will also rewrite the effected data, on the drive that failed the checksum -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Goffredo Baroncelli
On 2012-10-28 12:18, Michael Kjörling wrote: On 28 Oct 2012 11:59 +0100, from kreij...@gmail.com (Goffredo Baroncelli): On 2012-10-28 11:38, Martin Steigerwald wrote: But still if if can be arbitrarily long due to that per object replication config, a vertical output might and leaving

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Michael Kjörling
On 28 Oct 2012 13:25 +0100, from kreij...@inwind.it (Goffredo Baroncelli): On 2012-10-28 12:18, Michael Kjörling wrote: I would personally really prefer a tabular view that perhaps does not show _all_ details when no explicit parameters are given, and a --dump or similar that will give a

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Martin Steigerwald
Am Sonntag, 28. Oktober 2012 schrieb Goffredo Baroncelli: On 2012-10-28 12:18, Michael Kjörling wrote: On 28 Oct 2012 11:59 +0100, from kreij...@gmail.com (Goffredo Baroncelli): On 2012-10-28 11:38, Martin Steigerwald wrote: But still if if can be arbitrarily long due to that per object

Re: How does btrfs behave on checksum mismatch?

2012-10-28 Thread Martin Steigerwald
Am Sonntag, 28. Oktober 2012 schrieb Ronnie Collinson: In a raid1 situation, it will also rewrite the effected data, on the drive that failed the checksum Will it do so without an explicit scrub? -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B

Re: How does btrfs behave on checksum mismatch?

2012-10-28 Thread Hugo Mills
On Sun, Oct 28, 2012 at 02:23:51PM +0100, Martin Steigerwald wrote: Am Sonntag, 28. Oktober 2012 schrieb Ronnie Collinson: In a raid1 situation, it will also rewrite the effected data, on the drive that failed the checksum Will it do so without an explicit scrub? If a failed checksum

Re: How does btrfs behave on checksum mismatch?

2012-10-28 Thread Martin Steigerwald
Am Sonntag, 28. Oktober 2012 schrieb Hugo Mills: On Sun, Oct 28, 2012 at 02:23:51PM +0100, Martin Steigerwald wrote: Am Sonntag, 28. Oktober 2012 schrieb Ronnie Collinson: In a raid1 situation, it will also rewrite the effected data, on the drive that failed the checksum Will it do

Re: How does btrfs behave on checksum mismatch?

2012-10-28 Thread Hugo Mills
On Sun, Oct 28, 2012 at 02:36:24PM +0100, Martin Steigerwald wrote: Am Sonntag, 28. Oktober 2012 schrieb Hugo Mills: On Sun, Oct 28, 2012 at 02:23:51PM +0100, Martin Steigerwald wrote: Am Sonntag, 28. Oktober 2012 schrieb Ronnie Collinson: In a raid1 situation, it will also rewrite the

Re: [RFC v4 03/15] vfs,hot_track: add the function for collecting I/O frequency

2012-10-28 Thread Zhi Yong Wu
On Sun, Oct 28, 2012 at 3:55 PM, Zheng Liu gnehzuil@gmail.com wrote: Hi Zhiyong, On Thu, Oct 25, 2012 at 11:08:55PM +0800, zwu.ker...@gmail.com wrote: [snip] @@ -199,6 +342,54 @@ err: } /* + * Main function to update access frequency from read/writepage(s) hooks + */ +inline void

Re: Naming of (bootable) subvolumes

2012-10-28 Thread Chris Murphy
On Oct 28, 2012, at 3:25 AM, Goffredo Baroncelli kreij...@inwind.it wrote: On Sun, Oct 28, 2012 at 12:22 AM, Chris Murphy li...@colorremedies.com wrote: On my computer, when update-grub is called, it scans for all the subvolumes and show them in the menu, allowing the user to switch at

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Chris Murphy
On Oct 28, 2012, at 5:16 AM, Martin Steigerwald mar...@lichtvoll.de wrote: The question whether how these are all related to the question on how much space is free? This is my main confusion with 'filesystem df' is that it's very circumspect about this. For a way longer time than I think is

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Michael Kjörling
On 28 Oct 2012 12:27 -0600, from li...@colorremedies.com (Chris Murphy): I think we're all trained to see file size as absolute, even when the file is in effect duplicated. I think it's possible we need to be retrained to look at this from a storage point of view. The storage pool is 200GB in

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Chris Murphy
On Oct 28, 2012, at 1:06 PM, Michael Kjörling mich...@kjorling.se wrote: This makes a valid point. And there's another issue potentially at play: file system level compression, _which btrfs already supports_. Say you store a highly compressible file (a web server access log, perhaps) on a

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Chris Murphy
On Oct 28, 2012, at 1:06 PM, Michael Kjörling mich...@kjorling.se wrote: This makes a valid point. And there's another issue potentially at play: file system level compression, _which btrfs already supports_. Say you store a highly compressible file (a web server access log, perhaps) on a

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Michael Kjörling
On 28 Oct 2012 13:42 -0600, from ch...@colorremedies.com (Chris Murphy): * btrfs _is_ the file system administration tool. In a way, it makes sense that the data provided by it will be geared more toward technically minded people. It is a fair point. But this is not LVM or md raid. It's

Re: [RFC] New attempt to a better btrfs fi df

2012-10-28 Thread Chris Murphy
On Oct 28, 2012, at 2:09 PM, Michael Kjörling mich...@kjorling.se wrote: I did not mean to imply that sysadmins and storage experts aren't also ordinary people who can benefit from an at a glance view of their storage situation. Quite to the contrary, that's one point I have raised in this

warnings for silly/pointless raid levels

2012-10-28 Thread sam tygier
Hi, I recently got into a stick situation because i had a btrfs volume with data in raid1 but metadata as dup. when i removed one of the drives i could not mount as degraded. This was my error as i did not convert the metadata when i converted it from a single/dup volume. But i wonder if there

Re: [RFC v4 03/15] vfs,hot_track: add the function for collecting I/O frequency

2012-10-28 Thread Dave Chinner
On Sun, Oct 28, 2012 at 09:51:48PM +0800, Zhi Yong Wu wrote: On Sun, Oct 28, 2012 at 3:55 PM, Zheng Liu gnehzuil@gmail.com wrote: Hi Zhiyong, On Thu, Oct 25, 2012 at 11:08:55PM +0800, zwu.ker...@gmail.com wrote: [snip] @@ -199,6 +342,54 @@ err: } /* + * Main function to

Re: [RFC v4 03/15] vfs,hot_track: add the function for collecting I/O frequency

2012-10-28 Thread Zhi Yong Wu
On Mon, Oct 29, 2012 at 10:01 AM, Dave Chinner da...@fromorbit.com wrote: On Sun, Oct 28, 2012 at 09:51:48PM +0800, Zhi Yong Wu wrote: On Sun, Oct 28, 2012 at 3:55 PM, Zheng Liu gnehzuil@gmail.com wrote: Hi Zhiyong, On Thu, Oct 25, 2012 at 11:08:55PM +0800, zwu.ker...@gmail.com wrote:

Re: Need help mounting laptop corrupted root btrfs. Kernel BUG at fs/btrfs/volumes.c:3707

2012-10-28 Thread Marc MERLIN
On Thu, Oct 25, 2012 at 01:12:23PM -0700, Marc MERLIN wrote: On Thu, Oct 25, 2012 at 02:03:49PM -0600, cwillu wrote: 3) Want me to try btrfsck although it may make it impossible for me to reproduce the bug and test a fix, as well as potentially break the filesystem more (last time I

[RFC v4+ hot_track 16/19] btrfs: add hot tracking support

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Introduce one new mount option '-o hot_track', and add its parsing support. Its usage looks like: mount -o hot_track mount -o nouser,hot_track mount -o nouser,hot_track,loop mount -o hot_track,nouser Signed-off-by: Zhi Yong Wu

[RFC v4+ hot_track 18/19] ext4: add hot tracking support

2012-10-28 Thread zwu . kernel
From: Zheng Liu wenqing...@taobao.com Define a new mount option to add VFS hot tracking support in order to use it in ext4. CC: Zhi Yong Wu zwu.ker...@gmail.com Signed-off-by: Zheng Liu wenqing...@taobao.com --- fs/ext4/ext4.h |3 +++ fs/ext4/super.c | 13 - 2 files

[RFC v4+ hot_track 19/19] vfs: add documentation

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add one doc for VFS hot tracking feature Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- Documentation/filesystems/00-INDEX |2 + Documentation/filesystems/hot_tracking.txt | 262 2 files changed, 264

[RFC v4+ hot_track 15/19] sysfs: add two hot_track proc files

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add two proc files hot-kick-time and hot-update-delay under the dir /proc/sys/fs/ in order to turn TIME_TO_KICK and HEAT_UPDATE_DELAY into be tunable. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c| 12

[RFC v4+ hot_track 13/19] debugfs: introduce one function

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com The debugfs function is used to get expected dentry. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/debugfs/inode.c | 26 ++ include/linux/debugfs.h |9 + 2 files changed, 35 insertions(+), 0

[RFC v4+ hot_track 17/19] xfs: add hot tracking support

2012-10-28 Thread zwu . kernel
From: Dave Chinner dchin...@redhat.com Connect up the VFS hot tracking support so XFS filesystems can make use of it. Signed-off-by: Dave Chinner dchin...@redhat.com --- fs/xfs/xfs_mount.h |1 + fs/xfs/xfs_super.c | 16 2 files changed, 17 insertions(+), 0 deletions(-)

[RFC v4+ hot_track 14/19] vfs: add debugfs support

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add a /sys/kernel/debug/hot_track/device_name/ directory for each volume that contains two files. The first, `inode_stats', contains the heat information for inodes that have been brought into the hot data map structures. The second, `range_stats',

[RFC v4+ hot_track 04/19] vfs: add two map arrays

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Adds two map arrays which contains a lot of list and is used to efficiently look up the data temperature of a file or its ranges. In each list of map arrays, the array node will keep track of temperature info. Signed-off-by: Zhi Yong Wu

[RFC v4+ hot_track 06/19] vfs: add temp calculation function

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c | 74 + 1 files changed, 74 insertions(+), 0 deletions(-) diff --git a/fs/hot_tracking.c b/fs/hot_tracking.c index

[RFC v4+ hot_track 07/19] vfs: add map info update function

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c | 66 + fs/hot_tracking.h | 21 + 2 files changed, 87 insertions(+), 0 deletions(-) diff --git

[RFC v4+ hot_track 12/19] vfs: add one ioctl interface

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com FS_IOC_GET_HEAT_INFO: return a struct containing the various metrics collected in hot_freq_data structs, and also return a calculated data temperature based on those metrics. Optionally, retrieve the temperature from the hot data hash list instead of

[RFC v4+ hot_track 09/19] vfs: add one work queue

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add a per-superblock workqueue and a delayed_work to run periodic work to update map info on each superblock. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c| 85 ++

[RFC v4+ hot_track 10/19] vfs: introduce hot func register framework

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Introduce one framwork to enable that specific FS can register its own hot tracking functions. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c| 78 ++

[RFC v4+ hot_track 05/19] vfs: add hooks to enable hot tracking

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Miscellaneous features that implement hot data tracking and generally make the hot data functions a bit more friendly. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/direct-io.c |6 ++ mm/filemap.c|6 ++

[RFC v4+ hot_track 03/19] vfs: add I/O frequency update function

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add some util helpers to update access frequencies for one file or its range. Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c| 179 ++ fs/hot_tracking.h|7

[RFC v4+ hot_track 00/19] vfs: hot data tracking

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com NOTE: The patchset can be obtained via my kernel dev git on github: g...@github.com:wuzhy/kernel.git hot_tracking If you're interested, you can also can review them via https://github.com/wuzhy/kernel/commits/hot_tracking For more info, please

[RFC v4+ hot_track 01/19] vfs: introduce private radix tree structures

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com One root structure hot_info is defined, is hooked up in super_block, and will be used to hold radix tree root, hash list root and some other information, etc. Adds hot_inode_tree struct to keep track of frequently accessed files, and be keyed by

[RFC v4+ hot_track 02/19] vfs: initialize and free data structures

2012-10-28 Thread zwu . kernel
From: Zhi Yong Wu wu...@linux.vnet.ibm.com Add initialization function to create some key data structures when hot tracking is enabled; Clean up them when hot tracking is disabled Signed-off-by: Zhi Yong Wu wu...@linux.vnet.ibm.com --- fs/hot_tracking.c| 124

Re: Need help mounting laptop corrupted root btrfs. Kernel BUG at fs/btrfs/volumes.c:3707

2012-10-28 Thread Chris Murphy
On Oct 25, 2012, at 2:12 PM, Marc MERLIN m...@merlins.org wrote: I also have hourly, daily, weekly btrfs subvolume snapshots, but I can't use those currently since I can't mount the base filesystem. It might be worth unmounting it. Then only remounting a snapshot well before the problem