Re: [RFC v3 00/13] vfs: hot data tracking
On Tue, Oct 16, 2012 at 4:42 AM, Dave Chinner wrote: > On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> NOTE: >> >> The patchset is currently post out mainly to make sure >> it is going in the correct direction and hope to get some >> helpful comments from other guys. >> For more infomation, please check hot_tracking.txt in Documentation >> >> TODO List: > > 1) Fix OOM issues - the hot inode tracking caches grow very large > and don't get trimmed under memory pressure. From slabtop, after > creating roughly 24 million single byte files(*) on a machine with > 8GB RAM: > > OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME > 23859510 23859476 99%0.12K 795317 30 3181268K hot_range_item > 23859441 23859439 99%0.16K 1037367 23 4149468K hot_inode_item > 572530 572530 100%0.55K 817907327160K radix_tree_node > 241706 241406 99%0.22K 14218 17 56872K xfs_ili > 241206 241204 99%1.06K 804023321608K xfs_inode > > The inode tracking is trying to track all 24 million inodes even > though they have been written only once, and there are only 240,000 > inodes in the cache at this point in time. That was the last update > that slabtop got, so it is indicative of the impending OOM situation > that occurred. > >> Changelog from v2: >> 1.) Converted to Radix trees, not RB-tree [Zhiyong, Dave Chinner] >> 2.) Added memory shrinker [Dave Chinner] > > I haven't looked at the shrinker, but clearly it is not working, HI, Dave, Some guys suggest that when inode slab cache is shrinked, the hot_inode[range]_item slab is accordingly also shrinked, this will make hot tracking don't need to register its own shrinker. Do you think of it? If you don't like above idea. Do you have any good suggestion on how to remove hot_inode_item and hot_range_item? > otherwise the above OOM situation would not be occurring. > > Cheers, > > Dave. > > (*) Tested on an empty 17TB XFS filesystem with: > > $ sudo mkfs.xfs -f -l size=131072b,sunit=8 /dev/vdc > meta-data=/dev/vdc isize=256agcount=17, agsize=268435455 > blks > = sectsz=512 attr=2, projid32bit=0 > data = bsize=4096 blocks=4563402735, imaxpct=5 > = sunit=0 swidth=0 blks > naming =version 2 bsize=4096 ascii-ci=0 > log =internal log bsize=4096 blocks=131072, version=2 > = sectsz=512 sunit=1 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > $ sudo mount -o logbsize=256k /dev/vdc /mnt/scratch > $ sudo chmod 777 /mnt/scratch > $ fs_mark -D 1 -S0 -n 10 -s 1 -L 63 -d \ > /mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d \ > /mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d \ > /mnt/scratch/6 -d /mnt/scratch/7 > . > 0 21601 16679.3 12552262 > 0 22401 15412.4 12588587 > 0 23201 16367.6 14199322 > 0 24001 15680.4 15741205 > > > -- > Dave Chinner > da...@fromorbit.com -- Regards, Zhi Yong Wu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Thu, Oct 18, 2012 at 1:17 PM, Dave Chinner wrote: > On Thu, Oct 18, 2012 at 12:44:47PM +0800, Zhi Yong Wu wrote: >> On Thu, Oct 18, 2012 at 12:29 PM, Dave Chinner wrote: >> > On Wed, Oct 17, 2012 at 04:57:14PM +0800, Zhi Yong Wu wrote: >> >> On Tue, Oct 16, 2012 at 4:42 AM, Dave Chinner wrote: >> >> > On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: >> >> >> From: Zhi Yong Wu >> > >> >> > (*) Tested on an empty 17TB XFS filesystem with: >> >> > >> >> > $ sudo mkfs.xfs -f -l size=131072b,sunit=8 /dev/vdc >> >> > meta-data=/dev/vdc isize=256agcount=17, >> >> > agsize=268435455 blks >> >> > = sectsz=512 attr=2, projid32bit=0 >> >> > data = bsize=4096 blocks=4563402735, >> >> > imaxpct=5 >> >> > = sunit=0 swidth=0 blks >> >> > naming =version 2 bsize=4096 ascii-ci=0 >> >> > log =internal log bsize=4096 blocks=131072, version=2 >> >> > = sectsz=512 sunit=1 blks, lazy-count=1 >> >> > realtime =none extsz=4096 blocks=0, rtextents=0 >> >> > $ sudo mount -o logbsize=256k /dev/vdc /mnt/scratch >> >> > $ sudo chmod 777 /mnt/scratch >> >> > $ fs_mark -D 1 -S0 -n 10 -s 1 -L 63 -d \ >> >> > /mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d \ >> >> > /mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d \ >> >> > /mnt/scratch/6 -d /mnt/scratch/7 >> >> > . >> >> > 0 21601 16679.3 12552262 >> >> > 0 22401 15412.4 12588587 >> >> > 0 23201 16367.6 14199322 >> >> > 0 24001 15680.4 15741205 >> >> > >> >> In this test, i haven't see you enable hot_track function via >> >> mount, why did it meet OOM? >> > >> > I copied the wrong mount command. It was definitely enabled. >> OK, BTW: fs_mark is the script written by you? After xfsprogs is >> installed, i haven't found this command. > > # apt-get install fsmark thanks. let me try. > > Or get the source here: > > http://sourceforge.net/projects/fsmark/ > > Cheers, > > Dave. > -- > Dave Chinner > da...@fromorbit.com -- Regards, Zhi Yong Wu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Thu, Oct 18, 2012 at 12:44:47PM +0800, Zhi Yong Wu wrote: > On Thu, Oct 18, 2012 at 12:29 PM, Dave Chinner wrote: > > On Wed, Oct 17, 2012 at 04:57:14PM +0800, Zhi Yong Wu wrote: > >> On Tue, Oct 16, 2012 at 4:42 AM, Dave Chinner wrote: > >> > On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: > >> >> From: Zhi Yong Wu > > > >> > (*) Tested on an empty 17TB XFS filesystem with: > >> > > >> > $ sudo mkfs.xfs -f -l size=131072b,sunit=8 /dev/vdc > >> > meta-data=/dev/vdc isize=256agcount=17, > >> > agsize=268435455 blks > >> > = sectsz=512 attr=2, projid32bit=0 > >> > data = bsize=4096 blocks=4563402735, > >> > imaxpct=5 > >> > = sunit=0 swidth=0 blks > >> > naming =version 2 bsize=4096 ascii-ci=0 > >> > log =internal log bsize=4096 blocks=131072, version=2 > >> > = sectsz=512 sunit=1 blks, lazy-count=1 > >> > realtime =none extsz=4096 blocks=0, rtextents=0 > >> > $ sudo mount -o logbsize=256k /dev/vdc /mnt/scratch > >> > $ sudo chmod 777 /mnt/scratch > >> > $ fs_mark -D 1 -S0 -n 10 -s 1 -L 63 -d \ > >> > /mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d \ > >> > /mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d \ > >> > /mnt/scratch/6 -d /mnt/scratch/7 > >> > . > >> > 0 21601 16679.3 12552262 > >> > 0 22401 15412.4 12588587 > >> > 0 23201 16367.6 14199322 > >> > 0 24001 15680.4 15741205 > >> > > >> In this test, i haven't see you enable hot_track function via > >> mount, why did it meet OOM? > > > > I copied the wrong mount command. It was definitely enabled. > OK, BTW: fs_mark is the script written by you? After xfsprogs is > installed, i haven't found this command. # apt-get install fsmark Or get the source here: http://sourceforge.net/projects/fsmark/ Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Thu, Oct 18, 2012 at 12:29 PM, Dave Chinner wrote: > On Wed, Oct 17, 2012 at 04:57:14PM +0800, Zhi Yong Wu wrote: >> On Tue, Oct 16, 2012 at 4:42 AM, Dave Chinner wrote: >> > On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: >> >> From: Zhi Yong Wu > >> > (*) Tested on an empty 17TB XFS filesystem with: >> > >> > $ sudo mkfs.xfs -f -l size=131072b,sunit=8 /dev/vdc >> > meta-data=/dev/vdc isize=256agcount=17, agsize=268435455 >> > blks >> > = sectsz=512 attr=2, projid32bit=0 >> > data = bsize=4096 blocks=4563402735, imaxpct=5 >> > = sunit=0 swidth=0 blks >> > naming =version 2 bsize=4096 ascii-ci=0 >> > log =internal log bsize=4096 blocks=131072, version=2 >> > = sectsz=512 sunit=1 blks, lazy-count=1 >> > realtime =none extsz=4096 blocks=0, rtextents=0 >> > $ sudo mount -o logbsize=256k /dev/vdc /mnt/scratch >> > $ sudo chmod 777 /mnt/scratch >> > $ fs_mark -D 1 -S0 -n 10 -s 1 -L 63 -d \ >> > /mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d \ >> > /mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d \ >> > /mnt/scratch/6 -d /mnt/scratch/7 >> > . >> > 0 21601 16679.3 12552262 >> > 0 22401 15412.4 12588587 >> > 0 23201 16367.6 14199322 >> > 0 24001 15680.4 15741205 >> > >> In this test, i haven't see you enable hot_track function via >> mount, why did it meet OOM? > > I copied the wrong mount command. It was definitely enabled. OK, BTW: fs_mark is the script written by you? After xfsprogs is installed, i haven't found this command. > > Cheers, > > Dave. > > -- > Dave Chinner > da...@fromorbit.com -- Regards, Zhi Yong Wu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Wed, Oct 17, 2012 at 04:57:14PM +0800, Zhi Yong Wu wrote: > On Tue, Oct 16, 2012 at 4:42 AM, Dave Chinner wrote: > > On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: > >> From: Zhi Yong Wu > > (*) Tested on an empty 17TB XFS filesystem with: > > > > $ sudo mkfs.xfs -f -l size=131072b,sunit=8 /dev/vdc > > meta-data=/dev/vdc isize=256agcount=17, agsize=268435455 > > blks > > = sectsz=512 attr=2, projid32bit=0 > > data = bsize=4096 blocks=4563402735, imaxpct=5 > > = sunit=0 swidth=0 blks > > naming =version 2 bsize=4096 ascii-ci=0 > > log =internal log bsize=4096 blocks=131072, version=2 > > = sectsz=512 sunit=1 blks, lazy-count=1 > > realtime =none extsz=4096 blocks=0, rtextents=0 > > $ sudo mount -o logbsize=256k /dev/vdc /mnt/scratch > > $ sudo chmod 777 /mnt/scratch > > $ fs_mark -D 1 -S0 -n 10 -s 1 -L 63 -d \ > > /mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d \ > > /mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d \ > > /mnt/scratch/6 -d /mnt/scratch/7 > > . > > 0 21601 16679.3 12552262 > > 0 22401 15412.4 12588587 > > 0 23201 16367.6 14199322 > > 0 24001 15680.4 15741205 > > > In this test, i haven't see you enable hot_track function via > mount, why did it meet OOM? I copied the wrong mount command. It was definitely enabled. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Tue, Oct 16, 2012 at 4:42 AM, Dave Chinner wrote: > On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> NOTE: >> >> The patchset is currently post out mainly to make sure >> it is going in the correct direction and hope to get some >> helpful comments from other guys. >> For more infomation, please check hot_tracking.txt in Documentation >> >> TODO List: > > 1) Fix OOM issues - the hot inode tracking caches grow very large > and don't get trimmed under memory pressure. From slabtop, after > creating roughly 24 million single byte files(*) on a machine with > 8GB RAM: > > OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME > 23859510 23859476 99%0.12K 795317 30 3181268K hot_range_item > 23859441 23859439 99%0.16K 1037367 23 4149468K hot_inode_item > 572530 572530 100%0.55K 817907327160K radix_tree_node > 241706 241406 99%0.22K 14218 17 56872K xfs_ili > 241206 241204 99%1.06K 804023321608K xfs_inode > > The inode tracking is trying to track all 24 million inodes even > though they have been written only once, and there are only 240,000 > inodes in the cache at this point in time. That was the last update > that slabtop got, so it is indicative of the impending OOM situation > that occurred. > >> Changelog from v2: >> 1.) Converted to Radix trees, not RB-tree [Zhiyong, Dave Chinner] >> 2.) Added memory shrinker [Dave Chinner] > > I haven't looked at the shrinker, but clearly it is not working, > otherwise the above OOM situation would not be occurring. > > Cheers, > > Dave. > > (*) Tested on an empty 17TB XFS filesystem with: > > $ sudo mkfs.xfs -f -l size=131072b,sunit=8 /dev/vdc > meta-data=/dev/vdc isize=256agcount=17, agsize=268435455 > blks > = sectsz=512 attr=2, projid32bit=0 > data = bsize=4096 blocks=4563402735, imaxpct=5 > = sunit=0 swidth=0 blks > naming =version 2 bsize=4096 ascii-ci=0 > log =internal log bsize=4096 blocks=131072, version=2 > = sectsz=512 sunit=1 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > $ sudo mount -o logbsize=256k /dev/vdc /mnt/scratch > $ sudo chmod 777 /mnt/scratch > $ fs_mark -D 1 -S0 -n 10 -s 1 -L 63 -d \ > /mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d \ > /mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d \ > /mnt/scratch/6 -d /mnt/scratch/7 > . > 0 21601 16679.3 12552262 > 0 22401 15412.4 12588587 > 0 23201 16367.6 14199322 > 0 24001 15680.4 15741205 > In this test, i haven't see you enable hot_track function via mount, why did it meet OOM? > > -- > Dave Chinner > da...@fromorbit.com -- Regards, Zhi Yong Wu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > NOTE: > > The patchset is currently post out mainly to make sure > it is going in the correct direction and hope to get some > helpful comments from other guys. > For more infomation, please check hot_tracking.txt in Documentation > # mount -o hot_track /dev/vdc /mnt/scratch # umount /mnt/scratch hangs here on XFS: # echo w > /proc/sysrq-trigger [ 44.229252] SysRq : Show Blocked State [ 44.230044] taskPC stack pid father [ 44.231187] umount D 88021fd52dc0 3632 4107 4106 0x [ 44.231946] 880212a91bd8 0086 8802153da300 880212a91fd8 [ 44.231946] 880212a91fd8 880212a91fd8 880216cf6340 8802153da300 [ 44.231946] 7fff 880212a91d78 880212a91d80 [ 44.231946] Call Trace: [ 44.231946] [] schedule+0x29/0x70 [ 44.231946] [] schedule_timeout+0x159/0x220 [ 44.231946] [] ? do_raw_spin_lock+0x54/0x120 [ 44.231946] [] ? do_raw_spin_unlock+0x5d/0xb0 [ 44.231946] [] wait_for_common+0xee/0x190 [ 44.231946] [] ? try_to_wake_up+0x2f0/0x2f0 [ 44.231946] [] wait_for_completion+0x1d/0x20 [ 44.231946] [] flush_workqueue+0x14c/0x3f0 [ 44.231946] [] hot_track_exit+0x39/0x180 [ 44.231946] [] xfs_fs_put_super+0x23/0x70 [ 44.231946] [] generic_shutdown_super+0x61/0xf0 [ 44.231946] [] kill_block_super+0x30/0x80 [ 44.231946] [] deactivate_locked_super+0x45/0x70 [ 44.231946] [] deactivate_super+0x4e/0x70 [ 44.231946] [] mntput_no_expire+0x101/0x160 [ 44.231946] [] sys_umount+0x76/0x3a0 [ 44.231946] [] system_call_fastpath+0x16/0x1b because this is stuck: [ 200.064574] kworker/u:2 S 88021fc12dc0 5208 669 2 0x [ 200.064574] 88021532fc60 0046 88021532c7c0 88021532ffd8 [ 200.064574] 88021532ffd8 88021532ffd8 81fc3420 88021532c7c0 [ 200.064574] 88021532fc50 88021532fc98 8221e700 8221e700 [ 200.064574] Call Trace: [ 200.064574] [] schedule+0x29/0x70 [ 200.064574] [] schedule_timeout+0x11b/0x220 [ 200.064574] [] ? usleep_range+0x50/0x50 [ 200.064574] [] hot_temperature_update_work+0x16a/0x1c0 [ 200.064574] [] ? do_raw_spin_lock+0x54/0x120 [ 200.064574] [] ? _raw_spin_unlock_irq+0xe/0x30 [ 200.064574] [] ? finish_task_switch+0x5c/0x100 [ 200.064574] [] process_one_work+0x139/0x500 [ 200.064574] [] ? hot_range_update+0x1f0/0x1f0 [ 200.064574] [] worker_thread+0x15e/0x460 [ 200.064574] [] ? manage_workers+0x2f0/0x2f0 [ 200.064574] [] kthread+0x93/0xa0 [ 200.064574] [] kernel_thread_helper+0x4/0x10 [ 200.064574] [] ? __init_kthread_worker+0x40/0x40 [ 200.064574] [] ? gs_change+0x13/0x13 Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > NOTE: > > The patchset is currently post out mainly to make sure > it is going in the correct direction and hope to get some > helpful comments from other guys. > For more infomation, please check hot_tracking.txt in Documentation > > TODO List: 1) Fix OOM issues - the hot inode tracking caches grow very large and don't get trimmed under memory pressure. From slabtop, after creating roughly 24 million single byte files(*) on a machine with 8GB RAM: OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 23859510 23859476 99%0.12K 795317 30 3181268K hot_range_item 23859441 23859439 99%0.16K 1037367 23 4149468K hot_inode_item 572530 572530 100%0.55K 817907327160K radix_tree_node 241706 241406 99%0.22K 14218 17 56872K xfs_ili 241206 241204 99%1.06K 804023321608K xfs_inode The inode tracking is trying to track all 24 million inodes even though they have been written only once, and there are only 240,000 inodes in the cache at this point in time. That was the last update that slabtop got, so it is indicative of the impending OOM situation that occurred. > Changelog from v2: > 1.) Converted to Radix trees, not RB-tree [Zhiyong, Dave Chinner] > 2.) Added memory shrinker [Dave Chinner] I haven't looked at the shrinker, but clearly it is not working, otherwise the above OOM situation would not be occurring. Cheers, Dave. (*) Tested on an empty 17TB XFS filesystem with: $ sudo mkfs.xfs -f -l size=131072b,sunit=8 /dev/vdc meta-data=/dev/vdc isize=256agcount=17, agsize=268435455 blks = sectsz=512 attr=2, projid32bit=0 data = bsize=4096 blocks=4563402735, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=4096 blocks=131072, version=2 = sectsz=512 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 $ sudo mount -o logbsize=256k /dev/vdc /mnt/scratch $ sudo chmod 777 /mnt/scratch $ fs_mark -D 1 -S0 -n 10 -s 1 -L 63 -d \ /mnt/scratch/0 -d /mnt/scratch/1 -d /mnt/scratch/2 -d \ /mnt/scratch/3 -d /mnt/scratch/4 -d /mnt/scratch/5 -d \ /mnt/scratch/6 -d /mnt/scratch/7 . 0 21601 16679.3 12552262 0 22401 15412.4 12588587 0 23201 16367.6 14199322 0 24001 15680.4 15741205 -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Mon, Oct 15, 2012 at 8:39 AM, Zheng Liu wrote: > On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: >> From: Zhi Yong Wu >> >> NOTE: >> >> The patchset is currently post out mainly to make sure >> it is going in the correct direction and hope to get some >> helpful comments from other guys. >> For more infomation, please check hot_tracking.txt in Documentation > > Hi Zhi Yong, > > If I want to use this patch set in ext4, could I apply this patch set > directly or I need to call some functions like in btrfs. Thanks. Hi ,Zheng, It is the latter. If you have any questions, please feel free to catch me. > > Regards, > Zheng -- Regards, Zhi Yong Wu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC v3 00/13] vfs: hot data tracking
On Wed, Oct 10, 2012 at 06:07:22PM +0800, zwu.ker...@gmail.com wrote: > From: Zhi Yong Wu > > NOTE: > > The patchset is currently post out mainly to make sure > it is going in the correct direction and hope to get some > helpful comments from other guys. > For more infomation, please check hot_tracking.txt in Documentation Hi Zhi Yong, If I want to use this patch set in ext4, could I apply this patch set directly or I need to call some functions like in btrfs. Thanks. Regards, Zheng -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC v3 00/13] vfs: hot data tracking
From: Zhi Yong Wu NOTE: The patchset is currently post out mainly to make sure it is going in the correct direction and hope to get some helpful comments from other guys. For more infomation, please check hot_tracking.txt in Documentation TODO List: 1.) Need to do scalability or performance tests. 2.) Turn some Micro into be tunable TIME_TO_KICK, and HEAT_UPDATE_DELAY 3.) Rafactor hot_hash_is_aging() If you just made the timeout value a timespec and compared the _timespecs_, you would be doing a lot fewer conversions. 4.) Cleanup some unnecessary lock protect 5.) Add more comments to explain how to calc temperature How to "read" the avg read/write time (nanoseconds, microseconds, jiffies??) 6.) Make updating tempreture more parallel 7.) How to save the file tempreture among the umount to be able to preserve the file tempreture after reboot 8.) Add one new ioctl inteface to set temperature value. Ben Chociej, Matt Lupfer and Conor Scott originally wrote this code to be very btrfs-specific. I've taken their code and attempted to make it more generic and integrate it at the VFS level. Changelog from v2: 1.) Converted to Radix trees, not RB-tree [Zhiyong, Dave Chinner] 2.) Added memory shrinker [Dave Chinner] 3.) Converted to one workqueue to update map info periodically [Dave Chinner] 4.) Cleanedup a lot of other issues [Dave Chinner] v1: 1.) Reduce new files and put all in fs/hot_tracking.[ch] [Dave Chinner] 2.) The first three patches can probably just be flattened into one. [Marco Stornelli , Dave Chinner] Zhi Yong Wu (13): btrfs: add one new mount option '-o hot_track' vfs: introduce private radix tree structures vfs: Initialize and free main data structures vfs: add function for collecting raw access info vfs: add two map arrays vfs: add hooks to enable hot data tracking vfs: add function for updating map arrays vfs: add aging function for old map info vfs: add one wq to update map info periodically vfs: register one memory shrinker vfs: add 3 new ioctl interfaces vfs: add debugfs support vfs: add documentation Documentation/filesystems/00-INDEX |2 + Documentation/filesystems/hot_tracking.txt | 165 fs/Makefile|2 +- fs/btrfs/ctree.h |1 + fs/btrfs/super.c | 15 +- fs/compat_ioctl.c |9 + fs/direct-io.c |8 + fs/hot_tracking.c | 1321 fs/hot_tracking.h | 155 fs/ioctl.c | 122 +++ include/linux/fs.h |4 + include/linux/hot_tracking.h | 123 +++ mm/filemap.c |7 + mm/page-writeback.c| 13 + mm/readahead.c |7 + 15 files changed, 1952 insertions(+), 2 deletions(-) create mode 100644 Documentation/filesystems/hot_tracking.txt create mode 100644 fs/hot_tracking.c create mode 100644 fs/hot_tracking.h create mode 100644 include/linux/hot_tracking.h -- 1.7.6.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
