Re: [RFC PATCH v5 0/5] Online data deduplication
On Wed, Jul 31, 2013 at 05:20:27PM -0400, Josef Bacik wrote: On Wed, Jul 31, 2013 at 11:37:40PM +0800, Liu Bo wrote: Data deduplication is a specialized data compression technique for eliminating duplicate copies of repeating data.[1] This patch set is also related to Content based storage in project ideas[2]. PATCH 1 is a hang fix with deduplication on, but it's also useful without dedup in practice use. PATCH 2 and 3 are targetting delayed refs' scalability problems, which are uncovered by the dedup feature. PATCH 4 is a speed-up improvement, which is about dedup and quota. PATCH 5 is full of real things, all details about implementation of dedup. Plus, there is also a btrfs-progs patch which helps to enable/disable dedup feature. TODO: * a bit-to-bit comparison callback. Didn't pass my BUG_ON() search test, try again. I'm cleaning them up :) -liubo -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH v5 0/5] Online data deduplication
Data deduplication is a specialized data compression technique for eliminating duplicate copies of repeating data.[1] This patch set is also related to Content based storage in project ideas[2]. PATCH 1 is a hang fix with deduplication on, but it's also useful without dedup in practice use. PATCH 2 and 3 are targetting delayed refs' scalability problems, which are uncovered by the dedup feature. PATCH 4 is a speed-up improvement, which is about dedup and quota. PATCH 5 is full of real things, all details about implementation of dedup. Plus, there is also a btrfs-progs patch which helps to enable/disable dedup feature. TODO: * a bit-to-bit comparison callback. All comments are welcome! [1]: http://en.wikipedia.org/wiki/Data_deduplication [2]: https://btrfs.wiki.kernel.org/index.php/Project_ideas#Content_based_storage v4-v5: - go back to one dedup key with a special backref for dedup tree because the disk format understands backref well. - fix a fsync hang with dedup enabled. - rebase onto the latest btrfs. Liu Bo (5): Btrfs: skip merge part for delayed data refs Btrfs: improve the delayed refs process in rm case Btrfs: introduce a head ref rbtree Btrfs: disable qgroups accounting when quata_enable is 0 Btrfs: online data deduplication fs/btrfs/backref.c |9 + fs/btrfs/ctree.h | 59 fs/btrfs/delayed-ref.c | 141 +++ fs/btrfs/delayed-ref.h |8 + fs/btrfs/disk-io.c | 30 ++ fs/btrfs/extent-tree.c | 196 -- fs/btrfs/extent_io.c | 29 ++- fs/btrfs/extent_io.h | 16 ++ fs/btrfs/file-item.c | 217 +++ fs/btrfs/inode.c | 637 ++-- fs/btrfs/ioctl.c | 93 +++ fs/btrfs/ordered-data.c| 36 ++- fs/btrfs/ordered-data.h| 11 +- fs/btrfs/qgroup.c |6 + fs/btrfs/relocation.c |3 + fs/btrfs/super.c | 27 ++- fs/btrfs/transaction.c |4 +- include/uapi/linux/btrfs.h |5 + 18 files changed, 1356 insertions(+), 171 deletions(-) -- 1.7.7 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH v5 0/5] Online data deduplication
On Wed, Jul 31, 2013 at 11:37:40PM +0800, Liu Bo wrote: Data deduplication is a specialized data compression technique for eliminating duplicate copies of repeating data.[1] This patch set is also related to Content based storage in project ideas[2]. PATCH 1 is a hang fix with deduplication on, but it's also useful without dedup in practice use. PATCH 2 and 3 are targetting delayed refs' scalability problems, which are uncovered by the dedup feature. PATCH 4 is a speed-up improvement, which is about dedup and quota. PATCH 5 is full of real things, all details about implementation of dedup. Plus, there is also a btrfs-progs patch which helps to enable/disable dedup feature. TODO: * a bit-to-bit comparison callback. Didn't pass my BUG_ON() search test, try again. Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
