Re: [RFC] Online dedup for Btrfs

2013-04-18 Thread Martin
Apart from the dates, this sounds highly plausible :-) If the hashing is done before the compression and the compression is done for isolated blocks, then this could even work! Any takers? ;-) For a performance enhancement, keep a hash tree in memory for the "n" most recently used/seen blocks?.

Re: [RFC] Online dedup for Btrfs

2013-04-01 Thread Konstantinos Skarlatos
On 1/4/2013 6:38 μμ, Josef Bacik wrote: On Mon, Apr 01, 2013 at 08:50:34AM -0400, Josef Bacik wrote: Hello, I was bored this weekend so I hacked up online dedup for Btrfs. It's working quite well so I think it can be more widely tested. There are two ways to use it 1) Compatible mode - this

Re: [RFC] Online dedup for Btrfs

2013-04-01 Thread Harald Glatt
Oh man :D It was so elaborate that I really believed it :P On Mon, Apr 1, 2013 at 5:38 PM, Josef Bacik wrote: > On Mon, Apr 01, 2013 at 08:50:34AM -0400, Josef Bacik wrote: >> Hello, >> >> I was bored this weekend so I hacked up online dedup for Btrfs. It's working >> quite well so I think i

Re: [RFC] Online dedup for Btrfs

2013-04-01 Thread Josef Bacik
On Mon, Apr 01, 2013 at 08:50:34AM -0400, Josef Bacik wrote: > Hello, > > I was bored this weekend so I hacked up online dedup for Btrfs. It's working > quite well so I think it can be more widely tested. There are two ways to use > it > > 1) Compatible mode - this is a bit slower but will hand

Re: [RFC] Online dedup for Btrfs

2013-04-01 Thread Harald Glatt
On Mon, Apr 1, 2013 at 2:50 PM, Josef Bacik wrote: > Hello, > > I was bored this weekend so I hacked up online dedup for Btrfs. It's working > quite well so I think it can be more widely tested. There are two ways to use > it > > 1) Compatible mode - this is a bit slower but will handle being us

[RFC] Online dedup for Btrfs

2013-04-01 Thread Josef Bacik
Hello, I was bored this weekend so I hacked up online dedup for Btrfs. It's working quite well so I think it can be more widely tested. There are two ways to use it 1) Compatible mode - this is a bit slower but will handle being used by older kernels. We use the csum tree to find duplicate blo