Kai Krakow <hurikhan77+bt...@gmail.com> schrieb: > Gabriel de Perthuis <g2p.c...@gmail.com> schrieb: > >> It sounds simple, and was sort-of prompted by the new syscall taking >> short ranges, but it is tricky figuring out a sane heuristic (when to >> hash, when to bail, when to submit without comparing, what should be the >> source in the last case), and it's not something I have an immediate >> need for. It is also possible to use 9p (with standard cow and/or >> small-file dedup) and trade a bit of configuration for much more >> space-efficient VMs. >> >> Finer-grained tracking of which ranges have changed, and maybe some >> caching of range hashes, would be a good first step before doing any >> crazy large-file heuristics. The hash caching would actually benefit >> all use cases. > > Looking back to good old peer-2-peer days (I think we all got in touch > with that the one or the other way), one title pops back into my mind: > tiger- tree-hash... > > I'm not really into it, but would it be possible to use tiger-tree-hashes > to find identical blocks? Even accross different sized files...
While thinking about it: That hash was probably invented for the purpose of distributing the same content to multiple peers in as small deltas as possible. Well, deduplication is somehow the other way around: Coalescing all those wild distribution back into a single source of content. So some "inverse" of tiger-tree would probably work better / more efficient. Regards, Kai -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
