On Thu, 2013-07-25 at 11:16 -0700, Taras Glek wrote: > > > Vyacheslav Dubeyko wrote: > > On Jul 25, 2013, at 8:42 PM, Taras Glek wrote: > > > > [snip] > > > > To introduce transparent decompression. Let someone else do the > > > > compression for us, and supply decompressed data on demand (in this > > > > case a read call). Reduces the complexity which would otherwise have to > > > > be brought into the filesystem. > > > The main use for file compression for Firefox(it's useful on Linux > > > desktop too) is to improve IO-throughput and reduce startup latency. In > > > order for compression to be a net win an application should be aware of > > > what is being compressed and what isn't. For example patterns for IO on > > > large libraries (eg 30mb libxul.so) are well suited to compression, but > > > SQLite databases are not. Similarly for our disk cache: images should > > > not be compressed, but javascript should be. Footprint wins are useful on > > > android, but it's the increased IO throughput on crappy storage devices > > > that makes this most attractive. > > > > > > In addition of being aware of which files should be compressed, Firefox > > > is aware of patterns of usage of various files it could schedule > > > compression at the most optimal time. > > > > > > Above needs tie in nicely with the simplification of not implementing > > > compression at fs-level. > > > > There are many filesystems that uses compression as internal technique. > > And, as I understand, implementation > > of compression in different Linux kernel filesystem drivers has similar > > code patterns. So, from my point of view, > > it makes sense to generalize compression/decompression code in the form of > > library. The API of such generalized > > compression kernel library can be used in drivers of different file > > systems. Also such generalized compression > > library will simplify support of compression in file system drivers that > > don't support compression feature currently. > > > > Moreover, I think that it is possible to implement compression support on > > VFS level. Such feature gives > > opportunity to have compression support for filesystems that don't support > > compression feature as > > internal technique. > >From my conversations with Ted, it sounded like it was easier to > accomplish changes like this at fs level than vfs. Maybe things are > different now
If I understand the idea correctly, you want to have file-oriented compression. I mean that you want to have one type of files are compressed but another type of ones should be not compressed. So, from my understanding, a file is not ext4 specific concept. Every filesystem contains files. So, if you intend to implement compression support in ext4 then you need to use e2compr as basis for full-featured compression in ext4. But if you want to have something like file-oriented compression then it is not feature of concrete filesystem. Do you mean that Firefox will work only on ext4? So, it means for me that feature likewise file-oriented compression should be on VFS level or over VFS. And I am not fully confident that it should be a feature in kernel-space. > > > > [snip] > > > This transparent decompression idea is based on our experience with HFS+. > > > Apple uses the fs-attribute approach. OSX is able to compress application > > > libraries at installation-time, apps remain blissfully unaware but get an > > > extra boost in startup perf. > > > > > > > HFS+ supports compression as internal filesystem technique. It means that > > HFS+ volume layout has > > metadata structures for compression support (compressed xattrs or > > compressed resource forks). > > So, compression is supported on FS level. As I know, Mac OS X has native > > decompression support > > for compressed files but you need to use special tool for compression of > > files on HFS+. Maybe > > Mac OS X has internal library that give opportunity to compress application > > libraries at installation > > time. But I suppose that it is simply user-space tool or library that uses > > HFS+ compression support > > on kernel-space and volume layout levels. > OSX compression is done in userspace, all heuristics are done in > userspace. https://github.com/jrk/afsctool/blob/master/afsctool.c#L87 > Rest of the details are there too. Anyway, HFS+ keeps compressed files' content in com.apple.decmpfs xattr or in resource fork. It means for me that HFS+ volume layout has special metadata structures for compression support. Because if you will try to work with HFS+ compressed file under Linux then you will see file with zero byte in size. And, moreover, Mac OS X filesystem driver should know about these metadata structures. Otherwise, it cannot process HFS+ compressed files. We can discuss about HFS+ compression in more details but I think that it is not key topic for this patch set. With the best regards, Vyacheslav Dubeyko. > Taras > > > > With the best regards, > > Vyacheslav Dubeyko. > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
