Bug#409848: ccache: zlib compression support
On Tue, Feb 06, 2007 at 12:22:42AM +0200, Sami Liedes wrote: Compressed cache support would be nice, as the caches tend to grow quite large. After compressing the cache (with the script) my cache dir shrunk from 400 MiB to 94 MiB - more stuff fits in the cache. Seems to work, after some testing. Hi Sami, Sounds like an interesting idea. The only concern I have is how much overhead does that add to the compilation time? Since the whole point of using ccache is to speed up compilation, we have to be careful about adding extra overhead. Does the patch enable compression by default or does it simply add a flag (or environment variable) to allow users to enable it? Francois -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#409848: ccache: zlib compression support
On Tue, Feb 06, 2007 at 04:50:06PM -0500, Francois Marier wrote: Sounds like an interesting idea. The only concern I have is how much overhead does that add to the compilation time? Since the whole point of using ccache is to speed up compilation, we have to be careful about adding extra overhead. gzip, which it uses, is pretty fast nowadays, at least compared to compiling large functions/files. Years ago I benchmarked gcc and figured out compilation time of a function is roughly O(n^2 log n), where n is the number of local variables. I assume it has become slower with the addition of new optimizations and especially intraprocedural optimization. In comparison, gzipping is a linear time (O(n)) process. There are some benchmarks in http://www.gustaebel.de/lars/ccache/ : plain gcc run 729.55 sec first normal ccache run 747.12 sec second normal ccache run 92.11 sec cache size9235 k first patched ccache run751.92 sec second patched ccache run92.68 sec cache size5491 k So the gzipping has a very slight performance hit. However I would assume this is generally more than offset by more data fitting in the cache, hence increasing the frequency of cache hits. Here the reduction in cache size doesn't seem to be so dramatic as in my benchmarks; I assume this is caused by me building with debug symbols, which apparently compress well (and also has a habit of filling up the cache very fast, so compression would probably be more important). Does the patch enable compression by default or does it simply add a flag (or environment variable) to allow users to enable it? There's an environment variable, CCACHE_NOCOMPRESS, to turn the compression off. Of course if desired, compression could be changed to be off by default and enabled by (say) CCACHE_COMPRESS. There's also a downside of CCACHE_HARDLINK not working for compressed cache files. Sami signature.asc Description: Digital signature
Bug#409848: ccache: zlib compression support
On Wed, Feb 07, 2007 at 12:55:06AM +0200, Sami Liedes wrote: There are some benchmarks in http://www.gustaebel.de/lars/ccache/ Sorry, I should have looked at that. It looks like the impact of compression is negligible. There's an environment variable, CCACHE_NOCOMPRESS, to turn the compression off. Of course if desired, compression could be changed to be off by default and enabled by (say) CCACHE_COMPRESS. There's also a downside of CCACHE_HARDLINK not working for compressed cache files. I'm tending towards applying this patch but disabling compression by default. What do you think? Francois -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#409848: ccache: zlib compression support
On Tue, Feb 06, 2007 at 06:14:25PM -0500, Francois Marier wrote: I'm tending towards applying this patch but disabling compression by default. What do you think? Sounds reasonable to me, if you don't feel comfortable enabling it by default. Sami signature.asc Description: Digital signature
Bug#409848: ccache: zlib compression support
Package: ccache Version: 2.4-7 Severity: wishlist Tags: patch Compressed cache support would be nice, as the caches tend to grow quite large. There is a patch available at http://www.gustaebel.de/lars/ccache/ . It applied cleanly to the current debian package and produced a zlib-enabled ccache; in addition to this, it provides one script that needs to be installed. After compressing the cache (with the script) my cache dir shrunk from 400 MiB to 94 MiB - more stuff fits in the cache. Seems to work, after some testing. Sami -- System Information: Debian Release: 4.0 APT prefers unstable APT policy: (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Shell: /bin/sh linked to /bin/bash Kernel: Linux 2.6.18-3-amd64 Locale: LANG=C, [EMAIL PROTECTED] (charmap=ISO-8859-15) Versions of packages ccache depends on: ii libc6 2.3.6.ds1-11 GNU C Library: Shared libraries ccache recommends no packages. -- no debconf information signature.asc Description: Digital signature
