Hey :) Thanks for the interest in this optimisation! Unfortunately I wasn't pushy enough in my thread from May-June and it wasn't included in the Maverick LiveCD. A pending question is what to do to include the recompressed files into the archive's packages [1].
2010-10-06 16:08 GMT John McCabe-Dansted <gma...@gmail.com>: > In May, Louis Simard proposed rencoding PNG files and SVG files to > reduce their size [Quoted 1]. I note that we can save further space by: > > 1) Using advdef on the png files in addition to optipng. This is what > optimizegraphics does, and this shrinks the pngs on the Maverick RC > liveCD from about 100.1MB to 85.3MB providing a saving of 14.8MB. So it does; I didn't know about that. Reading the man file for advpng, it gave a warning that it was only supported for AdvanceMAME-generated PNG files, so I was skeptical, but it does shave off about 4% more filesize on average with 'advpng -z4'. > 2) Recompressing gz files with advdef. Using advdef, we can shrink the > gz files from 89.5MB to 84.8MB, and provides a saving of 4.7MB. That's an interesting optimisation; I didn't really know about it either. However, I did use 7zip's Deflate compressor to recompress a .zip file of OpenOffice.org's from 5.9 MB to 5.4 MB. The method was rather crude, but it did the job: mkdir extracted cd extracted unzip ../file.zip 7z a -tzip -mx=9 -mfb=258 file.repack.zip extracted/* rm -r extracted > 3) Recompressing jpeg files with jpegrescan. This only saves 0.5MB, > but implementing this would add just a couple more lines of code, and > jpegrescan does not lose any picture quality [Quoted 2]. jpegoptim indeed performs lossless optimisation of JPEG files by editing Huffman tables, and it's used as the basis of jpegrescan. However, jpegoptim doesn't make non-progressive files progressive, as I understand jpegrescan does. This may make jpegoptim's optimisations more transparent to applications that, for some reason, can't decode progressive JPEGs and thus have non-progressive JPEGs in their packages. However, most applications should be using libjpeg anyway, so perhaps this point is moot. > > Together these should shrink the liveCD by over 20MB. This is without > even considering the .xml and .svg optimizations Louis proposed. > > A further 10MB could be saved by recompressing the gz files as lzma. At what LZMA compression level? Default (7) or --best (9)? > This seems reasonable as lzma has reasonable decompression times (e.g. > 7ms to decompress a largish manpage like lsof). 7 ms? What's your CPU? :) > Since the liveCD is > compressed anyway, it seems that if a file is compressed with gzip. it > is worth compressing with lzma. The command "man" already seems to > have lzma support, but we'd want to test each application to ensure > that it functions correctly when its .gz files are replaced with lzma > files. We could also selectively recompress the gz files, as some .gz > files are actually smaller (by about 40 bytes) than the corresponding > lzma file. I hadn't considered this type of "transcoding" for the LiveCD. We may want to ourselves test which programs accept .lzma files in their directories in addition to .gz. Shall you do it, shall I, or shall we both do it? Also, is anyone else interested? Your point about files being compressed anyway is kind of interesting: both Deflate and LZMA recompress very poorly, so saving bytes by switching from one to the other makes sense. That would not shrink the *installed size* of these man pages much, though, because of default 4 KB blocks for ext[2-4]. > > Given that recoding SVG files can save 7MB [Quoted 1], simply recoding files > could free up 37MB for the Natty LiveCD (and presumably also reduce > the the average size of debs in the repos by about 5%). > > [Quoted 1] > http://www.mail-archive.com/ubuntu-devel-discuss@lists.ubuntu.com/msg11337.html > [Quoted 2] http://news.ycombinator.com/item?id=803839 > > I attach the script I used to check how much space would be saved. > This is purely for reproduction of my results, it is not integrated > into Louis's script. Do you want me to add to my script any of the optimisations discussed in your email? They are: Using AdvanceCOMP to recompress .png images and gzipped files; using either of jpegoptim or jpegrescan to losslessly recompress .jpg images; "transcoding" man pages from .gz to .lzma. I'm not going to add untested optimisations yet, such as transcoding *all* .gz files to .lzma. I'm still very interested in this, despite the lack of posting about the subject in the last 4 months! I've just been waiting for the guys at Debian to advise me on how to best integrate these optimisations into packages. Perhaps I should just devise a set of suitable build-depends additions (optipng, advancecomp, jpegoptim) and makefile rules for .png/.jpg/.gz, then file a single bug report on all of the packages that would benefit the most from optimisations? That way, package maintainers could opt in rather easily. - Louis [1] https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2010-June/011579.html -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
