On Thu, May 18, 2006 at 11:05:38PM +0200, Florent Daigni?re (NextGen$) wrote: > * Ian Clarke <ian at locut.us> [2006-05-18 13:11:34]: > > > High level question: Why is it necessary to support several > > compression standards? > > I'm not sure either that it is a good idea.
It is vital to support multiple compression standards for *decompression*. One more reason: In future we may add a new compression algorithm! Whether it is vital to try every possible algorithm for compression is an open question. > > NextGen$ > > > Wouldn't it be simpler just to support one > > (ie. the one that achieves the best compression, without suffering > > from legal issues)? > > > > Ian. > > > > On 18 May 2006, at 11:25, Matthew Toseland wrote: > > > > > http://www.7-zip.org/sdk.html - there is a java version. > > > > > > We should provide this as a compression codec. > > > > > > Thus, we can provide gzip, bzip2 and 7zip compressors, plus zip and > > > tar > > > archivers. > > > > > > I am currently testing some code which will allow larger freesites > > > to be > > > inserted, the next job after that is ZIP manifest support, which > > > judging > > > from the test site should make a substantial difference. > > > > > > I am currently inserting www.hardwarebook.net, just the HTML. On disk > > > this is 3.2MB. Inserting it at present uses 489 blocks = 16MB. As a > > > zip > > > file it would be 24 blocks plus 12 check blocks plus 1 redirect. As a > > > tar.gz it would be 9 blocks plus 5 check blocks plus 1 redirect. As a > > > tar.bz2 or a tar.7z it would be 6 blocks plus 3 check blocks plus 1 > > > redirect. > > > > > > Raw 489 > > > tar 133 > > > zip 37 > > > tar.gz 15 > > > tbz/t7z 10 > > > > > > Admittedly this wouldn't be quite so extreme if we included the > > > images. > > > > > > Obviously the big gain is from raw to zip. This is already implemented > > > for fetching but not yet for inserting. There were some legal issues > > > with the tar/bz2 libraries but these have been sorted out thanks to > > > GPL3's explicit compatibility with the ASL2. > > > > > > We probably don't want the whole site in one big archive. One > > > possibility would be to split it by content type: one container > > > includes > > > all the HTML, one includes all the GIFs etc, and files over a certain > > > size are inserted separately. > > > > > > One other minor point of note: We need to provide a means for > > > clients to > > > be able to tell us not only whether to not try to compress at all, but > > > also whether to try the really heavy compressors. > > > -- > > > Matthew J Toseland - toad at amphibian.dyndns.org > > > Freenet Project Official Codemonkey - http://freenetproject.org/ > > > ICTHUS - Nothing is impossible. Our Boss says so. > > > _______________________________________________ > > > Devl mailing list > > > Devl at freenetproject.org > > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -- Matthew J Toseland - toad at amphibian.dyndns.org Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20060519/e1c07847/attachment.pgp>
