* Ian Clarke <ian at locut.us> [2006-05-18 13:11:34]: > High level question: Why is it necessary to support several > compression standards?
I'm not sure either that it is a good idea. NextGen$ > Wouldn't it be simpler just to support one > (ie. the one that achieves the best compression, without suffering > from legal issues)? > > Ian. > > On 18 May 2006, at 11:25, Matthew Toseland wrote: > > > http://www.7-zip.org/sdk.html - there is a java version. > > > > We should provide this as a compression codec. > > > > Thus, we can provide gzip, bzip2 and 7zip compressors, plus zip and > > tar > > archivers. > > > > I am currently testing some code which will allow larger freesites > > to be > > inserted, the next job after that is ZIP manifest support, which > > judging > > from the test site should make a substantial difference. > > > > I am currently inserting www.hardwarebook.net, just the HTML. On disk > > this is 3.2MB. Inserting it at present uses 489 blocks = 16MB. As a > > zip > > file it would be 24 blocks plus 12 check blocks plus 1 redirect. As a > > tar.gz it would be 9 blocks plus 5 check blocks plus 1 redirect. As a > > tar.bz2 or a tar.7z it would be 6 blocks plus 3 check blocks plus 1 > > redirect. > > > > Raw 489 > > tar 133 > > zip 37 > > tar.gz 15 > > tbz/t7z 10 > > > > Admittedly this wouldn't be quite so extreme if we included the > > images. > > > > Obviously the big gain is from raw to zip. This is already implemented > > for fetching but not yet for inserting. There were some legal issues > > with the tar/bz2 libraries but these have been sorted out thanks to > > GPL3's explicit compatibility with the ASL2. > > > > We probably don't want the whole site in one big archive. One > > possibility would be to split it by content type: one container > > includes > > all the HTML, one includes all the GIFs etc, and files over a certain > > size are inserted separately. > > > > One other minor point of note: We need to provide a means for > > clients to > > be able to tell us not only whether to not try to compress at all, but > > also whether to try the really heavy compressors. > > -- > > Matthew J Toseland - toad at amphibian.dyndns.org > > Freenet Project Official Codemonkey - http://freenetproject.org/ > > ICTHUS - Nothing is impossible. Our Boss says so. > > _______________________________________________ > > Devl mailing list > > Devl at freenetproject.org > > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > _______________________________________________ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20060518/1829a60d/attachment.pgp>
