On Mon, Aug 04, 2003 at 03:35:49PM +0100, Gordan wrote: > On Monday 04 Aug 2003 15:03, [EMAIL PROTECTED] wrote: > > On Mon, 04 Aug 2003 06:05:45 -0700 Gordan <[EMAIL PROTECTED]> wrote: > > >Are there any objections to the above spec, other than objections > > >to using any non-manual compression? > > > > I've been following this thread with increasing disgust so that the time > > has come to express my thoughts on this issue. ;) > > An unbiased view, then. :-) > > > I think there are to many forward, backward and inter-node compatibility > > issues with general single file compression. And I doubt it will be > > generally useful. > > I do recognize the fact that there would be inter-operability issues for some > time. I do not, however, recognize that these issues would linger on for very > long. Upgrading is easy, and the development cycle is currently short enough > that all the old nodes would probably be replaced within a matter of weeks. > > > So, compression would be beneficial in only two cases: > > A) compress compressible data, mostly text/plain and text/html > > B) compress compressed data slightly to fit in a smaller log2size key > > I don't really consider those to be a separate case. If the compression > doesn't yield a size reduction required to go to a smaller key size, then we > throw away the file we were trying to compress and just insert it > uncompressed. Compressing data that is already compressed shouldn't really be > an option. > > > I only now of three applications, which would benefit from case A: > > 1. Frost uses plain text messages and control-messages. So Frost should > > improve its protocol and compress the messages on its own! Because of > > the restricted domain the compression algorithm could be highly optimized. > > 2. Freenet Message Board (FMB) uses plain text messages and control- > > messages. So FMB should improve its protocol and ... see point 1. > > Ideally there should be a layer between the node and the client that provides > a uniform access to insertion and retrieval of files. It would be this layer > that should, IMHO, do the compression... That way a lot of compatibility > issues are removed, and the node would be left to deal with files and > routing, without having to worry about the content or file types. > > In the meantime, fproxy provides an interface for inserting and retrieving > files using http, and most functions can be performed this way, using > existing tools and libraries. This is why it was suggested that this should > be implemented in fproxy. > > > 3. Freesites use plain text html files. I'm not much into this but in > > the good ol' web my browser sends an "Accept-Encoding: gzip, deflate" > > to the server and I know of at least one server sending me gzipped HTMLs. > > Can this be extended to Freenet? E.g. by including in the content's > > metadata a "Info.Encoding=gzip" line? > > I thought about this. The only difference there is whether you are going to > leave the decompression to the end-client or the proxy. For uniformity, it > should probably be the proxy. Additionally, seen as those of us supporting > the idea of compression have kind of almost agreed on zip format, this would > not work because zip compression is not compatible with gzip. > > > Case B (compress compressed data slightly) should happen very rarely. > > It could be that an uncompressed 520KiB PDF occupying an 1 MiB key could > > fit --when compressed-- into a 512 KiB key. But I think it's up to the > > kindness of the content provider to compress the data where it's gonna > > be useful. > > I don't believe in compressing compressed files, and I don't think this should > be left entirely to the content provider. > > > To summarize: > > - I see no need for an automatic compression done by Fproxy. And neither > > by the node! ... good grief! > > I agree that the node is the wrong place to apply this. Node should handle the > routing and the keys. Whether the file is compressed or not should be > irrelevant at that logical level. Compression should be performed on a higher > level, e.g. fproxy. > > > - Frost and FMB should compress their messages > > It depends on how big/compressible their messages are. I am not too sure that > most messages would give worth-while compression, because they are so small > (typically in the 4KB-8KB range, IIRC).
Actually, text of such size is fairly compressible, try it. However, the minimum key size is only 1kB. > > > - Freesites could be gzipped, if this is a web standard and if it can > > be done without reinventing the wheel. > > This would actually be a worse solution because fproxy would still have to > understand how to decompress data - not all browsers support all > decompression algorithms. It would create more exception handling, which is > not desirable. Additionally, it would mean including the gzip library into > the freenet distribution, when zip is already built in. Java includes gzip. ZIP/JAR uses the same algorithm. Tar or bzip2 would be a more serious problem. > > Gordan -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so.
pgp00000.pgp
Description: PGP signature
