Re: Using zlib without compressing data
joe bloggs [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Does anyone know why this extra data is added to every chunk passed to the deflate function? Probably to tell the decompression engine that the data is not compressed... Alan -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
RE: Using zlib without compressing data
From: joe bloggs [mailto:[EMAIL PROTECTED] If they decide not to compress the files, the app sets the deflate compression level to 0 (indicating that no compression is to take place). This works as expected except that the deflate function seems to add a few bytes [...] Does anyone know why this extra data is added to every chunk passed to the deflate function? That's just how the deflate format works, as defined in RFC 1951. Do your (optionally) compressed files consist only of the raw zlib stream? If so, and if you want the option to store a raw uncompressed byte stream as well (to avoid this inflation issue) then you'll need at least a small header on the front to indicate which format it is. If so, then consider using the full gzip format (RFC 1952). The gzip header has fields for the original file name and timestamp and whatnot, which may be useful as you are compressing files. It also has a field (CM) to specify compression method. Oddly the RFC defines only one allowable compression method (deflate = 8) and doesn't define one for the identity algorithm of raw bytes. But you could cheat a little and define your own compression method code to indicate raw bytes. -slj- -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
RE: Using zlib without compressing data
I see... and thank you for the knowledge. Just to clarify, I am using the full gzip format when file compression takes place (when compression level 0). When the data is not compressed I do not include the header or trailing data. Are saying that even if the data is not compressed, I still need to include the header and trailing data if I am going to pass the data to the deflate function? Are you also saying that if I do not want the extra bytes of info added to the file if it is not being compressed, I could alter the deflate function so when a compression method other than 8 is specified, no compression will take place and no extra data will be added? But you could cheat a little and define your own compression method code to indicate raw bytes. Cheers, Joe --- Scott Johnson [EMAIL PROTECTED] wrote: From: joe bloggs [mailto:[EMAIL PROTECTED] If they decide not to compress the files, the app sets the deflate compression level to 0 (indicating that no compression is to take place). This works as expected except that the deflate function seems to add a few bytes [...] Does anyone know why this extra data is added to every chunk passed to the deflate function? That's just how the deflate format works, as defined in RFC 1951. Do your (optionally) compressed files consist only of the raw zlib stream? If so, and if you want the option to store a raw uncompressed byte stream as well (to avoid this inflation issue) then you'll need at least a small header on the front to indicate which format it is. If so, then consider using the full gzip format (RFC 1952). The gzip header has fields for the original file name and timestamp and whatnot, which may be useful as you are compressing files. It also has a field (CM) to specify compression method. Oddly the RFC defines only one allowable compression method (deflate = 8) and doesn't define one for the identity algorithm of raw bytes. But you could cheat a little and define your own compression method code to indicate raw bytes. -slj- -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/ __ Do you Yahoo!? Yahoo! Finance Tax Center - File online. File on time. http://taxes.yahoo.com/filing.html -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
RE: Using zlib without compressing data
From: joe bloggs [mailto:[EMAIL PROTECTED] Just to clarify, I am using the full gzip format when file compression takes place (when compression level 0). When the data is not compressed I do not include the header or trailing data. OK, it sounds like your intent is to produce either compressed gzip files, or plain uncompressed files. If this is correct, then for the latter case can't you just store the original bytes and call it an uncompressed file? Then the issue becomes simply how to discover which format is used for a given file. You could use the .gz extension on gzipped files, and no such extension on a regular file name. This would be nicely compatible with the desktop gzip utility, since you did say this is for expansion cards, which might get mounted on a desktop. Then the likely formats of files named (say) Readme.txt and Bible.txt.gz would be apparent to someone browsing the card. Or if you don't want to use a filename extension, you could look for the gzip magic header value (see the RFC) at the front of the file content, which would be safe as long as an uncompressed file can't possibly start with those specific bytes. Are saying that even if the data is not compressed, I still need to include the header and trailing data if I am going to pass the data to the deflate function? A zlib stream with no compression is definitely not simply a stream consisting of the uncompressed bytes. Just treat any zlib stream as a black box. -slj- -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
RE: Using zlib without compressing data
From what you are saying it sounds like the easiest way for me to achieve my goal of allowing the user to choose between uncompressed and compresesd is to copy files directly (skipping deflate function) if no compression is required. That is simple enough and makes perfect sense. Many thanks, Joe --- Scott Johnson [EMAIL PROTECTED] wrote: From: joe bloggs [mailto:[EMAIL PROTECTED] Just to clarify, I am using the full gzip format when file compression takes place (when compression level 0). When the data is not compressed I do not include the header or trailing data. OK, it sounds like your intent is to produce either compressed gzip files, or plain uncompressed files. If this is correct, then for the latter case can't you just store the original bytes and call it an uncompressed file? Then the issue becomes simply how to discover which format is used for a given file. You could use the .gz extension on gzipped files, and no such extension on a regular file name. This would be nicely compatible with the desktop gzip utility, since you did say this is for expansion cards, which might get mounted on a desktop. Then the likely formats of files named (say) Readme.txt and Bible.txt.gz would be apparent to someone browsing the card. Or if you don't want to use a filename extension, you could look for the gzip magic header value (see the RFC) at the front of the file content, which would be safe as long as an uncompressed file can't possibly start with those specific bytes. Are saying that even if the data is not compressed, I still need to include the header and trailing data if I am going to pass the data to the deflate function? A zlib stream with no compression is definitely not simply a stream consisting of the uncompressed bytes. Just treat any zlib stream as a black box. -slj- -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/ __ Do you Yahoo!? Yahoo! Finance Tax Center - File online. File on time. http://taxes.yahoo.com/filing.html -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
