Hello!
Sorry for the delay.
On 2020-05-23 K L wrote:
> Could you give us the clue what can be the reason of such buffering
> error and how to overcome it (can it be done in lzma or in libarchive
> code?). Our test data is valid archive (at least several tools can
> read it successfully).
LZMA_BUF_ERROR means that either the input file is truncated/corrupt or
not enough output buffer space is available. The error isn't returned
immediately (allow_buf_error) because it makes one corner case simpler
in some applications, so that's why it is returned only on the third
call.
Perhaps the LZMA data in the .zip file doesn't contain end of stream
(EOS) marker (also known as end-of-payload marker (EOPM)). If the
decoder expect EOPM to be present but the input doesn't have it, the
decoder will think the file is truncated and thus LZMA_BUF_ERROR. The
.zip headers have a bit to indicate the presence of LZMA EOS marker
(sections 4.4.4 and 5.8.9 in APPNOTE.TXT version 6.3.7).
If you decompress with liblzma's lzma_raw_decoder(), then it will
always expect EOPM to be present. This is an API limitation that
doesn't exist in LZMA SDK. This should be improved in liblzma but the
current situation is this.
Internally liblzma can handle LZMA data without EOPM. It is used for
.lzma files and in LZMA2. With the current API, one can write a simple
hack to decode raw LZMA data without EOPM: use lzma_alone_decoder() and
prefix the raw LZMA data with a fake 13-byte .lzma header:
1 byte LZMA properties
4 bytes little endian dictionary size
8 bytes little endian uncompressed size (UINT64_MAX means
unknown and then (and only then) EOPM must be present)
The first five bytes are the same as the five LZMA properties bytes in
the .zip header. So initialize lzma_alone_decoder(), pass the five
bytes, then pass either the 64-bit uncompressed size (if EOS bit in .zip
headers is unset) or eight 0xFF bytes (if EOS bit in .zip headers is
set), and finally pass the actual LZMA data.
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
