[jira] [Commented] (COMPRESS-549) Inconsistency with latest PKZip standard
[
https://issues.apache.org/jira/browse/COMPRESS-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17296640#comment-17296640
]
Stefan Bodewig commented on COMPRESS-549:
-
If you encounter any file that combines a data descriptor with the stored
method, then your only safe way is to use {{ZipFile}} - either the java.util
variety or ours. Anything using a non-seekable stream can run into situations
where reading fails.
> Inconsistency with latest PKZip standard
>
>
> Key: COMPRESS-549
> URL: https://issues.apache.org/jira/browse/COMPRESS-549
> Project: Commons Compress
> Issue Type: Bug
>Affects Versions: 1.20
>Reporter: Sebastian Kürten
>Priority: Major
>
> I came across some Zip archives that cannot be read using
> ZipArchiveInputStream. Investigating the issue, I found that
> java.util.zip.ZipInputStream from the JDK shows the same behavior while
> java.util.zip.ZipFile seems to do fine. For java.util.zip.ZipInputStream, the
> issue has been reported here:
> [https://bugs.openjdk.java.net/browse/JDK-8143613] and an example file is
> provided. I copied the testing file into a repository for testing. Here's the
> example file:
> [https://github.com/sebkur/test-zip-impls/blob/master/src/test/java/de/topobyte/zip/tests/jdk8143613/TestData.java]
> and the test that fails reading that data using ZipArchiveInputStream:
> [https://github.com/sebkur/test-zip-impls/blob/master/src/test/java/de/topobyte/zip/tests/jdk8143613/TestCommonsZipInputStream.java]
>
>
> If this file is indeed a ZIP archive consistent with the PKZip spec, I think
> commons-compress does not work according to the spec. I would appreciate if
> someone could look into this and verify if this is indeed a bug in
> commons-compress. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-549) Inconsistency with latest PKZip standard
[ https://issues.apache.org/jira/browse/COMPRESS-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178892#comment-17178892 ] Sebastian Kürten commented on COMPRESS-549: --- [~peterlee], thank you for taking a look at this. I have to say I'm not the author of that test case. I originally had issues with real-world files (Encrypted OpenDocument files, which are zip files) and using the ZipInputStram from the java.util.zip package. As I found the bug report to OpenJDK mentioned above with a short test-case, I thought I had found a minimalist instance of the bug I encountered. I tried to run the test case using Commons Compress and conjured it had a bug, too as it failed. I'll report back to the OpenJDK bug that the test case is buggy. By the way, the real-world OpenDocument files work just fine using Commons Compress (it turned out I needed to pass allowStoredEntriesWithDataDescriptor=true to the ZipArchiveInputStream to avoid problems). As far as I get it now that is the bit that the java.util.zip.ZipInputStream implementation does not handle properly. > Inconsistency with latest PKZip standard > > > Key: COMPRESS-549 > URL: https://issues.apache.org/jira/browse/COMPRESS-549 > Project: Commons Compress > Issue Type: Bug >Affects Versions: 1.20 >Reporter: Sebastian Kürten >Priority: Major > > I came across some Zip archives that cannot be read using > ZipArchiveInputStream. Investigating the issue, I found that > java.util.zip.ZipInputStream from the JDK shows the same behavior while > java.util.zip.ZipFile seems to do fine. For java.util.zip.ZipInputStream, the > issue has been reported here: > [https://bugs.openjdk.java.net/browse/JDK-8143613] and an example file is > provided. I copied the testing file into a repository for testing. Here's the > example file: > [https://github.com/sebkur/test-zip-impls/blob/master/src/test/java/de/topobyte/zip/tests/jdk8143613/TestData.java] > and the test that fails reading that data using ZipArchiveInputStream: > [https://github.com/sebkur/test-zip-impls/blob/master/src/test/java/de/topobyte/zip/tests/jdk8143613/TestCommonsZipInputStream.java] > > > If this file is indeed a ZIP archive consistent with the PKZip spec, I think > commons-compress does not work according to the spec. I would appreciate if > someone could look into this and verify if this is indeed a bug in > commons-compress. Thanks! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (COMPRESS-549) Inconsistency with latest PKZip standard
[ https://issues.apache.org/jira/browse/COMPRESS-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178866#comment-17178866 ] Peter Lee commented on COMPRESS-549: I also found somp minor problem when debugging this test case - this problem has been in Commons Compress ever since 2010(according to the git log). [~bodewig] Considering this code was written in 10 years ago and have not been found, please have a look in my commit 86bb35a if you have time. :) > Inconsistency with latest PKZip standard > > > Key: COMPRESS-549 > URL: https://issues.apache.org/jira/browse/COMPRESS-549 > Project: Commons Compress > Issue Type: Bug >Affects Versions: 1.20 >Reporter: Sebastian Kürten >Priority: Major > > I came across some Zip archives that cannot be read using > ZipArchiveInputStream. Investigating the issue, I found that > java.util.zip.ZipInputStream from the JDK shows the same behavior while > java.util.zip.ZipFile seems to do fine. For java.util.zip.ZipInputStream, the > issue has been reported here: > [https://bugs.openjdk.java.net/browse/JDK-8143613] and an example file is > provided. I copied the testing file into a repository for testing. Here's the > example file: > [https://github.com/sebkur/test-zip-impls/blob/master/src/test/java/de/topobyte/zip/tests/jdk8143613/TestData.java] > and the test that fails reading that data using ZipArchiveInputStream: > [https://github.com/sebkur/test-zip-impls/blob/master/src/test/java/de/topobyte/zip/tests/jdk8143613/TestCommonsZipInputStream.java] > > > If this file is indeed a ZIP archive consistent with the PKZip spec, I think > commons-compress does not work according to the spec. I would appreciate if > someone could look into this and verify if this is indeed a bug in > commons-compress. Thanks! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (COMPRESS-549) Inconsistency with latest PKZip standard
[
https://issues.apache.org/jira/browse/COMPRESS-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178850#comment-17178850
]
Peter Lee commented on COMPRESS-549:
Hi [~sebkur]
I think there are some problems in your testData. The variable
_general_purpose_bit_flag_bit3_on_ seems to be used to indicate if Data
Descriptor is used or not. (you set it to be true in your test cases) But it
seems the test data lack of Data Descriptor. I modified the data a little bit
and it is passed.
{code:java}
public static byte[] get(boolean general_purpose_bit_flag_bit3_on)
{
final byte gpbf = (byte) (general_purpose_bit_flag_bit3_on ? 0x08
: 0x00);
return new byte[] {
// Local File header
'P', 'K', 3, 4, // Local File Header Signature
13, 0, // Version needed to extract
gpbf, 8, // General purpose bit flag
ZipEntry.STORED, 0, // Compression method
'q', 'l', 't', 'G', // Last Modification time & date
0, 0, 0, 0, // CRC32
0, 0, 0, 0, // Compressed Size
0, 0, 0, 0, // Uncompressed Size
12, 0, // File name length
0, 0, // Extra field length
'F', 'o', 'l', 'd', 'e', 'r', '_', 'n', 'a', 'm', 'e', '/',
//
// Data Descriptor, this sector should not appear if
general_purpose_bit_flag_bit3_on is false
'P', 'K', 7, 8, // Data Descriptor signature
0, 0, 0, 0, // CRC32 in Data Descriptor
0, 0, 0, 0, // compressed size in Data Descriptor
0, 0, 0, 0, // uncompressed size Data Descriptor
//
// File name
// Central directory file header
'P', 'K', 1, 2, // Central Directory File Header Signature
13, 0, // Version made by
13, 0, // Version needed to extract
gpbf, 8, // General purpose bit flag
ZipEntry.STORED, 0, // Compression method
'q', 'l', 't', 'G', // Last Modification time & date
0, 0, 0, 0, // CRC32
0, 0, 0, 0, // Compressed Size
0, 0, 0, 0, // Uncompressed Size
12, 0, // File name length
0, 0, // Extra field length
0, 0, // File comment length
0, 0, // Disk number where file starts
0, 0, // Internal File attributes
0, 0, 0, 0, // External File attributes
0, 0, 0, 0, // Relative offset of local header file
'F', 'o', 'l', 'd', 'e', 'r', '_', 'n', 'a', 'm', 'e', '/',
// File name
// End of Central Directory Record
'P', 'K', 5, 6, // Local File Header Signature
0, 0, // Number of this disk
0, 0, // Disk where CD starts
1, 0, // Number of CD records on this disk
1, 0, // Total number of records
58, 0, 0, 0, // Size of CD
42, 0, 0, 0, // Offset of start of CD
0, 0, // Comment length
};
}
{code}
Please note the data I added between Local File Header and Central Directory
file.
> Inconsistency with latest PKZip standard
>
>
> Key: COMPRESS-549
> URL: https://issues.apache.org/jira/browse/COMPRESS-549
> Project: Commons Compress
> Issue Type: Bug
>Affects Versions: 1.20
>Reporter: Sebastian Kürten
>Priority: Major
>
> I came across some Zip archives that cannot be read using
> ZipArchiveInputStream. Investigating the issue, I found that
> java.util.zip.ZipInputStream from the JDK shows the same behavior while
> java.util.zip.ZipFile seems to do fine. For java.util.zip.ZipInputStream, the
> issue has been reported here:
> [https://bugs.openjdk.java.net/browse/JDK-8143613] and an example file is
> provided. I copied the testing file into a repository for testing. Here's the
> example file:
> [https://github.com/sebkur/test-zip-impls/blob/master/src/test/java/de/topobyte/zip/tests/jdk8143613/TestData.java]
> and the test that fails reading that data using ZipArchiveInputStream:
> [https://github.com/sebkur/test-zip-impls/blob/master/src/test/java/de/topobyte/zip/tests/jdk8143613/TestCommonsZipInputStream.java]
>
>
> If this file is indeed a ZIP archive consistent with the PKZip spec, I think
> commons-compress does not work according to the spec. I would appreciate if
> someone could look into this and verify if this is indeed a bug in
> commons-compress. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
