[ https://issues.apache.org/jira/browse/MTOOLCHAINS-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17446123#comment-17446123 ]
Michael Osipov commented on MTOOLCHAINS-39: ------------------------------------------- Well, I can confirm this and thanks to you I learned a bit more about ZIP today. Lets dive into the analysis: My setup: {noformat} osipovmi@deblndw011x:/tmp/maven-toolchains $ uname -a FreeBSD deblndw011x.ad001.siemens.net 12.2-STABLE FreeBSD 12.2-STABLE #0: Thu Sep 23 18:58:50 CEST 2021 osipovmi@deblndw011x:/tmp/maven-toolchains $ which unzip /usr/bin/unzip osipovmi@deblndw011x:/tmp/maven-toolchains $ pkg which /usr/local/bin/zipdetails /usr/local/bin/zipdetails was installed by package perl5-5.30.3_1 osipovmi@deblndw011x:/tmp/maven-toolchains $ java -version openjdk version "1.8.0_302" OpenJDK Runtime Environment (build 1.8.0_302-b08) OpenJDK 64-Bit Server VM (build 25.302-b08, mixed mode) {noformat} Lets unzip first: {noformat} $ unzip maven-toolchain-1.0.jar Archive: maven-toolchain-1.0.jar extracting: META-INF extracting: META-INF/MANIFEST.MF extracting: org extracting: org/apache extracting: org/apache/maven extracting: org/apache/maven/toolchain extracting: org/apache/maven/toolchain/java extracting: org/apache/maven/toolchain/java/DefaultJavaToolchainFactory.class extracting: org/apache/maven/toolchain/java/JavaToolChain.class extracting: org/apache/maven/toolchain/java/DefaultJavaToolChain.class extracting: org/apache/maven/toolchain/ToolchainManagerPrivate.class extracting: org/apache/maven/toolchain/DefaultToolchain.class extracting: org/apache/maven/toolchain/ToolchainPrivate.class extracting: org/apache/maven/toolchain/MisconfiguredToolchainException.class extracting: org/apache/maven/toolchain/ToolchainManager.class extracting: org/apache/maven/toolchain/ToolchainFactory$1.class extracting: org/apache/maven/toolchain/model extracting: org/apache/maven/toolchain/model/ToolchainModel.class extracting: org/apache/maven/toolchain/model/PersistedToolchains.class extracting: org/apache/maven/toolchain/model/io extracting: org/apache/maven/toolchain/model/io/xpp3 extracting: org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Reader.class extracting: org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Writer.class extracting: org/apache/maven/toolchain/RequirementMatcherFactory$ExactMatcher.class extracting: org/apache/maven/toolchain/Toolchain.class extracting: org/apache/maven/toolchain/RequirementMatcherFactory.class extracting: org/apache/maven/toolchain/RequirementMatcherFactory$1.class extracting: org/apache/maven/toolchain/ToolchainManagerPrivate$1.class extracting: org/apache/maven/toolchain/DefaultToolchainManager.class extracting: org/apache/maven/toolchain/ToolchainFactory.class extracting: org/apache/maven/toolchain/ToolchainManager$1.class extracting: org/apache/maven/toolchain/RequirementMatcherFactory$VersionMatcher.class extracting: org/apache/maven/toolchain/RequirementMatcher.class extracting: META-INF/plexus extracting: META-INF/plexus/components.xml extracting: META-INF/maven extracting: META-INF/maven/org.apache.maven extracting: META-INF/maven/org.apache.maven/maven-toolchain extracting: META-INF/maven/org.apache.maven/maven-toolchain/pom.xml extracting: META-INF/maven/org.apache.maven/maven-toolchain/pom.properties osipovmi@deblndw011x:/tmp/maven-toolchains $ unzip maven-toolchain-2.0.9.jar Archive: maven-toolchain-2.0.9.jar creating: META-INF/ extracting: META-INF/MANIFEST.MF extracting: META-INF/LICENSE extracting: META-INF/NOTICE creating: META-INF/plexus/ extracting: META-INF/plexus/components.xml creating: org/ creating: org/apache/ creating: org/apache/maven/ creating: org/apache/maven/toolchain/ extracting: org/apache/maven/toolchain/DefaultToolchain.class extracting: org/apache/maven/toolchain/DefaultToolchainManager.class creating: org/apache/maven/toolchain/java/ extracting: org/apache/maven/toolchain/java/DefaultJavaToolChain.class extracting: org/apache/maven/toolchain/java/DefaultJavaToolchainFactory.class extracting: org/apache/maven/toolchain/java/JavaToolChain.class extracting: org/apache/maven/toolchain/MisconfiguredToolchainException.class creating: org/apache/maven/toolchain/model/ creating: org/apache/maven/toolchain/model/io/ creating: org/apache/maven/toolchain/model/io/xpp3/ extracting: org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Reader.class extracting: org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Writer.class extracting: org/apache/maven/toolchain/model/PersistedToolchains.class extracting: org/apache/maven/toolchain/model/ToolchainModel.class extracting: org/apache/maven/toolchain/RequirementMatcher.class extracting: org/apache/maven/toolchain/RequirementMatcherFactory$1.class extracting: org/apache/maven/toolchain/RequirementMatcherFactory$ExactMatcher.class extracting: org/apache/maven/toolchain/RequirementMatcherFactory$VersionMatcher.class extracting: org/apache/maven/toolchain/RequirementMatcherFactory.class extracting: org/apache/maven/toolchain/Toolchain.class extracting: org/apache/maven/toolchain/ToolchainFactory$1.class extracting: org/apache/maven/toolchain/ToolchainFactory.class extracting: org/apache/maven/toolchain/ToolchainManager$1.class extracting: org/apache/maven/toolchain/ToolchainManager.class extracting: org/apache/maven/toolchain/ToolchainManagerPrivate$1.class extracting: org/apache/maven/toolchain/ToolchainManagerPrivate.class extracting: org/apache/maven/toolchain/ToolchainPrivate.class creating: META-INF/maven/ creating: META-INF/maven/org.apache.maven/ creating: META-INF/maven/org.apache.maven/maven-toolchain/ extracting: META-INF/maven/org.apache.maven/maven-toolchain/pom.xml extracting: META-INF/maven/org.apache.maven/maven-toolchain/pom.properties {noformat} You already see that the output is different in 1.0 it is extracting dirs and in 2.0.x is creating them. {{unzip}} handles this case for you. Where is the difference now? Spec says: {quote} 4.4.15 external file attributes: (4 bytes) The mapping of the external attributes is host-system dependent (see 'version made by'). For MS-DOS, the low order byte is the MS-DOS directory attribute byte. If input came from standard input, this field is set to zero. {noformat} Lets check the ZIP details on {{META-INF}} for 1.0: {noformat} .... 0000 0004 50 4B 03 04 LOCAL HEADER #1 04034B50 0004 0001 14 Extract Zip Spec 14 '2.0' 0005 0001 00 Extract OS 00 'MS-DOS' 0006 0002 08 00 General Purpose Flag 0008 [Bits 1-2] 0 'Normal Compression' [Bit 3] 1 'Streamed' 0008 0002 08 00 Compression Method 0008 'Deflated' 000A 0004 70 9F 5C 38 Last Mod Time 385C9F70 'Thu Feb 28 19:59:32 2008' 000E 0004 00 00 00 00 CRC 00000000 0012 0004 00 00 00 00 Compressed Length 00000000 0016 0004 00 00 00 00 Uncompressed Length 00000000 001A 0002 08 00 Filename Length 0008 001C 0002 04 00 Extra Length 0004 001E 0008 4D 45 54 41 Filename 'META-INF' 2D 49 4E 46 0026 0002 FE CA Extra ID #0001 CAFE 'Java Executable' 0028 0002 00 00 Length 0000 002A 0002 03 00 PAYLOAD .. ... 7246 0004 50 4B 01 02 CENTRAL HEADER #1 02014B50 724A 0001 14 Created Zip Spec 14 '2.0' 724B 0001 00 Created OS 00 'MS-DOS' 724C 0001 14 Extract Zip Spec 14 '2.0' 724D 0001 00 Extract OS 00 'MS-DOS' 724E 0002 08 00 General Purpose Flag 0008 [Bits 1-2] 0 'Normal Compression' [Bit 3] 1 'Streamed' 7250 0002 08 00 Compression Method 0008 'Deflated' 7252 0004 70 9F 5C 38 Last Mod Time 385C9F70 'Thu Feb 28 19:59:32 2008' 7256 0004 00 00 00 00 CRC 00000000 725A 0004 02 00 00 00 Compressed Length 00000002 725E 0004 00 00 00 00 Uncompressed Length 00000000 7262 0002 08 00 Filename Length 0008 7264 0002 04 00 Extra Length 0004 7266 0002 00 00 Comment Length 0000 7268 0002 00 00 Disk Start 0000 726A 0002 00 00 Int File Attributes 0000 [Bit 0] 0 'Binary Data' 726C 0004 00 00 00 00 Ext File Attributes 00000000 7270 0004 00 00 00 00 Local Header Offset 00000000 7274 0008 4D 45 54 41 Filename 'META-INF' 2D 49 4E 46 727C 0002 FE CA Extra ID #0001 CAFE 'Java Executable' 727E 0002 00 00 Length 0000 ... {noformat} Lets check the ZIP details on {{META-INF}} for 2.0.x: {noformat} .... 0000 0004 50 4B 03 04 LOCAL HEADER #1 04034B50 0004 0001 14 Extract Zip Spec 14 '2.0' 0005 0001 00 Extract OS 00 'MS-DOS' 0006 0002 08 00 General Purpose Flag 0008 [Bits 1-2] 0 'Normal Compression' [Bit 3] 1 'Streamed' 0008 0002 08 00 Compression Method 0008 'Deflated' 000A 0004 D4 62 87 38 Last Mod Time 388762D4 'Mon Apr 7 12:22:40 2008' 000E 0004 00 00 00 00 CRC 00000000 0012 0004 00 00 00 00 Compressed Length 00000000 0016 0004 00 00 00 00 Uncompressed Length 00000000 001A 0002 09 00 Filename Length 0009 001C 0002 04 00 Extra Length 0004 001E 0009 4D 45 54 41 Filename 'META-INF/' 2D 49 4E 46 2F 0027 0002 FE CA Extra ID #0001 CAFE 'Java Executable' 0029 0002 00 00 Length 0000 002B 0002 03 00 PAYLOAD .. ... 8583 0004 50 4B 01 02 CENTRAL HEADER #1 02014B50 8587 0001 14 Created Zip Spec 14 '2.0' 8588 0001 00 Created OS 00 'MS-DOS' 8589 0001 14 Extract Zip Spec 14 '2.0' 858A 0001 00 Extract OS 00 'MS-DOS' 858B 0002 08 00 General Purpose Flag 0008 [Bits 1-2] 0 'Normal Compression' [Bit 3] 1 'Streamed' 858D 0002 08 00 Compression Method 0008 'Deflated' 858F 0004 D4 62 87 38 Last Mod Time 388762D4 'Mon Apr 7 12:22:40 2008' 8593 0004 00 00 00 00 CRC 00000000 8597 0004 02 00 00 00 Compressed Length 00000002 859B 0004 00 00 00 00 Uncompressed Length 00000000 859F 0002 09 00 Filename Length 0009 85A1 0002 04 00 Extra Length 0004 85A3 0002 00 00 Comment Length 0000 85A5 0002 00 00 Disk Start 0000 85A7 0002 00 00 Int File Attributes 0000 [Bit 0] 0 'Binary Data' 85A9 0004 00 00 00 00 Ext File Attributes 00000000 85AD 0004 00 00 00 00 Local Header Offset 00000000 85B1 0009 4D 45 54 41 Filename 'META-INF/' 2D 49 4E 46 2F 85BA 0002 FE CA Extra ID #0001 CAFE 'Java Executable' 85BC 0002 00 00 Length 0000 ... {noformat} Well, although this is DOS vendor, the dir byte is not set, but the filename contains a trailing slash to denote a directory. Now why does Java fail? {code:java} public boolean isDirectory() { return name.endsWith("/"); } {code} Why does my FreeBSD {{unzip}} does work? Magic from {{libarchive}}: https://github.com/libarchive/libarchive/blob/master/libarchive/archive_read_support_format_zip.c {code:c} } else if (zip_entry->system == 0) { // Interpret MSDOS directory bit if (0x10 == (external_attributes & 0x10)) { zip_entry->mode = AE_IFDIR | 0775; } else { zip_entry->mode = AE_IFREG | 0664; } if (0x01 == (external_attributes & 0x01)) { // Read-only bit; strip write permissions zip_entry->mode &= 0555; } {code} and {code:c} /* Make sure that entries with a trailing '/' are marked as directories * even if the External File Attributes contains bogus values. If this * is not a directory and there is no type, assume a regular file. */ if ((zip_entry->mode & AE_IFMT) != AE_IFDIR) { int has_slash; wp = archive_entry_pathname_w(entry); if (wp != NULL) { len = wcslen(wp); has_slash = len > 0 && wp[len - 1] == L'/'; } else { cp = archive_entry_pathname(entry); len = (cp != NULL)?strlen(cp):0; has_slash = len > 0 && cp[len - 1] == '/'; } /* Correct file type as needed. */ if (has_slash) { zip_entry->mode &= ~AE_IFMT; zip_entry->mode |= AE_IFDIR; zip_entry->mode |= 0111; } else if ((zip_entry->mode & AE_IFMT) == 0) { zip_entry->mode |= AE_IFREG; } } /* Make sure directories end in '/' */ if ((zip_entry->mode & AE_IFMT) == AE_IFDIR) { wp = archive_entry_pathname_w(entry); if (wp != NULL) { len = wcslen(wp); if (len > 0 && wp[len - 1] != L'/') { struct archive_wstring s; archive_string_init(&s); archive_wstrcat(&s, wp); archive_wstrappend_wchar(&s, L'/'); archive_entry_copy_pathname_w(entry, s.s); archive_wstring_free(&s); } } else { cp = archive_entry_pathname(entry); len = (cp != NULL)?strlen(cp):0; if (len > 0 && cp[len - 1] != '/') { struct archive_string s; archive_string_init(&s); archive_strcat(&s, cp); archive_strappend_char(&s, '/'); archive_entry_set_pathname(entry, s.s); archive_string_free(&s); } } } {code} Although the payload is not empty (0x03, 0x00), it is decompressed, although it does not make sense for dirs, zlib inflates this and since no data is there is is extracted with nothing writtten. Subsequent files in that dir likely notice that parent does not exist and create it. So, actually both files are crap, but 1.0 jus stinks more. Lets have a look at a more recent file: {noformat} osipovmi@deblndw011x:~/apache-maven-4.0.0-alpha-1-SNAPSHOT/lib $ zipdetails maven-core-4.0.0-alpha-1-SNAPSHOT.jar | less ... 000E3 LOCAL HEADER #2 04034B50 000E7 Extract Zip Spec 0A '1.0' 000E8 Extract OS 00 'MS-DOS' 000E9 General Purpose Flag 0800 [Bit 11] 1 'Language Encoding' 000EB Compression Method 0000 'Stored' 000ED Last Mod Time 52854189 'Mon Apr 5 08:12:18 2021' 000F1 CRC 00000000 000F5 Compressed Length 00000000 000F9 Uncompressed Length 00000000 000FD Filename Length 0009 000FF Extra Length 0000 00101 Filename 'META-INF/' ... 967E6 CENTRAL HEADER #2 02014B50 967EA Created Zip Spec 14 '2.0' 967EB Created OS 03 'Unix' 967EC Extract Zip Spec 0A '1.0' 967ED Extract OS 00 'MS-DOS' 967EE General Purpose Flag 0800 [Bit 11] 1 'Language Encoding' 967F0 Compression Method 0000 'Stored' 967F2 Last Mod Time 52854189 'Mon Apr 5 08:12:18 2021' 967F6 CRC 00000000 967FA Compressed Length 00000000 967FE Uncompressed Length 00000000 96802 Filename Length 0009 96804 Extra Length 0000 96806 Comment Length 0000 96808 Disk Start 0000 9680A Int File Attributes 0000 [Bit 0] 0 'Binary Data' 9680C Ext File Attributes 41ED0010 [Bit 4] Directory 96810 Local Header Offset 000000E3 96814 Filename 'META-INF/' ... {noformat} Looks just the way is should look like, stored, trailing slash and 0x10 for directory is set int he extra file attributes. Do you rely on this file? > Can't unzip/unjar maven-toolchain-1.0.jar that is available at Maven Central > ---------------------------------------------------------------------------- > > Key: MTOOLCHAINS-39 > URL: https://issues.apache.org/jira/browse/MTOOLCHAINS-39 > Project: Maven Toolchains Plugin > Issue Type: Bug > Affects Versions: 1.0 > Reporter: Thomas Cunningham > Priority: Major > > jar tvf maven-toolchain-1.0.jar that is available from maven central reports > reasonable results > However.... > jar xvf doesn't work > ❯ jar xvf maven-toolchain-1.0.jar > inflated: META-INF > java.io.IOException: META-INF : could not create directory > at jdk.jartool/sun.tools.jar.Main.extractFile(Main.java:1449) > at jdk.jartool/sun.tools.jar.Main.extract(Main.java:1364) > at jdk.jartool/sun.tools.jar.Main.run(Main.java:409) > at jdk.jartool/sun.tools.jar.Main.main(Main.java:1681) > unzip doesn't work > ❯ unzip maven-toolchain-1.0.jar > Archive: maven-toolchain-1.0.jar > replace META-INF? [y]es, [n]o, [A]ll, [N]one, [r]ename: A > inflating: META-INF > checkdir error: META-INF exists but is not directory > unable to process META-INF/MANIFEST.MF. > inflating: org > checkdir error: org exists but is not directory > unable to process org/apache. > checkdir error: org exists but is not directory > unable to process org/apache/maven. > checkdir error: org exists but is not directory > unable to process org/apache/maven/toolchain. > checkdir error: org exists but is not directory > unable to process org/apache/maven/toolchain/java. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/java/DefaultJavaToolchainFactory.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/java/JavaToolChain.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/java/DefaultJavaToolChain.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/ToolchainManagerPrivate.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/DefaultToolchain.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/ToolchainPrivate.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/MisconfiguredToolchainException.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/ToolchainManager.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/ToolchainFactory$1.class. > checkdir error: org exists but is not directory > unable to process org/apache/maven/toolchain/model. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/model/ToolchainModel.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/model/PersistedToolchains.class. > checkdir error: org exists but is not directory > unable to process org/apache/maven/toolchain/model/io. > checkdir error: org exists but is not directory > unable to process org/apache/maven/toolchain/model/io/xpp3. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Reader.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Writer.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/RequirementMatcherFactory$ExactMatcher.class. > checkdir error: org exists but is not directory > unable to process org/apache/maven/toolchain/Toolchain.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/RequirementMatcherFactory.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/RequirementMatcherFactory$1.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/ToolchainManagerPrivate$1.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/DefaultToolchainManager.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/ToolchainFactory.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/ToolchainManager$1.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/RequirementMatcherFactory$VersionMatcher.class. > checkdir error: org exists but is not directory > unable to process > org/apache/maven/toolchain/RequirementMatcher.class. > checkdir error: META-INF exists but is not directory > unable to process META-INF/plexus. > checkdir error: META-INF exists but is not directory > unable to process META-INF/plexus/components.xml. > checkdir error: META-INF exists but is not directory > unable to process META-INF/maven. > checkdir error: META-INF exists but is not directory > unable to process META-INF/maven/org.apache.maven. > checkdir error: META-INF exists but is not directory > unable to process > META-INF/maven/org.apache.maven/maven-toolchain. > checkdir error: META-INF exists but is not directory > unable to process > META-INF/maven/org.apache.maven/maven-toolchain/pom.xml. > checkdir error: META-INF exists but is not directory > unable to process > META-INF/maven/org.apache.maven/maven-toolchain/pom.properties. > -- This message was sent by Atlassian Jira (v8.20.1#820001)