[ https://issues.apache.org/jira/browse/COMPRESS-639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary D. Gregory updated COMPRESS-639: ------------------------------------- Summary: NullPointerException when adding multiple files with the same path and when Zip64Mode is in use. (was: Crash when adding multiple files with the same path and when Zip64Mode is in use.) > NullPointerException when adding multiple files with the same path and when > Zip64Mode is in use. > ------------------------------------------------------------------------------------------------ > > Key: COMPRESS-639 > URL: https://issues.apache.org/jira/browse/COMPRESS-639 > Project: Commons Compress > Issue Type: Bug > Components: Archivers, Compressors > Affects Versions: 1.21, 1.22 > Environment: Tested on MacBook Pro 2019 (2.6 GHz 6-Core Intel Core > i7, 32GB DDR4) > MacOS 13.1 > JDK 11.0.13 > Tested with commons-compress 1.21, 1.22 and 1.23-SNAPSHOT > Reporter: Andrew Gawron > Priority: Major > > Crash when adding 2 zip entries to a large archive. The entries had the same > name. > After the investigation we found out that ZipArchiveOutputStream has a race > condition. When adding two entries with the same entry name an entry is being > added to _entries_ LinkedList and then again it is being added to _metaData_ > HashMap. If the modification time ({_}race condition{_} here), name and other > params are the same then the metaData is not being updated for the second > entry. Then when createCentralFileHeader iterates over _entries_ the first > entry is being found in metaData keyset. It gets modified later by adding > extras. Then second entry tries to find its metadata but it fails because > metaData key has been changed. > Potential solution: container keys should be immutable and they should not be > modified after being added to the container. > Sample code that triggers exception: > {code:java} > @Test > public void shouldThrowDueToRaceConditionInZipArchiveOutputStream() throws > IOException, ExecutionException, InterruptedException { > var testOutputStream = new ByteArrayOutputStream(); > String fileContent = "A"; > final int NUM_OF_FILES = 100; > var inputStreams = new LinkedList<InputStream>(); > for (int i = 0; i < NUM_OF_FILES; i++) { > inputStreams.add(new > ByteArrayInputStream(fileContent.getBytes(StandardCharsets.UTF_8))); > } > var zipCreator = new ParallelScatterZipCreator(); > var zipArchiveOutputStream = new > ZipArchiveOutputStream(testOutputStream); > zipArchiveOutputStream.setUseZip64(Zip64Mode.Always); > for (int i = 0; i < inputStreams.size(); i++) { > ZipArchiveEntry zipArchiveEntry = new > ZipArchiveEntry("./dir/myfile.txt"); > zipArchiveEntry.setMethod(ZipEntry.DEFLATED); > final var inputStream = inputStreams.get(i); > zipCreator.addArchiveEntry(zipArchiveEntry, () -> inputStream); > } > zipCreator.writeTo(zipArchiveOutputStream); > zipArchiveOutputStream.close(); // it will throw NullPointerException > here > } {code} > Exception: > {code:java} > /* java.lang.NullPointerException at > org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream$EntryMetaData.access$800(ZipArchiveOutputStream.java:1998) > at > org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.createCentralFileHeader(ZipArchiveOutputStream.java:1356) > at > org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.writeCentralDirectoryInChunks(ZipArchiveOutputStream.java:580) > at > org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.finish(ZipArchiveOutputStream.java:546) > at > org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream.close(ZipArchiveOutputStream.java:1090) > at > com.xxx.yyy.impl.backuprestore.backup.container.StreamZipWriterTest.shouldThrowDueToRaceConditionInZipArchiveOutputStream(StreamZipWriterTest.java:130) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.mockito.internal.runners.DefaultInternalRunner$1$1.evaluate(DefaultInternalRunner.java:55) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at > org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at > org.junit.runners.ParentRunner.run(ParentRunner.java:413) at > org.mockito.internal.runners.DefaultInternalRunner$1.run(DefaultInternalRunner.java:100) > at > org.mockito.internal.runners.DefaultInternalRunner.run(DefaultInternalRunner.java:107) > at org.mockito.internal.runners.StrictRunner.run(StrictRunner.java:41) at > org.mockito.junit.MockitoJUnitRunner.run(MockitoJUnitRunner.java:163) at > org.junit.runner.JUnitCore.run(JUnitCore.java:137) at > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) > at > com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) > at > com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235) > at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54) */ {code} > Workaround: > Add a unique comment for each file so it will make the entry always unique > (ZipArchiveEntry#addComment) -- This message was sent by Atlassian Jira (v8.20.10#820010)