This is an automated email from the ASF dual-hosted git repository.
pjfanning pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/poi.git
The following commit(s) were added to refs/heads/trunk by this push:
new e69a23ed53 Buffer large/unknown-size zip entries to temp files in
poi-integration
e69a23ed53 is described below
commit e69a23ed53a40f47831139abd967e535d72ee232
Author: PJ Fanning <[email protected]>
AuthorDate: Tue Sep 1 11:17:37 2026 +0100
Buffer large/unknown-size zip entries to temp files in poi-integration
The CI builds have been failing in poi-integration with OutOfMemoryError,
with tests getting progressively slower before the heap gives out. The
trigger is spreadsheet/deep-data.xlsx: a ~965KB file whose xl/styles.xml
entry (Deflate64, size hidden behind a data descriptor) inflates to ~92MB -
just above the 0.01 minimum inflate ratio and just under the 100MB
per-entry cap, so neither guard fires. Every stream-based open of the file
materialises that entry on the heap (~216MB of allocations per open), and
TestAllFiles opens it several times across handlers and operations while
other tests run in parallel in the same 2GB JVM.
Use POI's existing defence: set
ZipInputStreamZipEntrySource.setThresholdBytesForTempFiles(16MB) so that
large or unknown-size entries are buffered to temp files instead of being
held in memory. This also gives the stress suite coverage of the
temp-file-backed entry path.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
---
.../src/test/java/org/apache/poi/stress/TestAllFiles.java | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git
a/poi-integration/src/test/java/org/apache/poi/stress/TestAllFiles.java
b/poi-integration/src/test/java/org/apache/poi/stress/TestAllFiles.java
index 573bda9b9a..9e178ffbd0 100644
--- a/poi-integration/src/test/java/org/apache/poi/stress/TestAllFiles.java
+++ b/poi-integration/src/test/java/org/apache/poi/stress/TestAllFiles.java
@@ -35,6 +35,7 @@ import java.util.stream.Stream;
import org.apache.poi.POIDataSamples;
import org.apache.poi.hssf.record.crypto.Biff8EncryptionKey;
+import org.apache.poi.openxml4j.util.ZipInputStreamZipEntrySource;
import org.apache.poi.util.IOUtils;
import org.apache.tools.ant.DirectoryScanner;
import org.junit.jupiter.api.function.Executable;
@@ -81,6 +82,14 @@ public class TestAllFiles {
}
ROOT_DIR = dir;
+
+ // Buffer large (or unknown-size) zip entries to temp files
instead of holding them in
+ // memory. Some test files inflate to nearly the 100MB
per-entry limit (e.g.
+ // spreadsheet/deep-data.xlsx has an entry that inflates to
~92MB while staying just
+ // above the minimum inflate ratio); materialising those on the
heap on every open,
+ // with tests running in parallel in a 2GB JVM, drives the CI
builds into
+ // OutOfMemoryError.
+
ZipInputStreamZipEntrySource.setThresholdBytesForTempFiles(16_000_000);
}
public static final String[] SCAN_EXCLUDES = {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]