yadavay-amzn opened a new pull request, #3525: URL: https://github.com/apache/parquet-java/pull/3525
## What changes were made? Fix integer overflow in `CapacityByteArrayOutputStream.addSlab()` that causes `ArithmeticException` when writing large `ARRAY<STRING>` columns (issue #3261). ### Root cause (identified by @Kimahriman) The overflow check in `addSlab()` used `bytesUsed` to detect overflow, but `bytesUsed` is not updated until *after* `addSlab()` returns in `write()`. Additionally, `nextSlabSize` can be larger than `minimumSize` (due to the doubling strategy), so checking only `bytesUsed + minimumSize` was insufficient. This meant `bytesAllocated = Math.addExact(this.bytesAllocated, nextSlabSize)` could overflow without being caught by the guard, throwing an uncaught `ArithmeticException` instead of the intended `OutOfMemoryError`. ### Fix 1. **Use `bytesAllocated` instead of `bytesUsed`** for the overflow check — `bytesAllocated` is always up to date when `addSlab()` is called. 2. **Cap `nextSlabSize`** when it would cause `bytesAllocated` to overflow `Integer.MAX_VALUE`, preventing the uncaught `ArithmeticException` on the `Math.addExact` call. ### Tests Added `TestCapacityByteArrayOutputStreamOverflow` with two tests: - Verifies that slab allocation near `Integer.MAX_VALUE` succeeds (previously threw `ArithmeticException`) - Verifies that a true overflow still throws `OutOfMemoryError` as intended -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
