hhhizzz opened a new pull request, #11791: URL: https://github.com/apache/arrow-rs/pull/11791
# Which issue does this PR close? - Relates to #7973 and #11478. First part of the solution. This addresses panic handling only. Reading oversized `Utf8`/`Binary` batches successfully and bounding Mask decoding memory are separate changes. # Rationale for this change Reading `DELTA_LENGTH_BYTE_ARRAY` data can panic when the cumulative payload exceeds the output offset range. Map decoding can also panic when an underlying reader returns an error. These failures should return explicit errors through the normal reader error path. # What changes are included in this PR? - Use checked arithmetic when validating delta-length page lengths. - Check cumulative output offsets before reserving or copying payloads. On overflow, return an error without modifying the output or advancing the decoder. - Preserve the existing bulk-copy path for valid input. - Propagate Map child-reader errors instead of unwrapping them. # Are these changes tested? Yes. Added regression tests cover: - Offset boundaries, integer overflow, and values that fit 64-bit offsets but exceed 32-bit offsets. - Appending decoded values, empty strings, skipping, and end of input. - Propagation of errors from malformed Map data. - A real large-buffer overflow, verifying that the output and decoder state remain unchanged and subsequent reads still work. This memory-intensive test is ignored by default and was run explicitly. Both panic regressions fail against the original implementation and pass with these changes. Workspace tests and the additional Parquet async/experimental tests also pass. # Are there any significant user-facing changes? No additional changes beyond the error handling described above. There are no public API changes. AI assistance: OpenAI Codex assisted with implementation and regression tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
