fallintoplace opened a new pull request, #953: URL: https://github.com/apache/arrow-go/pull/953
### Rationale for this change Parquet page sizes come from file metadata and are used to resize buffers. Positive int32 values can request multi-gigabyte allocations before decoding proves that the page is usable. Refs #950 ### What changes are included in this PR? - Add configurable compressed and uncompressed page-size limits to ReaderProperties. - Use defaults of 64 MiB compressed and 256 MiB uncompressed. - Apply the limits to dictionary pages and data page versions 1 and 2 before buffer allocation or decompression. - Preserve explicit overrides for workloads with unusually large valid pages. - Apply the same limits to encrypted and unencrypted column readers. ### Are these changes tested? Yes. Focused tests cover compressed, uncompressed, negative, and math.MaxInt32 declarations. Page-reader tests and vet checks pass. The full Parquet suite also requires the external parquet-testing data checkout, which is not present in this environment. ### Are there any user-facing changes? Readers now reject pages above the default limits. Applications that intentionally read larger pages can raise MaxCompressedPageSize and MaxUncompressedPageSize in ReaderProperties. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
