fallintoplace opened a new pull request, #1322: URL: https://github.com/apache/arrow-go/pull/1322
**What** - Reuse DELTA length and prefix scratch buffers across decoder pages. - Add tests for scratch reuse and page-to-page correctness. **Why** - The column reader caches decoders by encoding. - Each page currently allocates fresh length and prefix arrays even when the decoder already has enough capacity. **Implementation** - Keep stable backing buffers for decoded lengths and prefix lengths. - Expose sliced views to the existing Decode and Discard paths. - Keep the existing page-size behavior when a larger page needs more capacity. - The existing repeated `SetData + Decode` benchmark now shows 18 to 15 allocations/op and about 512 KiB less allocation for a full 65,536-value prefix-heavy page. Tests: - `go test ./parquet/internal/encoding -count=1` - `go test ./parquet/file -run '^(TestWithEOFReader|TestInvalidHeaders|TestInvalidFooter|TestIncompleteMetadata|TestDeltaLengthByteArrayPackingWithNulls|TestDeltaBinaryPackedMultipleBatches|TestPageStreaming.*|TestPrimitiveReader|TestFullSeekRow|TestSkipEmptyRepeatedRows)$' -count=1` - `go test -race ./parquet/internal/encoding -run 'TestDelta(ByteArrayDecoderReusesPageScratch|LengthByteArrayDecoderReusesLengthScratch|ByteArrayDecoderKeepsResultsAcrossPages|ByteArrayDecoderRejectsInvalidPrefixes)$' -count=1` - `go vet ./parquet/internal/encoding` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
