fallintoplace opened a new pull request, #1238: URL: https://github.com/apache/arrow-go/pull/1238
## Summary - Pool gzip writers by compression level. - Reset a writer before returning it to the pool. - Avoid retaining the destination page buffer in the cached writer. - Add a concurrent correctness test and page-size benchmarks. ## Benchmark Apple M1 Pro. The output buffer is reused. Median of 3 runs. | Input | upstream main | this PR | change | | --- | ---: | ---: | ---: | | Repeated, 64 KiB | 115 us, 1.08 MiB, 15 allocs | 25.7 us, 48 B, 1 alloc | 4.5x faster | | Random, 64 KiB | 94.7 us, 1.08 MiB, 15 allocs | 15.6 us, 117 B, 1 alloc | 6.1x faster | ```text go test ./parquet/compress -run '^$' -bench '^BenchmarkGzipEncodePages$' -benchmem -benchtime=200ms -count=3 ``` ## Tests - `PARQUET_TEST_DATA=/path/to/parquet-testing/data go test ./parquet/... -count=1` - `PARQUET_TEST_DATA=/path/to/parquet-testing/data go test -race ./parquet/compress ./parquet/file ./parquet/pqarrow -count=1` - `go vet ./parquet/compress ./parquet/file` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
