fallintoplace opened a new pull request, #1269: URL: https://github.com/apache/arrow-go/pull/1269
This speeds up fixed-width Parquet dictionary materialization on AVX2 CPUs. **What changed** - Gather 8 x 32-bit or 4 x 64-bit dictionary values per SIMD iteration. - Cover int32, float32, int64, and float64. - Keep the scalar path for short batches, unsupported types, non-AVX2 CPUs, and noasm builds. - Reuse the existing dictionary index validation before the gather. - Add correctness tests for tails, boundaries, fallback behavior, and exact float bit patterns. - Add a benchmark matrix for dictionary sizes, batch sizes, and index distributions. **Benchmarks** `BenchmarkCopyDictionary` compares scalar and dispatch paths for: - dictionary sizes 16, 256, 4096, and 65536 - output batches of 1024 and 65536 values - sequential, clustered, and uniform indexes **Tests** - `go test -count=1 ./parquet/internal/... ./parquet/compress ./parquet/metadata ./parquet/schema ./parquet/variant` - `go test -race -count=1 ./parquet/internal/utils ./parquet/internal/encoding` - `go test -count=1 -tags noasm ./parquet/internal/utils ./parquet/internal/encoding` - `go vet -composites=false ./parquet/internal/utils ./parquet/internal/encoding` - Linux amd64 and arm64 test-package cross-builds -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
