hcrosse opened a new pull request, #1527: URL: https://github.com/apache/arrow-go/pull/1527
### Rationale for this change Dictionary column statistics can exclude `""` from their bounds, causing readers that prune by statistics to skip matching rows. ### What changes are included in this PR? `ByteArrayStatistics.UpdateFromArrow` skips nulls by validity, not length. A missing value buffer becomes an empty slice because `nil` means an unset bound. This also keeps bytes in null slots out of the bounds. ### Are these changes tested? `TestDictionaryArrayStatisticsEmptyString` writes dictionary columns and checks their stored bounds. `TestByteArrayStatisticsUpdateFromArrowSkipsNullValueBytes` checks a null slot that still holds bytes. Both tests fail on main and pass with this change. The case in the issue produces `min="" max="az"` with this change. With Go 1.26.10, vet, affected `-race -tags assert,test` tests, full `go test -tags assert,test ./parquet/...` with pinned test data, lint, regeneration and the Release Audit Tool passed. ### Are there any user-facing changes? Yes. New files include empty strings and exclude null slots in these statistics. Existing files are unchanged. There is no API change. Closes https://github.com/apache/arrow-go/issues/1524. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
