hcrosse opened a new pull request, #1543: URL: https://github.com/apache/arrow-go/pull/1543
### Rationale for this change V2 pages report zero nulls when page statistics are absent, even if the pages contain nulls. ### What changes are included in this PR? The writer tracks levels below the maximum definition level for each page, the value Arrow C++ keeps in `num_buffered_nulls_`. `buildDataPageV2` uses that count instead of statistics, and `FlushCurrentPage` resets it. The plain path adds the levels it does not encode, and the spaced path counts the levels below the maximum. Both include null and empty lists. ### Are these changes tested? `TestDataPageV2NumNulls` has three rows: a spaced list with statistics off, `INTERVAL` with statistics on, and an optional column spanning two pages with statistics off. All three fail on main and pass with this change. The issue's reproduction reports `num_nulls=2` with statistics both on and off. With Go 1.26.10, vet, race tests on `parquet/file` and `parquet/pqarrow`, full `./parquet/...` tests with pinned `arrow-testing` and `parquet-testing` checked out, `golangci-lint` 2.13.2 and the Release Audit Tool passed. ### Are there any user-facing changes? Yes. V2 page headers report the correct `num_nulls`. Files already written keep the wrong value. There is no API change. Closes https://github.com/apache/arrow-go/issues/1542. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
