xgbtxy opened a new pull request, #54725: URL: https://github.com/apache/arrow/pull/54725
### Rationale for this change `drop_null` checks physical null counts and validity bitmaps, which miss the logical nulls in dense and sparse unions. As a result, it can retain null values or rows across Array, ChunkedArray, RecordBatch and Table inputs. This addresses the **union portion** of #43851. REE selection support remains outside this patch; it does not resolve the whole issue. ### What changes are included in this PR? Use logical null counts and `is_valid` masks for unions across the four input kinds. Keep the existing physical-bitmap paths for other types, including dictionary arrays. A `MayHaveLogicalNulls` check avoids scanning union values when their children cannot contain nulls. Regressions cover dense/sparse unions, non-default type codes, slices, empty/all-null inputs, chunk boundaries, and row filtering with both union and ordinary columns. ### Are these changes tested? Windows x64, MSVC 19.44, Release, static Arrow/GTest 1.18.0 test linkage: - The same three union regressions fail with unmodified production source and pass with this patch. - `arrow-compute-vector-selection-test`: **170 passed**, one pre-existing disabled test. This includes the three union regressions. - `clang-format` 18.1.8 and `git diff --check`: pass. The original behavior was also reproduced with PyArrow 26.0.0, using explicit validity filtering and Python row selection as controls. The patched validation above is the native C++ suite. ### Are there any user-facing changes? Logical union nulls are now removed, including the corresponding full rows in RecordBatch/Table inputs. **This PR contains a "Critical Fix"** under the incorrect-data criterion: `drop_null` previously returned rows/values that should have been removed. There are no public API signature changes. ### Was AI used for this PR? Codex generated the implementation and regression tests, performed code review, and ran the local validation described above. **PR code and description written by:** - [ ] Human - [x] AI **Reviewed before submission by:** - [ ] Human - [x] AI - [ ] Not reviewed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
