1932819331hy-debug opened a new pull request, #53948: URL: https://github.com/apache/arrow/pull/53948
### Rationale for this change Casting a list value field to non-nullable currently accepts null values, producing an array whose data contradicts its nested field nullability. For example, `[[1, null, 3], [], [4, 5]]` can be cast to `list<item: int64 not null>`. Fixes #43146. ### What changes are included in this PR? - Validate the converted values in list, large-list, and fixed-size-list casts. Checking the output also catches nulls introduced by dictionary decoding. - Check only values referenced by valid parent lists, respecting both parent and child offsets. - Propagate ancestor validity before recursive nested casts, including mixed struct/list nesting. Compact hidden map entries with the existing `Flatten` utility so nested keys retain the map's non-null key constraint. ### Are these changes tested? - Built `arrow-compute-scalar-cast-test` from the current Arrow 27 source with Apple Clang 17 in Debug mode; all **113 tests passed**. - Five new C++ regression tests cover safe/unsafe casts, all list/fixed-size-list conversion directions, dictionary decoding, parent/child slices, unknown null counts, null and empty lists, NullType, and masked descendants through lists, structs, and maps. List-view inputs also have rejection coverage. - Added four PyArrow `Table.cast` regression cases for list/large-list and safe/unsafe modes. All four reproduce the missing exception against released PyArrow 26.0.0. The Python extension was not rebuilt locally; the fixed behavior was validated through the native C++ suite. - Applied clang-format and checked the diff for whitespace errors. ### Are there any user-facing changes? List casts that would produce null values in a non-nullable value field now raise `ArrowInvalid`, including with `safe=False`. The added list-value check respects null ancestors and slice boundaries. **This PR contains a "Critical Fix"** for casts that currently produce data inconsistent with the declared nested nullability. ### Was AI used for this PR? Code, regression tests, and this description were generated with AI assistance. The implementation was independently reviewed by AI agents, exercised against the reported failure, and validated with the native test suite above. **PR code and description written by:** - [ ] Human - [x] AI **Reviewed before submission by:** - [ ] Human - [x] AI - [ ] Not reviewed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
