This is an automated email from the ASF dual-hosted git repository.
AlenkaF pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 3db7076bad1 GH-50847: [Python] Add Type_RUN_END_ENCODED to
_NESTED_TYPES set (#50848)
3db7076bad1 is described below
commit 3db7076bad124ab46c41bee3ba8f31509342d1de
Author: nishad shabbir <[email protected]>
AuthorDate: Thu Aug 27 19:31:56 2026 +0530
GH-50847: [Python] Add Type_RUN_END_ENCODED to _NESTED_TYPES set (#50848)
### Rationale for this change
`pyarrow.types.is_nested()` said a run-end encoded type wasn't nested,
while the C++ `arrow::is_nested()` says it is:
```python
>>> import pyarrow as pa
>>> t = pa.run_end_encoded(pa.int32(), pa.string())
>>> t.num_fields
2
>>> pa.types.is_nested(t)
False
```
The Python side reads from a hardcoded `_NESTED_TYPES` set in
`python/pyarrow/types.py` rather than from the C++ trait, so it has to be kept
in step by hand. Lining that set up against `is_nested()` in
`cpp/src/arrow/type_traits.h`, run-end encoded was the only type the two still
disagreed about — the list-view types and fixed-size list are already there.
It's also out of step with the type itself. Run-end encoded has two
children, the run ends and the values, and every other type in pyarrow that has
children answers `True` here.
This is the same thing that happened to fixed-size list in #40171, fixed by
#40172, and the list-view types were added after that. This looks like the last
one missed when run-end encoding went in.
### What changes are included in this PR?
Adds `Type_RUN_END_ENCODED` to `_NESTED_TYPES`, which is the whole fix.
In the test I've added the run-end encoded case, and while I was there also
a map and a dictionary. Map was nested already but wasn't asserted anywhere,
and dictionary is the interesting negative — it wraps a value type but is
deliberately not nested in either implementation, so pinning it means a later
change can't quietly sweep it in.
### Are these changes tested?
Yes. `test_is_nested_or_struct` fails on the current code and passes with
the change.
I don't have a local C++ build, so I checked this by running the updated
`test_types.py` against an installed pyarrow 25.0.0 with the same one-line
change applied to its `types.py`. Before the change that file had 87 passing
with `test_is_nested_or_struct` failing; after it, 88 passing. The two errors
and one failure I see in both runs are environmental on my machine and
unrelated — the errors are the `pickle_module` fixture, which comes from a
conftest I wasn't loading, and the failu [...]
### Are there any user-facing changes?
Yes, though it's small. `pa.types.is_nested()` now returns `True` for
run-end encoded types where it previously returned `False`. Anything branching
on that predicate will take the nested path for these types, which is the
intended answer and what the C++ implementation has always given. Nothing
inside pyarrow reads `is_nested` or `_NESTED_TYPES`, so the effect is limited
to callers.
* GitHub Issue: #50847
Authored-by: nishad shabbir <[email protected]>
Signed-off-by: AlenkaF <[email protected]>
---
python/pyarrow/tests/test_types.py | 3 +++
python/pyarrow/types.py | 3 ++-
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/python/pyarrow/tests/test_types.py
b/python/pyarrow/tests/test_types.py
index e0d74775e05..4f251d14334 100644
--- a/python/pyarrow/tests/test_types.py
+++ b/python/pyarrow/tests/test_types.py
@@ -244,7 +244,10 @@ def test_is_nested_or_struct():
assert types.is_nested(pa.large_list(pa.int32()))
assert types.is_nested(pa.list_view(pa.int32()))
assert types.is_nested(pa.large_list_view(pa.int32()))
+ assert types.is_nested(pa.map_(pa.string(), pa.int32()))
+ assert types.is_nested(pa.run_end_encoded(pa.int32(), pa.string()))
assert not types.is_nested(pa.int32())
+ assert not types.is_nested(pa.dictionary(pa.int32(), pa.string()))
def test_is_union():
diff --git a/python/pyarrow/types.py b/python/pyarrow/types.py
index ab4e5d1b992..b72427e54d0 100644
--- a/python/pyarrow/types.py
+++ b/python/pyarrow/types.py
@@ -45,7 +45,8 @@ _TEMPORAL_TYPES = ({lib.Type_TIMESTAMP,
_UNION_TYPES = {lib.Type_SPARSE_UNION, lib.Type_DENSE_UNION}
_NESTED_TYPES = {lib.Type_LIST, lib.Type_FIXED_SIZE_LIST, lib.Type_LARGE_LIST,
lib.Type_LIST_VIEW, lib.Type_LARGE_LIST_VIEW,
- lib.Type_STRUCT, lib.Type_MAP} | _UNION_TYPES
+ lib.Type_STRUCT, lib.Type_MAP,
+ lib.Type_RUN_END_ENCODED} | _UNION_TYPES
class TypesEnum(IntEnum):
