&res created ARROW-18411: ---------------------------- Summary: [Python] MapType comparison ignores nullable flag of item_field Key: ARROW-18411 URL: https://issues.apache.org/jira/browse/ARROW-18411 Project: Apache Arrow Issue Type: Bug Components: Python Environment: pyarrow==10.0.1
Reporter: &res By default MapType value fields are nullable: {code:java} pa.map_(pa.string(), pa.int32()).item_field.nullable == True {code} It is possible to mark the value field of a MapType as not-nullable: {code:java} pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False)).item_field.nullable == False{code} But comparing these two types, that are semantically different, returns True: {code:java} pa.map_(pa.string(), pa.int32()) == pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False)) # Returns True {code} So it looks like the comparison omits the nullable flag. {code:java} import pyarrow as pa import pytest print(pa.__version__) map_type = pa.map_(pa.string(), pa.int32()) pa.array( [[("one", 1), ("two", 2), ("null", None)]], map_type ) with pytest.raises(pa.ArrowInvalid, match=r"Invalid Map: key field can not contain null values"): pa.array( [[("one", 1), ("two", 2), (None, None)]], map_type ) map_type = pa.map_(pa.string(), pa.int32()) pa.array( [[("one", 1), ("two", 2), ("null", None)]], map_type ) non_null_map_type = pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=False)) nullable_map_type = pa.map_(pa.string(), pa.field("value", pa.int32(), nullable=True)) pa.array( [[("one", 1), ("two", 2), ("null", None)]], map_type ) assert nullable_map_type == map_type # Should be different assert str(nullable_map_type) == str(map_type) assert non_null_map_type == map_type assert non_null_map_type.item_type == map_type.item_type assert non_null_map_type.item_field != map_type.item_field assert non_null_map_type.item_field.nullable != map_type.item_field.nullable assert non_null_map_type.item_field.name == map_type.item_field.name assert str(non_null_map_type) != str(map_type.item_field.name){code} -- This message was sent by Atlassian Jira (v8.20.10#820010)