L. C. Hsieh created SPARK-60919:
-----------------------------------
Summary: Make Row hashable like a tuple in the Spark Connect Rust
drop-in
Key: SPARK-60919
URL: https://issues.apache.org/jira/browse/SPARK-60919
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: L. C. Hsieh
Fix For: connect-rust-4.3.0
In the spark-connect-rust Python drop-in, {{Row}} defines {{__eq__}} but no
{{__hash__}}, so {{hash(Row(a=1))}} raises {{TypeError: unhashable type:
'builtins.Row'}}.
In PySpark, {{Row}} subclasses {{tuple}} and hashes like a tuple of its values
({{hash(Row(a=1)) == hash((1,))}}), so Rows can be put in sets and used as dict
keys. Rows holding unhashable values (lists, dicts) raise {{TypeError}}, as a
tuple holding them does.
The drop-in {{Row}} should hash the same way, consistently with its {{__eq__}}.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
