DB Tsai created SPARK-60685:
-------------------------------
Summary: Avoid generic scalar probes for exact Python date inputs
in Spark Connect Rust
Key: SPARK-60685
URL: https://issues.apache.org/jira/browse/SPARK-60685
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: DB Tsai
Assignee: DB Tsai
Fix For: connect-rust-4.3.0
Frozen10k x3 date tuple input takes32.79ms in Rust vs1.66ms Python; pandas
dates30.77ms vs1.79ms. py_to_value recognizes datetime.date only after
Decimal/numeric/string/container checks and repeated datetime module lookup.
Investigate a narrow exact datetime.date fast path preserving epoch days, valid
range, date subclasses and datetime precedence. Keep subclasses/datetime/pandas
Timestamp on existing semantics. This is distinct from exact numeric PR111 and
borrowed Arrow input PR114. Require measured benefit and correctness at
epoch/range boundaries before publishing.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
