DB Tsai created SPARK-60695:
-------------------------------
Summary: Print schema metadata instead of executing rows in Spark
Connect Rust printSchema
Key: SPARK-60695
URL: https://issues.apache.org/jira/browse/SPARK-60695
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: DB Tsai
Assignee: DB Tsai
Fix For: connect-rust-4.3.0
The Python DataFrame.printSchema binding currently calls
self.dataframe.show(0), which executes a limit(0).collect() action and prints
no actual schema. The optional level is ignored despite existing
tree_string_with_depth support. Reference Python Connect prints
schema.treeString(level). A full combined-draft benchmark still shows repeated
depth256 printSchema2.574ms vsPython0.00474ms because schema caching is
bypassed by an execute RPC. Correct the binding to resolve schema and print the
depth-aware tree string, using Python stdout capture semantics; validate exact
text, level behavior, errors, zero ExecutePlan calls and schema RPC counts. Use
paired baseline/candidate with identical SPARK-60677/PR120 schema cache for
incremental timing, preserving master applicability. The unchanged benchmark
only validates None, so independent output/protocol assertions are required.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
