[
https://issues.apache.org/jira/browse/SPARK-60695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60695:
-----------------------------------
Labels: pull-request-available (was: )
> Print schema metadata instead of executing rows in Spark Connect Rust
> printSchema
> ---------------------------------------------------------------------------------
>
> Key: SPARK-60695
> URL: https://issues.apache.org/jira/browse/SPARK-60695
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: DB Tsai
> Assignee: DB Tsai
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> The Python DataFrame.printSchema binding currently calls
> self.dataframe.show(0), which executes a limit(0).collect() action and prints
> no actual schema. The optional level is ignored despite existing
> tree_string_with_depth support. Reference Python Connect prints
> schema.treeString(level). A full combined-draft benchmark still shows
> repeated depth256 printSchema2.574ms vsPython0.00474ms because schema caching
> is bypassed by an execute RPC. Correct the binding to resolve schema and
> print the depth-aware tree string, using Python stdout capture semantics;
> validate exact text, level behavior, errors, zero ExecutePlan calls and
> schema RPC counts. Use paired baseline/candidate with identical
> SPARK-60677/PR120 schema cache for incremental timing, preserving master
> applicability. The unchanged benchmark only validates None, so independent
> output/protocol assertions are required.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
