[
https://issues.apache.org/jira/browse/SPARK-60697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60697:
-----------------------------------
Labels: pull-request-available (was: )
> Avoid materializing DecimalTuple for exact Python Decimal input conversion
> --------------------------------------------------------------------------
>
> Key: SPARK-60697
> URL: https://issues.apache.org/jira/browse/SPARK-60697
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: DB Tsai
> Assignee: DB Tsai
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> Client-only combined benchmark shows decimal pandas input23.80ms
> vsPython5.84ms and tuple21.88ms vs8.89ms for10000x3 values. cProfile of10
> constructions/300000cells spends0.043s of0.273s in DecimalTuple namedtuple
> construction from decimal_to_value.as_tuple (about16percent). Binding already
> obtains str(value); exact built-in Decimal scale can potentially be derived
> from that canonical representation without materializing every coefficient
> digit. Evaluate a narrow optimization preserving exact value,
> inferred/explicit scale including trailing zeros, exponent notation, special
> values, Decimal subclasses and exceptions. Keep subclass/as_tuple overrides
> on existing fallback and do not bundle scientific parser changes tracked
> SPARK-60696. Require separate frozen baseline/candidate client measurements,
> unchanged harness and independent IPC/schema correctness including inferred
> inputs. No JVM.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
