[
https://issues.apache.org/jira/browse/SPARK-55206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-55206:
-----------------------------------
Labels: pull-request-available (was: )
> Support basic single parameter addition Python UDF transpilation
> ----------------------------------------------------------------
>
> Key: SPARK-55206
> URL: https://issues.apache.org/jira/browse/SPARK-55206
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: Holden Karau
> Priority: Major
> Labels: pull-request-available
>
> For the sake of having code land in testable/reviewable chunks I think the
> first chunk we should do should be the bare bones for simple numerical PyUDF
> transpilation. To accomplish that we'll need to include the following:
> * Parameter for enabing transpilation (clearly marked as experimental)
> * AST extraction
> * AST transpilation
> * UDF node extended to support both transpiled and "original" UDF code
> (possible with an explanation string for why no transpilation)
> * UDF node printout in the query plan should be clear if transpiled or not
> * Property-based tests for transpiled UDFS ensuring correctness
> * Type promotion equivalence (does not need to be complete given numeric
> only)
> My theory here is that by putting all of this together in one first PR, we
> both establish a solid framework for future work and ensure the framework we
> land is actually usable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
