[ https://issues.apache.org/jira/browse/SPARK-27240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Cutler resolved SPARK-27240. ---------------------------------- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24177 https://github.com/apache/spark/pull/24177 > Use pandas DataFrame for struct type argument in Scalar Pandas UDF. > ------------------------------------------------------------------- > > Key: SPARK-27240 > URL: https://issues.apache.org/jira/browse/SPARK-27240 > Project: Spark > Issue Type: Improvement > Components: PySpark > Affects Versions: 2.4.0 > Reporter: Takuya Ueshin > Assignee: Takuya Ueshin > Priority: Major > Fix For: 3.0.0 > > > Now that we support returning pandas DataFrame for struct type in Scalar > Pandas UDF. > If we chain another Pandas UDF after the Scalar Pandas UDF returning pandas > DataFrame, the argument of the chained UDF will be pandas DataFrame, but > currently we don't support pandas DataFrame as an argument of Scalar Pandas > UDF. That means there is an inconsistency between the chained UDF and the > single UDF. > We should support taking pandas DataFrame for struct type argument in Scala > Pandas UDF to be consistent. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org