[
https://issues.apache.org/jira/browse/SPARK-44479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takuya Ueshin resolved SPARK-44479.
-----------------------------------
Assignee: Takuya Ueshin
Resolution: Fixed
Issue resolved by pull request 42161
https://github.com/apache/spark/pull/42161
> Support Python UDTFs with empty schema
> --------------------------------------
>
> Key: SPARK-44479
> URL: https://issues.apache.org/jira/browse/SPARK-44479
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
> Affects Versions: 3.5.0
> Reporter: Takuya Ueshin
> Assignee: Takuya Ueshin
> Priority: Major
>
> Support UDTFs with empty schema, for example:
> {code:python}
> >>> class TestUDTF:
> ... def eval(self):
> ... yield tuple()
> {code}
> Currently it fails with `useArrow=True`:
> {code:python}
> >>> udtf(TestUDTF, returnType=StructType())().collect()
> Traceback (most recent call last):
> ...
> ValueError: not enough values to unpack (expected 2, got 0)
> {code}
> whereas without Arrow:
> {code:python}
> >>> udtf(TestUDTF, returnType=StructType(), useArrow=False)().collect()
> [Row()]
> {code}
> Otherwise, we should raise an error without Arrow, too, to be consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
