[
https://issues.apache.org/jira/browse/SPARK-56351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yicong Huang updated SPARK-56351:
---------------------------------
Description: Currently, the Arrow Python UDF documentation is a small
section (about 30 lines) embedded within the Pandas UDF documentation page
(arrow_pandas.rst). Given that Arrow Python UDFs are a distinct feature
(regular Python UDFs using Arrow for serialization, enabled via useArrow=True
or spark.sql.execution.pythonUDF.arrow.enabled), they deserve a dedicated
documentation page. (was: Currently, the Arrow Python UDF documentation is a
small section (about 30 lines) embedded within the Pandas UDF documentation
page (arrow_pandas.rst). Given that Arrow Python UDFs are a distinct feature
(regular Python UDFs using Arrow for serialization, enabled via useArrow=True
or spark.sql.execution.pythonUDF.arrow.enabled), they deserve a dedicated
documentation page.
Proposed changes:
1. Create a new arrow_python_udf.rst page with comprehensive documentation for
Arrow Python UDFs, including usage examples, type coercion details, and
configuration options.
2. Move the existing Arrow Python UDFs section from arrow_pandas.rst to the new
page, replacing it with a cross-reference link.
3. Add the new page to the tutorial/sql/index.rst table of contents.)
> Add dedicated documentation page for Arrow Python UDFs
> ------------------------------------------------------
>
> Key: SPARK-56351
> URL: https://issues.apache.org/jira/browse/SPARK-56351
> Project: Spark
> Issue Type: Improvement
> Components: Documentation, PySpark
> Affects Versions: 4.2.0
> Reporter: Yicong Huang
> Priority: Major
>
> Currently, the Arrow Python UDF documentation is a small section (about 30
> lines) embedded within the Pandas UDF documentation page (arrow_pandas.rst).
> Given that Arrow Python UDFs are a distinct feature (regular Python UDFs
> using Arrow for serialization, enabled via useArrow=True or
> spark.sql.execution.pythonUDF.arrow.enabled), they deserve a dedicated
> documentation page.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
