[
https://issues.apache.org/jira/browse/SPARK-34265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-34265:
------------------------------------
Assignee: Apache Spark
> Add SQLMetrics instrumentation to Python UDF
> --------------------------------------------
>
> Key: SPARK-34265
> URL: https://issues.apache.org/jira/browse/SPARK-34265
> Project: Spark
> Issue Type: Improvement
> Components: PySpark, SQL
> Affects Versions: 3.1.1
> Reporter: Luca Canali
> Assignee: Apache Spark
> Priority: Minor
> Attachments: PythonSQLMetrics_Jira_Picture.png
>
>
> This proposes to add SQLMetrics instrumentation for Python UDF. This is aimed
> at improving monitoring and performance troubleshooting of Python code called
> by Spark, via UDF, Pandas UDF or with MapPartittions.
> The introduced metrics are exposed to the end users via the WebUI interface,
> in the SQL tab for execution steps related to Python UDF execution, namely
> BatchEvalPython, ArrowEvalPython, AggregateInPandas, FlaMapGroupsInPandas,
> FlatMapsCoGroupsInPandas, WindowsInPandas.
> See also the attached screenshot.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
