Nishanth created SPARK-54223:
--------------------------------
Summary: Add Task Context Information to PythonRunner Logs
Key: SPARK-54223
URL: https://issues.apache.org/jira/browse/SPARK-54223
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 4.0.1, 3.5.1
Reporter: Nishanth
Currently, log messages in {{PythonRunner}} (used to execute Python UDFs) do
not include Spark Task Context information such as Task ID or Partition ID.
When debugging UDF performance issues or hangs, it’s difficult to trace which
Spark task or partition corresponds to the specific Python process that logged
the message.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
