Haotian Sun created SPARK-62067:
-----------------------------------

             Summary: Report Python worker timing metrics in microseconds
                 Key: SPARK-62067
                 URL: https://issues.apache.org/jira/browse/SPARK-62067
             Project: Spark
          Issue Type: Sub-task
          Components: PySpark
    Affects Versions: 5.0.0
            Reporter: Haotian Sun


Python worker timing metrics are floored to whole milliseconds per task, so 
sub-millisecond work reports 0 and sums over many tasks undercount. Report the 
worker's timestamps and durations in microseconds, and expose the Python timing 
SQL metrics as microsecond sums whose keys and names include the unit.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to