[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31522: [SPARK-34399][SQL] Add commit duration to SQL tab's graph node.

[GitBox]

AngersZhuuuu commented on a change in pull request #31522:
URL: https://github.com/apache/spark/pull/31522#discussion_r674770724



##########
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/BasicWriteStatsTracker.scala
##########
@@ -230,7 +244,10 @@ object BasicWriteJobStatsTracker {
       NUM_FILES_KEY -> SQLMetrics.createMetric(sparkContext, "number of 
written files"),
       NUM_OUTPUT_BYTES_KEY -> SQLMetrics.createSizeMetric(sparkContext, 
"written output"),
       NUM_OUTPUT_ROWS_KEY -> SQLMetrics.createMetric(sparkContext, "number of 
output rows"),
-      NUM_PARTS_KEY -> SQLMetrics.createMetric(sparkContext, "number of 
dynamic part")
+      NUM_PARTS_KEY -> SQLMetrics.createMetric(sparkContext, "number of 
dynamic part"),
+      TASK_COMMIT_TIME ->
+        SQLMetrics.createTimingMetric(sparkContext, "time of committing the 
tasks"),

Review comment:
       Done

##########
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/BasicWriteStatsTracker.scala
##########
@@ -230,7 +244,10 @@ object BasicWriteJobStatsTracker {
       NUM_FILES_KEY -> SQLMetrics.createMetric(sparkContext, "number of 
written files"),
       NUM_OUTPUT_BYTES_KEY -> SQLMetrics.createSizeMetric(sparkContext, 
"written output"),
       NUM_OUTPUT_ROWS_KEY -> SQLMetrics.createMetric(sparkContext, "number of 
output rows"),
-      NUM_PARTS_KEY -> SQLMetrics.createMetric(sparkContext, "number of 
dynamic part")
+      NUM_PARTS_KEY -> SQLMetrics.createMetric(sparkContext, "number of 
dynamic part"),
+      TASK_COMMIT_TIME ->
+        SQLMetrics.createTimingMetric(sparkContext, "time of committing the 
tasks"),
+      JOB_COMMIT_TIME -> SQLMetrics.createTimingMetric(sparkContext, "time of 
committing the job")

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org