[ https://issues.apache.org/jira/browse/SPARK-28332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17002148#comment-17002148 ]
EdisonWang commented on SPARK-28332: ------------------------------------ I've taken it [~cloud_fan] > SQLMetric wrong initValue > -------------------------- > > Key: SPARK-28332 > URL: https://issues.apache.org/jira/browse/SPARK-28332 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.0 > Reporter: Song Jun > Priority: Minor > Fix For: 3.0.0 > > > Currently SQLMetrics.createSizeMetric create a SQLMetric with initValue set > to -1. > If there is a ShuffleMapStage with lots of Tasks which read 0 bytes data, > these tasks will send the metric(the metric value still be the initValue with > -1) to Driver, then Driver do metric merge for this Stage in > DAGScheduler.updateAccumulators, this will cause the merged metric value of > this Stage set to be a negative value. > This is incorrect, we should set the initValue to 0 . > Another same case in SQLMetrics.createTimingMetric. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org