[
https://issues.apache.org/jira/browse/SPARK-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davies Liu updated SPARK-3465:
------------------------------
Description:
In local mode, after onExecutorMetricsUpdate(), t.taskMetrics will be the same
object with that in TaskContext (because there is no serialization for
MetricsUpdate in local mode), then all the upcoming changes in metrics will be
lost, because updateAggregateMetrics() only counts the difference in these two.
This bug was introduced in https://issues.apache.org/jira/browse/SPARK-2099, cc
@sandy rayza
was:
In local mode, after onExecutorMetricsUpdate(), t.taskMetrics will be the same
object with that in TaskContext (because there is no serialization for
MetricsUpdate in local mode), then all the upcoming changes in metrics will be
lost, because updateAggregateMetrics() only counts the difference in these two.
This bug was introduced in https://issues.apache.org/jira/browse/SPARK-2099.
> Task metrics are not aggregated correctly in local mode
> -------------------------------------------------------
>
> Key: SPARK-3465
> URL: https://issues.apache.org/jira/browse/SPARK-3465
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.1.0
> Reporter: Davies Liu
> Assignee: Davies Liu
> Priority: Blocker
>
> In local mode, after onExecutorMetricsUpdate(), t.taskMetrics will be the
> same object with that in TaskContext (because there is no serialization for
> MetricsUpdate in local mode), then all the upcoming changes in metrics will
> be lost, because updateAggregateMetrics() only counts the difference in these
> two.
> This bug was introduced in https://issues.apache.org/jira/browse/SPARK-2099,
> cc @sandy rayza
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
