[ https://issues.apache.org/jira/browse/BEAM-9065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17011875#comment-17011875 ]
Etienne Chauchot commented on BEAM-9065: ---------------------------------------- [~udim] If 2.18 is not released when the patch is ready I prefer that it is cherry picked to 2.18 Also I'm still investigating in the precise environment to reproduce this observed behavior so it might not come before 2.19 > Spark runner accumulates metrics (incorrectly) between runs > ----------------------------------------------------------- > > Key: BEAM-9065 > URL: https://issues.apache.org/jira/browse/BEAM-9065 > Project: Beam > Issue Type: Bug > Components: runner-spark > Reporter: Etienne Chauchot > Assignee: Etienne Chauchot > Priority: Major > Fix For: 2.18.0 > > Time Spent: 40m > Remaining Estimate: 0h > > When pipeline.run() is called, MetricsAccumulator (wrapper of > MetricsContainerStepMap spark accumulator) is initialized. Spark needs this > class to be a singleton for failover. The problem is that when several > pipelines are run inside the same JVM, the initialization of > MetricsAccumulator singleton does not reset the underlying spark accumulator > causing metrics to be accumulated between runs. -- This message was sent by Atlassian Jira (v8.3.4#803005)