[ https://issues.apache.org/jira/browse/SPARK-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Rosen reassigned SPARK-4772: --------------------------------- Assignee: Josh Rosen > Accumulators leak memory, both temporarily and permanently > ---------------------------------------------------------- > > Key: SPARK-4772 > URL: https://issues.apache.org/jira/browse/SPARK-4772 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.0.0 > Reporter: Nathan Kronenfeld > Assignee: Josh Rosen > Labels: accumulators > Original Estimate: 24h > Remaining Estimate: 24h > > Accumulators.localAccums is cleared at the beginning of a task, and not at > the end. > This means that any locally accumulated values hang around until another task > is run on that thread. > If for some reason, the thread dies, said values hang around indefinitely. > This is really only a big issue with very large accumulators. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org