Github user nkronenfeld commented on a diff in the pull request:
https://github.com/apache/spark/pull/3570#discussion_r21572834
--- Diff: core/src/main/scala/org/apache/spark/Accumulators.scala ---
@@ -281,7 +282,9 @@ object AccumulatorParam {
private object Accumulators {
// TODO: Use soft references? => need to make readObject work properly
then
val originals = Map[Long, Accumulable[_, _]]()
- val localAccums = Map[Thread, Map[Long, Accumulable[_, _]]]()
+ val localAccums = new ThreadLocal[Map[Long, Accumulable[_, _]]]() {
+ override protected def initialValue() = Map[Long, Accumulable[_, _]]()
+ }
var lastId: Long = 0
--- End diff --
you mean the lastId?
That should only ever get used on the client - it's only called from the
constructor of an individual accumulator, and if someone is creating one of
those on a worker, they're already in trouble - so it shoudl be ok as is.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
