Re: AccumulatorV2 += operator

2016-08-03 Thread Holden Karau
Ah in that case the programming guides text is still talking about the deprecated accumulator API despite having an updated code sample (the way it suggests making an accumulator is also deprecated). I think the fix is updating the programming guide rather than adding += to the API. On Wednesday,

Re: AccumulatorV2 += operator

2016-08-03 Thread Bryan Cutler
No, I was referring to the programming guide section on accumulators, it says " Tasks running on a cluster can then add to it using the add method or the += operator (in Scala and Python)." On Aug 2, 2016 2:52 PM, "Holden Karau" wrote: > I believe it was intentional with the idea that it would b

Re: AccumulatorV2 += operator

2016-08-02 Thread Holden Karau
I believe it was intentional with the idea that it would be more unified between Java and Scala APIs. If your talking about the javadoc mention in https://github.com/apache/spark/pull/14466/files - I believe the += is meant to refer to what the internal implementation of the add function can be for

AccumulatorV2 += operator

2016-08-02 Thread Bryan Cutler
It seems like the += operator is missing from the new accumulator API, although the docs still make reference to it. Anyone know if it was intentionally not put in? I'm happy to do a PR for it or update the docs to just use the add() method, just want to check if there was some reason first. Bry