I see Javadoc Style documentation but nothing that looks like a code sample I tried the following before asking
public static class LongAccumulableParam implements AccumulableParam<Long,Long>,Serializable { @Override public Long addAccumulator(final Long r, final Long t) { return r + t; } @Override public Long addInPlace(final Long r1, final Long r2) { return r1 + r2; } @Override public Long zero(final Long initialValue) { return 0L; } } sparkContext.currentContext.accumulator(0L, "myAccumulator, new LongAccumulableParam ()); does not compile which is why I ask for code samples The Javados for accumulator says *accumulator <https://spark.apache.org/docs/1.0.0/api/java/org/apache/spark/api/java/JavaSparkContext.html#accumulator(T, org.apache.spark.AccumulatorParam)>*(T initialValue, AccumulatorParam <https://spark.apache.org/docs/1.0.0/api/java/org/apache/spark/AccumulatorParam.html> <T> accumulatorParam) Create an Accumulator <https://spark.apache.org/docs/1.0.0/api/java/org/apache/spark/Accumulator.html> variable of a given type, which tasks can "add" values to using the add method. which is a LONG way from a working code sample On Wed, Nov 12, 2014 at 8:18 PM, Sean Owen <so...@cloudera.com> wrote: > It's the exact same API you've already found, and it's documented: > > http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.AccumulatorParam > > JavaSparkContext has helper methods for int and double but not long. You > can just make your own little implementation of AccumulatorParam<Long> > right? ... which would be nice to add to JavaSparkContext. > > On Wed, Nov 12, 2014 at 11:05 PM, Steve Lewis <lordjoe2...@gmail.com> > wrote: > >> JavaSparkContext currentContext = ...; >> Accumulator<Integer> accumulator = currentContext.accumulator(0, >> "MyAccumulator"); >> >> will create an Accumulator of Integers. For many large Data problems >> Integer is too small and Long is a better type. >> I see a call like the following >> >> AccumulatorParam<Long> param = ?? how do I get one of these; >> // no compiler is unhappy with this call >> Accumulator<Long> accumulatorLong = >> currentContext.accumulator(new Long(0), acc ,param); >> >> but NONE - ZERO documentation on its use, how to get a >> AccumulatorParam<Long> or how to turn one into an Accumulator. >> Any ideas >> >> >> > -- Steven M. Lewis PhD 4221 105th Ave NE Kirkland, WA 98033 206-384-1340 (cell) Skype lordjoe_com