On Spark 1.2:

I am trying to capture # records read from a kafka topic:

val inRecords = ssc.sparkContext.accumulator(0, "InRecords")

..

kInStreams.foreach( k =>
                {

                 k.foreachRDD ( rdd =>  inRecords += rdd.count().toInt  )
                 inRecords.value


Question is how do I get the accumulator to show up in the UI? I tried
"inRecords.value" but that didn't help. Pretty sure it isn't showing up in
Stage metrics.

What's the trick here? collect?

Thanks,

Tim

Reply via email to