It's much easier than all this. Spark Streaming gives you a DStream of
RDDs. You want the count for each RDD. DStream.count() gives you
exactly that: a DStream of Longs which are the counts of events in
each mini batch.
On Tue, Sep 30, 2014 at 8:42 PM, Andy Davidson
a...@santacruzintegration.com
: Sean Owen so...@cloudera.com
Date: Wednesday, October 1, 2014 at 2:32 AM
To: Andrew Davidson a...@santacruzintegration.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: how to get actual count from as long from JavaDStream ?
It's much easier than all this. Spark Streaming gives you
Hi
I have a simple streaming app. All I want to do is figure out how many lines
I have received in the current mini batch. If numLines was a JavaRDD I could
simply call count(). How do you do something similar in Streaming?
Here is my psudo code
JavaDStreamString msg =
Hi Andy
I'm new to Spark and have been working with Scala not Java but I see
there's a dstream() method to convert from JavaDStream to DStream. Then within
DStream
http://people.apache.org/~pwendell/spark-1.1.0-rc4-docs/api/java/org/apache/spark/streaming/dstream/DStream.html
there is a
: user@spark.apache.org user@spark.apache.org
Subject: Re: how to get actual count from as long from JavaDStream ?
Hi Andy
I'm new to Spark and have been working with Scala not Java but I see there's a
dstream() method to convert from JavaDStream to DStream. Then within DStream
http
