The method you're referring to is a method of RDD, not DStream. If you
want to do something with a sample of each RDD in the DStream, then
call
streamtoread.foreachRDD { rdd =>
val sampled = rdd.sample(...)
...
}
On Sun, Dec 28, 2014 at 10:44 PM, Josh J <[email protected]> wrote:
> Hi,
>
> I'm trying to using sampling with Spark Streaming. I imported the following
>
> import org.apache.spark.{SparkConf, SparkContext}
> import org.apache.spark.SparkContext._
>
>
> I then call sample
>
>
> val streamtoread = KafkaUtils.createStream(ssc, zkQuorum, group,
> topicMap,StorageLevel.MEMORY_AND_DISK).map(_._2)
>
> streamtoread.sample(withReplacement = true, fraction = fraction)
>
>
> How do I use the sample() method with Spark Streaming?
>
>
> Thanks,
>
> Josh
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
