sample is not a member of org.apache.spark.streaming.dstream.DStream

2014-12-28 Thread Josh J
Hi, I'm trying to using sampling with Spark Streaming. I imported the following import org.apache.spark.{SparkConf, SparkContext} import org.apache.spark.SparkContext._ I then call sample val streamtoread = KafkaUtils.createStream(ssc, zkQuorum, group,

Re: sample is not a member of org.apache.spark.streaming.dstream.DStream

2014-12-28 Thread Sean Owen
The method you're referring to is a method of RDD, not DStream. If you want to do something with a sample of each RDD in the DStream, then call streamtoread.foreachRDD { rdd = val sampled = rdd.sample(...) ... } On Sun, Dec 28, 2014 at 10:44 PM, Josh J joshjd...@gmail.com wrote: Hi, I'm