Re: Help with Spark Streaming

I guess, maybe you don’t need invoke reduceByKey() after mapToPair, because updateStateByKey had covered it. For your reference, here is a sample written by scala using text file stream instead of socket as below: object LocalStatefulWordCount extends App { val sparkConf = new

Using Spark Streaming to listen to HDFS directory and handle different files by file name

StreamContext provide the similar function to listen to the incoming files on HDFS? So that I can handle different files by file name on Spark Streaming. -- ZhangYi (张逸) Developer tel: 15023157626 blog: agiledon.github.com weibo: tw张逸 Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

Re: My talk on Spark: The Next Top (Compute) Model

Very Useful material. Currently, I am trying to persuade my client choose Spark instead of Hadoop MapReduce. Your slide give me more evidence to support my opinion. -- ZhangYi (张逸) Developer tel: 15023157626 blog: agiledon.github.com weibo: tw张逸 Sent with Sparrow (http