So, to answer my own n00b question, if case anyone ever needs it. You have to enable checkpointing (by ssc.checkpoint(hdfsPath)). Windowed operations need to be *checkpointed*, otherwise windows just won't work (and how could they).
On Tue, Oct 28, 2014 at 10:24 AM, diogo <di...@uken.com> wrote: > Hi there, I'm trying to use Window operations on streaming, but everything > I perform a windowed computation, I stop getting results. > > For example: > > val wordCounts = pairs.reduceByKey(_ + _) > wordCounts.print() > > Will print the output to the stdout on 'batch duration' interval. Now if I > replace it with: > > val wordCounts = pairs.reduceByKeyAndWindow(_+_, _-_, Seconds(4), > Seconds(2)) > wordCounts.print() > > It will never output. What did I get wrong? > > Thanks. >
