Spark Streaming and Windows, it always counts the logs during all the windows. Why?

2014-12-26 Thread Guillermo Ortiz
I'm trying to make some operation with windows and intervals. I get data every15 seconds, and want to have a windows of 60 seconds with batch intervals of 15 seconds. I''m injecting data with ncat. if I inject 3 logs in the same interval I get into the do something each 15 secods during one

Re: Spark Streaming and Windows, it always counts the logs during all the windows. Why?

2014-12-26 Thread Guillermo Ortiz
I'm trying to understand why it's not working and I typed some println to check what the code was executing.. def ruleSqlInjection(lines: ReceiverInputDStream[String]) = { println(1); //Just one time, when I start the program val filterSql = lines.filter(line =

Re: Spark Streaming and Windows, it always counts the logs during all the windows. Why?

2014-12-26 Thread Guillermo Ortiz
Oh, I didn't understand what I was doing, my fault (too much parties these xmas). Thought windows works in another weird way. Sorry for the questions.. 2014-12-26 13:42 GMT+01:00 Guillermo Ortiz konstt2...@gmail.com: I'm trying to understand why it's not working and I typed some println to