Re: Working with slides. How do I know how many times a RDD has been processed?

2015-05-19 Thread Guillermo Ortiz
I tried to insert an flag in the RDD, so I could set in the last position a counter, when the counter gets X, I could do something. But in each slide comes the original RDD although I modificated it. I did this code to check if this is possible but it doesn't work. val rdd1WithFlag = rdd1.map {

Working with slides. How do I know how many times a RDD has been processed?

2015-05-18 Thread Guillermo Ortiz
Hi, I have two streaming RDD1 and RDD2 and want to cogroup them. Data don't come in the same time and sometimes they could come with some delay. When I get all data I want to insert in MongoDB. For example, imagine that I get: RDD1 -- T 0 RDD2 --T 0.5 I do cogroup between them but I couldn't