http://spark.apache.org/docs/latest/streaming-programming-guide.html#deployi ng-applications
Gives a brief discussion about max rate and back pressure Its not clear to me what will happen. I use an unreliable reciever. Let say me app is running and process time is less then window length. Happy happy After a while there is a burst of data and processing time now exceeds windows length What happens? Am I going to loose data.(I have implemented checkpoints) Okay time goes by data rate returns to normal processing time is < window length again. Does the system catch up? Or is data lost? Kind regards Andy
