We are using Spark Streaming (Dstreams) with Kinesis batch interval as 10sec. For every random batch, processing time is taking very long. While checking logs, we found below log lines when ever we are getting spike in processing time:
<http://apache-spark-user-list.1001560.n3.nabble.com/file/t10925/Untitled.png> Processing Time: <http://apache-spark-user-list.1001560.n3.nabble.com/file/t10925/Untitled2.png> We dont have any code to manually remove RDDs. How to get rid of these? Thanks in Advance. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
