dadazheng created SPARK-21357: --------------------------------- Summary: FileInputDStream not remove out of date RDD Key: SPARK-21357 URL: https://issues.apache.org/jira/browse/SPARK-21357 Project: Spark Issue Type: Bug Components: DStreams Affects Versions: 2.1.0 Reporter: dadazheng
The method in org.apache.spark.streaming.dstream.FileInputDSteam.clearMetadata at line 166 will not remove out of date RDDs in generatedRDDs. This will cause leak of memory and OOM. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org