Nikunj Bansal created SPARK-25302:
-------------------------------------
Summary: ReducedWindowedDStream not using checkpoints for reduced
RDDs
Key: SPARK-25302
URL: https://issues.apache.org/jira/browse/SPARK-25302
Project: Spark
Issue Type: Bug
Components: DStreams
Affects Versions: 2.3.1, 2.3.0, 2.2.2, 2.2.1, 2.2.0, 2.1.3, 2.1.2, 2.1.1,
2.1.0, 2.0.2, 2.0.1, 2.0.0
Reporter: Nikunj Bansal
When using reduceByKeyAndWindow() using inverse reduce function, it eventually
creates a ReducedWindowedDStream. This class creates a reducedDStream but only
persists it and does not checkpoint it. The result is that it ends up using
cached RDDs and does not cut lineage to the input DStream resulting in
eventually caching the input RDDs for much longer than they are needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
