[ https://issues.apache.org/jira/browse/SPARK-25303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599566#comment-16599566 ]
Nikunj Bansal edited comment on SPARK-25303 at 9/3/18 6:41 AM: --------------------------------------------------------------- I have a potential fix for this and SPARK-25302 available. was (Author: nikunj): I have a potential fix for this and SPARK-25202 available. > A DStream that is checkpointed should allow its parent(s) to be removed and > not persisted > ----------------------------------------------------------------------------------------- > > Key: SPARK-25303 > URL: https://issues.apache.org/jira/browse/SPARK-25303 > Project: Spark > Issue Type: Bug > Components: DStreams > Affects Versions: 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, > 2.2.1, 2.2.2, 2.3.0, 2.3.1 > Reporter: Nikunj Bansal > Priority: Major > Labels: Streaming, streaming > > A checkpointed DStream is supposed to cut the lineage to its parent(s) such > that any persisted RDDs for the parent(s) are removed. However, combined with > the issue in SPARK-25302, they result in the Input Stream RDDs being > persisted a lot longer than they are actually required. > See also related bug SPARK-25302. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org