[ https://issues.apache.org/jira/browse/SPARK-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guoqiang Li resolved SPARK-3625. -------------------------------- Resolution: Won't Fix > In some cases, the RDD.checkpoint does not work > ----------------------------------------------- > > Key: SPARK-3625 > URL: https://issues.apache.org/jira/browse/SPARK-3625 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 1.0.2, 1.1.0 > Reporter: Guoqiang Li > Assignee: Guoqiang Li > > The reproduce code: > {code} > sc.setCheckpointDir(checkpointDir) > val c = sc.parallelize((1 to 1000)).map(_ + 1) > c.count > val dep = c.dependencies.head.rdd > c.checkpoint() > c.count > assert(dep != c.dependencies.head.rdd) > {code} > This limit is too strict , This makes it difficult to implement SPARK-3623 . -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org