After checkpoint(), please call count(). This is similar to cache(), the RDD is only marked as to be checked with checkpoint(). -Xiangrui
On Sun, Jun 22, 2014 at 3:14 PM, dash <b...@nd.edu> wrote: > Hi, > > I'm doing iterative computing now, and due to lineage chain, we need to > checkpoint the RDD in order to cut off lineage and prevent StackOverflow > error. > > The following code still having StackOverflowError, I checked > `isCheckpointed` and the result is true. Also, I write a function to count > the lineage, but the lineage is not big. Any idea about that? Please give me > some hit so I can dig into the source code and try to fix it. > > > > > Best, > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/Checkpointed-RDD-still-causing-StackOverflow-tp7066.html > Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
