Guoqiang Li created SPARK-6697: ---------------------------------- Summary: PeriodicGraphCheckpointer is not clear Edges. Key: SPARK-6697 URL: https://issues.apache.org/jira/browse/SPARK-6697 Project: Spark Issue Type: Bug Components: GraphX, MLlib Affects Versions: 1.3.0 Reporter: Guoqiang Li
When I run this [branch(lrGraphxSGD)| https://github.com/witgo/spark/tree/lrGraphxSGD] . PeriodicGraphCheckpointer only clear the vertices. {code} def run(iterations: Int): Unit = { for (iter <- 1 to iterations) { logInfo(s"Start train (Iteration $iter/$iterations)") val margin = forward() margin.setName(s"margin-$iter").persist(storageLevel) println(s"train (Iteration $iter/$iterations) cost : ${error(margin)}") var gradient = backward(margin) gradient = updateDeltaSum(gradient, iter) dataSet = updateWeight(gradient, iter) dataSet.vertices.setName(s"vertices-$iter") dataSet.edges.setName(s"edges-$iter") dataSet.persist(storageLevel) graphCheckpointer.updateGraph(dataSet) margin.unpersist(blocking = false) gradient.unpersist(blocking = false) logInfo(s"End train (Iteration $iter/$iterations)") innerIter += 1 } graphCheckpointer.deleteAllCheckpoints() } {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org