RE: GraphX PageRank keeps 3 copies of graph in memory

2015-10-07 Thread Ulanov, Alexander
Hi Ankur, Could you help with explanation of the problem below? Best regards, Alexander From: Ulanov, Alexander Sent: Friday, October 02, 2015 11:39 AM To: 'Robin East' Cc: dev@spark.apache.org Subject: RE: GraphX PageRank keeps 3 copies of graph in memory Hi Robin, Sounds interesting. I am

RE: GraphX PageRank keeps 3 copies of graph in memory

2015-10-02 Thread Ulanov, Alexander
] Sent: Friday, October 02, 2015 12:27 AM To: Ulanov, Alexander Cc: dev@spark.apache.org Subject: Re: GraphX PageRank keeps 3 copies of graph in memory Alexander, I’ve just run the benchmark and only end up with 2 sets of RDDs in the Storage tab. This is on 1.5.0, what version are you using? Robin

GraphX PageRank keeps 3 copies of graph in memory

2015-09-30 Thread Ulanov, Alexander
Dear Spark developers, I would like to understand GraphX caching behavior with regards to PageRank in Spark, in particular, the following implementation of PageRank: https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/lib/PageRank.scala On each iteration