1. The for loop is executed in your driver program so will send each Pregel request serially to be executed on the cluster 2. Whilst caching/persisting may improve the runtime it shouldn't affect the memory bounds - if you ask to cache more than is available then cached RDDs will be dropped out of the cache. How are you running the program? via spark-submit - if so what parameters are you using?
----- Robin East Spark GraphX in Action Michael Malak and Robin East Manning Publications Co. http://www.manning.com/books/spark-graphx-in-action -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Failing-to-execute-Pregel-shortest-path-on-22k-nodes-tp25528p25531.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
