Hello, I tryed running GraphX Pregel for single source shortest path(very similar to example in https://spark.apache.org/docs/latest/graphx-programming-guide.html#pregel-api) using around 17K vertices and 36K edges. On a simple 8 vertex, 10 edge graph the Pregel algorithm works very well. When I run it using graph mentioned above on a single 8 core (16GB ram, SSD) it shows around 30K pending stages and runs around 600 stages for 30 minutes. In Spark configuration I have assigned 7 executors and each executor has 1 core. I even tryed changing these number but the result is the same. I understand that there are a lot of messages being passed around, but what could be the resaon for such a slow execution?
Thank you -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Reasons-for-Pregel-being-slow-tp22861.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
