Hi, I have been trying to run some algorithms i have implemented using GraphX and Spark. I have been running these algorithms locally by starting a local spark instance through IntelliJ (in scala). However when I try to run them on a cluster with 10 machines I get
java.lang.ClassNotFoundException: org.centrality.spark.DegreeCentrality at java.net.URLClassLoader.findClass(URLClassLoader.java:381) when running spark-submit --master spark://sparkmaster:7077 --class org.centrality.spark.DegreeCentrality target/spark-1.0-SNAPSHOT.jar the jar is a fat jar which includes all the dependencies.. I also tried running each algorithm as a script which is being loaded in spark shell ("spark-shell -i DegreeCentrality.scala", I also include the System.exit(0) at the end.) . However when I ran the script it gets stuck while the graph is being created "val graph = GraphLoader.edgeListFile(sc, "hdfs://sparkmaster:7077/user/ilias/graphx/followers.txt"). I have been able to run a simple WordCount (by reading from an hdfs file as well) example but have still not been able to create a graph from a textfile. Thanks a lot... Ilias -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/GraphX-Running-on-a-Cluster-tp27236.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org