Lost of vertex when running a Bagel based program in non-local mode

Hi. I wrote a program to implement KMeans algorithm with Bagel. The program runs OK in local mode. While when I try to run it in the cluster, it seems that the Bagel.run method returns only part of vertices. That is, I used test data with 12 nodes to run the program and only 6 nodes was

Problem when trying to modify data generated with collect() method from RDD

Hi, everyone. I'm using scala to implement a connected component algorithm in Spark. And the question codes are as follows: 1type Graph = ListBuffer[Array[String]] 2type CCS = ListBuffer[Graph] 3val ccs_array:Array[CCS] = graphs_rdd.map{ graph = find_cc(graph)}.collect() 4var

Exception: Input path does not exist when using function objectFile

Hi, all. I wrote two programs: A.scala and B.scala. A.scala writes trained model to HDFS with: _wcount_rdd.saveAsObjectFile(save_path) I used the command hadoop fs -ls $save_path, and find a directory named $save_path: [root@gd39 spark-0.8.0-incubating] # hadoop fs -ls

Can not get the expected output when running the BroadcastTest example program.

Hi, all. I'm using spark-0.8.0-incubating. I tried the example BroadcastTest in local mode. ./run-example org.apache.spark.examples.BroadcastTest local 1 2/dev/null This works fine and get the result: Iteration 0 === 100 100 100 100 100 100 100 100 100