Graphx Spark Accumulator

2015-12-13 Thread prasad223
Hi All, I'm new to Spark and Scala , I'm unable to create a Array of Integer Accumulators in Spark. val diameterAccumulator = sparkContext.accumulator(Array.fill(maxDegree)(0))(Array(maxDegree)[AccumulatorParam[Int]]) Can anyone give me a simple example of how to create an array of Int

Kryo Serialization in Spark

2015-12-07 Thread prasad223
Hi All, I'm unable to use Kryo serializer in my Spark program. I'm loading a graph from an edgelist file using GraphLoader and performing a BFS using pregel API. But I get the below mentioned error while I'm running. Can anybody tell me what is the right way to serialize a class in Spark and what

Spark GraphX default Storage Level

2015-12-06 Thread prasad223
Hi All, I was trying to generate a subset of graphs using GraphGenerators provided in GraphX Package. My code is as shown below def generateGraph(config: Config, sparkContext: SparkContext) = { if (config.graphType == "LogNormal") {