Re: counting degrees graphx

2014-05-26 Thread daze5112
that is brilliant Ankur, thanks so much for your help! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/counting-degrees-graphx-tp6370p6409.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: counting degrees graphx

2014-05-26 Thread Ankur Dave
On closer inspection it looks like Map normally is serializable, and it's just a bug in mapValues, so I changed to using the .map(identity) workaround described in https://issues.scala-lang.org/browse/SI-7005. Ankur

Re: counting degrees graphx

2014-05-26 Thread Ankur Dave
Oh, looks like the Scala Map isn't serializable. I switched the code to use java.util.HashMap, which should work. Ankur On Mon, May 26, 2014 at 3:21 PM, daze5112 wrote: > Excellent thanks Ankur, looks like what im looking for Only one problem > the > line > > val d

Re: counting degrees graphx

2014-05-26 Thread daze5112
Excellent thanks Ankur, looks like what im looking for Only one problem the line val dists = initDists.pregel[DistanceMap](Map())(vprog, sendMsg, mergeMsg) produces an error Job aborted: Task 268.0:5 had a not serializable result: java.io.NotSerializableException: scala.collection.immutable.M

Re: counting degrees graphx

2014-05-26 Thread Ankur Dave
OK, here's some code to do this using the GraphX Pregel API: https://gist.github.com/ankurdave/4a17596669b36be06100 Ankur

Re: counting degrees graphx

2014-05-25 Thread dizzy5112
sorry directions of edges in this image -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/counting-degrees-graphx-tp6370p6384.html Sent from the Apache Spark User List mailing list ar

Re: counting degrees graphx

2014-05-25 Thread dizzy5112
yes thats correct I want the vertex set for each source vertice in the graph. Which of course leads me on to my next question is to add a level to each of these. For example the image shows the in and out links of the g

Re: counting degrees graphx

2014-05-25 Thread ankurdave
Sorry, I missed vertex 6 in that example. It should be [{1}, {1}, {1}, {1}, {1, 6}, {6}, {7}, {7}]. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/counting-degrees-graphx-tp6370p6378.html Sent from the Apache Spark User List mailing list archive at Nabble.c

Re: counting degrees graphx

2014-05-25 Thread Ankur Dave
I'm not sure I understand what you're looking for. Could you provide some more examples to clarify? One interpretation is that you want to tag the source vertices in a graph (those with zero indegree) and find for each vertex the set of sources that lead to that vertex. For vertices 1-8 in the gra