Re: counting degrees graphx

[daze5112]

Excellent thanks Ankur, looks like what im looking for  Only one problem the
line 

val dists = initDists.pregel[DistanceMap](Map())(vprog, sendMsg, mergeMsg)

produces an error 
 Job aborted: Task 268.0:5 had a not serializable result:
java.io.NotSerializableException: scala.collection.immutable.MapLike$$anon$2

any ideas?

thanks



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/counting-degrees-graphx-tp6370p6405.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: counting degrees graphx

[Ankur Dave]

Oh, looks like the Scala Map isn't serializable. I switched the code to use
java.util.HashMap, which should work.

Ankur http://www.ankurdave.com/


On Mon, May 26, 2014 at 3:21 PM, daze5112 david.zeel...@ato.gov.au wrote:

 Excellent thanks Ankur, looks like what im looking for  Only one problem
 the
 line

 val dists = initDists.pregel[DistanceMap](Map())(vprog, sendMsg, mergeMsg)

 produces an error
  Job aborted: Task 268.0:5 had a not serializable result:
 java.io.NotSerializableException:
 scala.collection.immutable.MapLike$$anon$2

 any ideas?

 thanks



 --
 View this message in context:
 http://apache-spark-user-list.1001560.n3.nabble.com/counting-degrees-graphx-tp6370p6405.html
 Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: counting degrees graphx

[Ankur Dave]

On closer inspection it looks like Map normally is serializable, and it's
just a bug in mapValues, so I changed to using the .map(identity)
workaround described in https://issues.scala-lang.org/browse/SI-7005.

Ankur http://www.ankurdave.com/

Re: counting degrees graphx

[Ankur Dave]

I'm not sure I understand what you're looking for. Could you provide some
more examples to clarify?

One interpretation is that you want to tag the source vertices in a graph
(those with zero indegree) and find for each vertex the set of sources that
lead to that vertex. For vertices 1-8 in the graph, this would give you
[{1}, {1}, {1}, {1}, {1, 6}, {7}, {7}].

Ankur http://www.ankurdave.com/

Re: counting degrees graphx

[ankurdave]

Sorry, I missed vertex 6 in that example. It should be [{1}, {1}, {1}, {1},
{1, 6}, {6}, {7}, {7}].



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/counting-degrees-graphx-tp6370p6378.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: counting degrees graphx

[dizzy5112]

yes thats correct I want the vertex set for each source vertice in the graph.
Which of course leads me on to my next question is to add a level to each of
these.  
http://apache-spark-user-list.1001560.n3.nabble.com/file/n6383/image1.jpg 

For example the image shows the in and out links of the graph and shows my
structure.  i want the list of vertices in 1,6 and 7.  i need to show that
vertice 1 has members 2,3,4,5 and vertice 6 has members 5 and vertice 7 has
members 8. 

Ideally i would also like to go one step further and identify which level
each vertice is on ie vertices 1,6 and 7 are level 0, verices 2,3,and and
are level 1, vertices 4 and 5 are level 2 (5 is also a level 1 when looked
at through vertice 6). 

Hope that is clearer.

cheers




--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/counting-degrees-graphx-tp6370p6383.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.