Re: GraphX: Get edges for a vertex
Hi Mas, I never actually worked with GraphX, but one idea: As far as I know, you can directly access the vertex and edge RDDs of your Graph object. Why not simply run a .filter() on the edge RDD to get all edges that originate from or end at your vertex? Regards, Jeff 2015-03-18 10:52 GMT+01:00 mas mas.ha...@gmail.com: Hi, Just to continue with the question. I need to find the edges of one particular vertex. However, (collectNeighbors/collectNeighborIds) provides the neighbors/neighborids for all the vertices of the graph. Any help in this regard will be highly appreciated. Thanks, -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/GraphX-Get-edges-for-a-vertex-tp18880p22115.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
GraphX: Get edges for a vertex
Hello, I'm attempting to implement a clustering algorithm on top of Pregel implementation in GraphX, however I'm hitting a wall. Ideally, I'd like to be able to get all edges for a specific vertex, since they factor into the calculation. My understanding was that sendMsg function would receive all relevant edges in participating vertices (all initially, declining as they converge and stop changing state), and I was planning to keep vertex edges associated to each vertex and propagate to other vertices that need to know about these edges. What I'm finding is that not all edges get iterated on by sendMsg before sending messages to vprog. Even if I try to keep track of edges, I don't account all of them, leading to incorrect results. The graph I'm testing on has edges between all nodes, one for each direction, and I'm using EdgeDirection.Both. Anyone seen something similar, and have some suggestions? Dan
Re: GraphX: Get edges for a vertex
Hi, I think that there are two solutions; 1. Invalid edges send Iterator.empty messages in sendMsg of the Pregel API. These messages make no effect on corresponding vertices. 2. Use GraphOps.(collectNeighbors/collectNeighborIds), not the Pregel API so as to handle active edge lists by yourself. I think that it is hard to handle edge active lists in Pregel APIs. Thought? Best regards, takeshi On Fri, Nov 14, 2014 at 7:32 AM, Daniil Osipov daniil.osi...@shazam.com wrote: Hello, I'm attempting to implement a clustering algorithm on top of Pregel implementation in GraphX, however I'm hitting a wall. Ideally, I'd like to be able to get all edges for a specific vertex, since they factor into the calculation. My understanding was that sendMsg function would receive all relevant edges in participating vertices (all initially, declining as they converge and stop changing state), and I was planning to keep vertex edges associated to each vertex and propagate to other vertices that need to know about these edges. What I'm finding is that not all edges get iterated on by sendMsg before sending messages to vprog. Even if I try to keep track of edges, I don't account all of them, leading to incorrect results. The graph I'm testing on has edges between all nodes, one for each direction, and I'm using EdgeDirection.Both. Anyone seen something similar, and have some suggestions? Dan
