Hello I have two graphRDDs one is property Graph and another one is connected Component graph like: * /var propGraph = Graph(vertexArray,edgeArray).cache()/*
with triplets: /((0,<http://data.linkedmdb.org/resource/actor/10140>),(14,null),<http://data.linkedmdb.org/resource/movie/producer_name>) ((1,null),(11,“Kviswanath”.),<http://data.linkedmdb.org/resource/movie/actor_name>) ((13,“ManiRatnam”.),(12,null),<http://data.linkedmdb.org/resource/movie/diecrtor_name>)/ and another one is connected components with graphRDD: */var cc = propGraph.connectedComponents().cache()/* with triplets: /((0,0),(14,0),<http://data.linkedmdb.org/resource/movie/producer_name>) ((1,1),(11,1),<http://data.linkedmdb.org/resource/movie/actor_name>) ((13,12),(12,12),<http://data.linkedmdb.org/resource/movie/diecrtor_name>)/ Now my question is I need to get triplets containing triplets from propGrap and cc graph like in following way after join: /((0,<http://data.linkedmdb.org/resource/actor/10140>),(14,null),<http://data.linkedmdb.org/resource/movie/producer_name>), *ccID1* ((1,null),(11,“Kviswanath”.),<http://data.linkedmdb.org/resource/movie/actor_name>) *ccID2* ((13,“ManiRatnam”.),(12,null),<http://data.linkedmdb.org/resource/movie/diecrtor_name>) *ccID2*/ In this graph ccIDs are 0,1,12 which I got from connectedComponents(). So in this case I am doing following stuff but not working: /val triplets = propGraph.joinVertices(cc.vertices)/ How to do this kind of join or joinVertices() FYI: My algorithm says: var cc = propGraph.connectedComponents() var triplest= propGraph.join(cc).triplets() //join original graph with so that each vertex know's it's connected component ID and then extract the triplets var rdfGraphs= triplets.mapPartition(func:genRDFTriplets).reduceByKey(func:concat) //Store the propery graph in RDD where each row has an ID and RDF graph (in the N-triplets format) FYI: genRDFTriplets function is /def genRDFTriplets(iter:Iterator[((Int,String),(Int,String),String,Int)]):Iterator[(Int,String)]={ var result = List[(Int,String)]() while (iter.hasNext) { val temp = iter.next() //println(s"tempRDF is ${temp}") result = result .:: (temp._4,(temp._1._2+" "+temp._3+" "+temp._2._2 +".\n").toString) } // println(s"resultRDF is ${result}") result.iterator }/ Please help me I am newbie in Spark. I love big data technologies. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Regarding-Join-between-two-graphs-tp25566.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
