[jira] [Commented] (SPARK-7894) Graph Union Operator
[ https://issues.apache.org/jira/browse/SPARK-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605739#comment-14605739 ] Arnab commented on SPARK-7894: -- Short description of changes: Introduced union functionality in EdgeRDD, VertexRDD and Graph classes (there is no union functionality in EdgeRdd and VertexRdd directly as pointed out by shijinkui) Added code for merging partitions in Edge and Vertex partitions Added test case for graph union (as in Jira) , also unit tests for union of edges and vertices > Graph Union Operator > > > Key: SPARK-7894 > URL: https://issues.apache.org/jira/browse/SPARK-7894 > Project: Spark > Issue Type: Sub-task > Components: GraphX >Reporter: Andy Huang > Labels: graph, union > Attachments: union_operator.png > > > This operator aims to union two graphs and generate a new graph directly. The > union of two graphs is the union of their vertex sets and their edge > families.Vertexes and edges which are included in either graph will be part > of the new graph. > bq. G ∪ H = (VG ∪ VH, EG ∪ EH). > The below image shows a union of graph G and graph H > !union_operator.png|width=600px,align=center! > A Simple interface would be： > bq. def union[VD: ClassTag, ED: ClassTag](other: Graph[VD, ED]): Graph[VD, ED] > However, inevitably vertexes and edges overlapping will happen between > borders of graphs. For vertex, it's quite nature to just make a union and > remove those duplicate ones. But for edges, a mergeEdges function seems to be > more reasonable. > bq. def union[VD: ClassTag, ED: ClassTag](other: Graph[VD, ED], mergeEdges: > (ED, ED) => ED): Graph[VD, ED] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7894) Graph Union Operator
[ https://issues.apache.org/jira/browse/SPARK-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14604830#comment-14604830 ] Apache Spark commented on SPARK-7894: - User 'arnabguin' has created a pull request for this issue: https://github.com/apache/spark/pull/7074 > Graph Union Operator > > > Key: SPARK-7894 > URL: https://issues.apache.org/jira/browse/SPARK-7894 > Project: Spark > Issue Type: Sub-task > Components: GraphX >Reporter: Andy Huang > Labels: graph, union > Attachments: union_operator.png > > > This operator aims to union two graphs and generate a new graph directly. The > union of two graphs is the union of their vertex sets and their edge > families.Vertexes and edges which are included in either graph will be part > of the new graph. > bq. G ∪ H = (VG ∪ VH, EG ∪ EH). > The below image shows a union of graph G and graph H > !union_operator.png|width=600px,align=center! > A Simple interface would be： > bq. def union[VD: ClassTag, ED: ClassTag](other: Graph[VD, ED]): Graph[VD, ED] > However, inevitably vertexes and edges overlapping will happen between > borders of graphs. For vertex, it's quite nature to just make a union and > remove those duplicate ones. But for edges, a mergeEdges function seems to be > more reasonable. > bq. def union[VD: ClassTag, ED: ClassTag](other: Graph[VD, ED], mergeEdges: > (ED, ED) => ED): Graph[VD, ED] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7894) Graph Union Operator
[ https://issues.apache.org/jira/browse/SPARK-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581204#comment-14581204 ] shijinkui commented on SPARK-7894: -- hey [~ilganeli] there are no union in EdgeRDD and VertexRDD directly. The `union` in `RDD` can't merge same element in one operation. in the uion of Graph, we need merge same Vertex Attribute and same Edge attribute > Graph Union Operator > > > Key: SPARK-7894 > URL: https://issues.apache.org/jira/browse/SPARK-7894 > Project: Spark > Issue Type: Sub-task > Components: GraphX >Reporter: Andy Huang > Labels: graph, union > Attachments: union_operator.png > > > This operator aims to union two graphs and generate a new graph directly. The > union of two graphs is the union of their vertex sets and their edge > families.Vertexes and edges which are included in either graph will be part > of the new graph. > bq. G ∪ H = (VG ∪ VH, EG ∪ EH). > The below image shows a union of graph G and graph H > !union_operator.png|width=600px,align=center! > A Simple interface would be： > bq. def union[VD: ClassTag, ED: ClassTag](other: Graph[VD, ED]): Graph[VD, ED] > However, inevitably vertexes and edges overlapping will happen between > borders of graphs. For vertex, it's quite nature to just make a union and > remove those duplicate ones. But for edges, a mergeEdges function seems to be > more reasonable. > bq. def union[VD: ClassTag, ED: ClassTag](other: Graph[VD, ED], mergeEdges: > (ED, ED) => ED): Graph[VD, ED] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7894) Graph Union Operator
[ https://issues.apache.org/jira/browse/SPARK-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14579283#comment-14579283 ] Ilya Ganelin commented on SPARK-7894: - How is this functionality different from the existing ```union``` functions within ```VertexRDD``` and ```EdgeRDD``` ? > Graph Union Operator > > > Key: SPARK-7894 > URL: https://issues.apache.org/jira/browse/SPARK-7894 > Project: Spark > Issue Type: Sub-task > Components: GraphX >Reporter: Andy Huang > Labels: graph, union > Attachments: union_operator.png > > > This operator aims to union two graphs and generate a new graph directly. The > union of two graphs is the union of their vertex sets and their edge > families.Vertexes and edges which are included in either graph will be part > of the new graph. > bq. G ∪ H = (VG ∪ VH, EG ∪ EH). > The below image shows a union of graph G and graph H > !union_operator.png|width=600px,align=center! > A Simple interface would be： > bq. def union[VD: ClassTag, ED: ClassTag](other: Graph[VD, ED]): Graph[VD, ED] > However, inevitably vertexes and edges overlapping will happen between > borders of graphs. For vertex, it's quite nature to just make a union and > remove those duplicate ones. But for edges, a mergeEdges function seems to be > more reasonable. > bq. def union[VD: ClassTag, ED: ClassTag](other: Graph[VD, ED], mergeEdges: > (ED, ED) => ED): Graph[VD, ED] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
