[
https://issues.apache.org/jira/browse/SPARK-39452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
PEIYUAN SUN updated SPARK-39452:
--------------------------------
Description:
Currently, the
*[EdgePartition1D|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/PartitionStrategy.scala#L101]*
use only the source vertex ID.
If doing topological like algorithm on graph, directly partition based on
destination vertex ID would improve the performance significantly.
The proposal here is either adding a new case object like *EdgePartition1DDst*
or update the *EdgePartition1D* such that user can configure the src or dst
directly.
was:
Currently, the
*[EdgePartition1D|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/PartitionStrategy.scala#L101]*
use only the source vertex ID.
If doing topological like algorithm on graph, directly partition based on
destination vertex ID would improve the performance significantly.
The proposal here is either adding a new case object like *EdgePartition1DDst*
or ** update the ** *EdgePartition1D* such that user can configure the src or
dst.{*}{*}
> Extend EdgePartition1D with Destination based Strategy
> ------------------------------------------------------
>
> Key: SPARK-39452
> URL: https://issues.apache.org/jira/browse/SPARK-39452
> Project: Spark
> Issue Type: New Feature
> Components: GraphX
> Affects Versions: 3.1.2, 3.2.1
> Reporter: PEIYUAN SUN
> Priority: Minor
> Labels: graph, partition
> Fix For: 3.3.0, 3.3.1
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> Currently, the
> *[EdgePartition1D|https://github.com/apache/spark/blob/master/graphx/src/main/scala/org/apache/spark/graphx/PartitionStrategy.scala#L101]*
> use only the source vertex ID.
> If doing topological like algorithm on graph, directly partition based on
> destination vertex ID would improve the performance significantly.
> The proposal here is either adding a new case object like
> *EdgePartition1DDst* or update the *EdgePartition1D* such that user can
> configure the src or dst directly.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
