[
https://issues.apache.org/jira/browse/SPARK-12737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-12737.
-------------------------------
Resolution: Won't Fix
> Decrease the redundant activeIds sent to remote mirrors in
> "aggregateMessagesWithActiveSet"
> -------------------------------------------------------------------------------------------
>
> Key: SPARK-12737
> URL: https://issues.apache.org/jira/browse/SPARK-12737
> Project: Spark
> Issue Type: Improvement
> Components: GraphX
> Affects Versions: 1.5.2
> Reporter: qbwu
> Labels: newbie
>
> Hi, I found that it is not necessary to send the activeIds to all the mirrors
> of some master in the activeSetOpt which is passed to
> aggregateMessagesWithActiveSet. Because through the passed EdgeDirection, we
> can infer what kind of mirrors (classified by their position) will be checked
> isActive later, we can send the activeIds to the mirrors only at some
> position. In some cases even no activeId needs to be sent.
> I have implemented it, and did some tests using PageRank and
> ConnectedComponent, the shuffle size and the running time are decreased. But
> the number of runing turn is not changed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
