GitHub user bookling opened a pull request: https://github.com/apache/spark/pull/14940
[SPARK-17383][GRAPHX]LabelPropagation In the labelPropagation of graphx lib, node is initialized with a unique label and at every step each node adopts the label that most of its neighbors currently have, but ignore the label it currently have. I think it is unreasonable, because the labe a node had is also useful. When a node trend to has a stable label, this means there is an association between two iterations, so a node not only affected by its neighbors, but also its current label. so I change the code, and use both the label of its neighbors and itself. This iterative process densely connected groups of nodes form a consensus on a unique label to form communities. But the communities of the LabelPropagation often discontinuous. Because when the label that most of its neighbors currents have are many,e.g, node "0" has 6 neigbors labed {"1","1","2","2","3","3"},it maybe randomly select a label. in order to get a stable label of communities, and prevent the randomness, so I chose the max lable of node. you can test graph with Edges: {10L->11L,10L->12L, 11L->12L,11L->14L,12L->14L,13L->14L,13L->15L,13L->16L,15L->16L,15L->17L,16L->17L };or dandelion shape {1L->2L,2L->7L,2L->3L,2L->4L,2L->5L,2L->6L},etc. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bookling/spark master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14940.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14940 ---- commit 11bdab6bb042cd2102570c96db17279cf6ebbd92 Author: bookling <huth...@163.com> Date: 2016-08-30T17:51:43Z to solve "label shock " I have test the result, which is more reasonable Because the LabelPropagation often suffers "labe shock"ï¼ and the result of communities are often non-adjacent. I think the label of node is userful between adjacent supersteps, and the adjacent supersteps are relevant. commit bb875fef8f47ec99878d972f2c17b50123375a4c Author: bookling <huth...@163.com> Date: 2016-08-30T17:55:06Z to reduce "label shock " I have test the result, which is more reasonable Because the LabelPropagation often suffers "labe shock"ï¼ and the result of communities are often non-adjacent. I think the label of node is userful between adjacent supersteps, and the adjacent supersteps are relevant. commit 60e6f0ee2a3cdfb2b526a6d12887513f3aabed42 Author: XiaoSen Lee <huth...@sina.com> Date: 2016-09-02T18:57:29Z Improvement labelPropagation of garphx lib In the labelPropagation of graphx lib, node is initialized with a unique label and at every step each node adopts the label that most of its neighbors currently have, but ignore the label it currently have. I think it is unreasonable, because the labe a node had is also useful. When a node trend to has a stable label, this means there is an association between two iterations, so a node not only affected by its neighbors, but also its current label. so I change the code, and use both the label of its neighbors and itself. This iterative process densely connected groups of nodes form a consensus on a unique label to form communities. But the communities of the LabelPropagation often discontinuous. Because when the label that most of its neighbors currents have are many,e.g, node "0" has 6 neigbors labed {"1","1","2","2","3","3"},it maybe randomly select a label. in order to get a stable label of communities, and prevent the randomness, so I chose the max lable of node. you can test graph with Edges: {10L->11L,10L->12L, 11L->12L,11L->14L,12L->14L,13L->14L,13L->15L,13L->16L,15L->16L,15L->17L,16L->17L };or dandelion shape {1L->2L,2L->7L,2L->3L,2L->4L,2L->5L,2L->6L},etc. ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org