[jira] [Commented] (SPARK-20839) Incorrect Dynamic PageRank calculation
[
https://issues.apache.org/jira/browse/SPARK-20839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413032#comment-16413032
]
Ziyan LIANG commented on SPARK-20839:
-
Got the same confusion, is there some documents or reference about this design
because it's quite different from the original theory description. Thanks.
> Incorrect Dynamic PageRank calculation
> --
>
> Key: SPARK-20839
> URL: https://issues.apache.org/jira/browse/SPARK-20839
> Project: Spark
> Issue Type: Bug
> Components: GraphX
>Affects Versions: 2.1.1
>Reporter: BahaaEddin AlAila
>Priority: Major
>
> Correct me if I am wrong
> I think there are three places where the pagerank calculation is incorrect
> 1st) in the VertexProgram (line 318 of PageRank.scala in spark 2.1.1)
> val newPR = oldPR + (1.0 - resetProb) * msgSum
> it should be
> val newPR = resetProb + (1.0 - resetProb) * msgSum
> 2nd) in the message sending part (line 336 of the same file)
> Iterator((edge.dstId, edge.srcAttr._2 * edge.attr))
> should be
> Iterator((edge.dstId, edge.srcAttr._1 * edge.attr))
> as we should be sending the edge weight multiplied by the current pagerank of
> the source vertex (not the vertex's delta)
> 3rd) the tol check against the abs of the delta (line 335)
> if (edge.srcAttr._2 > tol) {
> should be
> if (Math.abs(edge.srcAttr._2) > tol) {
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-20839) Incorrect Dynamic PageRank calculation
[
https://issues.apache.org/jira/browse/SPARK-20839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049214#comment-16049214
]
Andrew Ray commented on SPARK-20839:
1 & 2 work together to do the algorithm properly with an active set, this is by
design.
3 although it may look wrong, the deltas (and thus messages) will always be
positive so this is not a problem.
> Incorrect Dynamic PageRank calculation
> --
>
> Key: SPARK-20839
> URL: https://issues.apache.org/jira/browse/SPARK-20839
> Project: Spark
> Issue Type: Bug
> Components: GraphX
>Affects Versions: 2.1.1
>Reporter: BahaaEddin AlAila
>
> Correct me if I am wrong
> I think there are three places where the pagerank calculation is incorrect
> 1st) in the VertexProgram (line 318 of PageRank.scala in spark 2.1.1)
> val newPR = oldPR + (1.0 - resetProb) * msgSum
> it should be
> val newPR = resetProb + (1.0 - resetProb) * msgSum
> 2nd) in the message sending part (line 336 of the same file)
> Iterator((edge.dstId, edge.srcAttr._2 * edge.attr))
> should be
> Iterator((edge.dstId, edge.srcAttr._1 * edge.attr))
> as we should be sending the edge weight multiplied by the current pagerank of
> the source vertex (not the vertex's delta)
> 3rd) the tol check against the abs of the delta (line 335)
> if (edge.srcAttr._2 > tol) {
> should be
> if (Math.abs(edge.srcAttr._2) > tol) {
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-20839) Incorrect Dynamic PageRank calculation
[ https://issues.apache.org/jira/browse/SPARK-20839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16019625#comment-16019625 ] Sean Owen commented on SPARK-20839: --- Why do you think so? A test case demonstrating the actual vs expected output, and why the expected output is correct, would help. Please review existing issues about the PageRank implementation in JIRA, some of which were closed as not a bug. > Incorrect Dynamic PageRank calculation > -- > > Key: SPARK-20839 > URL: https://issues.apache.org/jira/browse/SPARK-20839 > Project: Spark > Issue Type: Bug > Components: GraphX >Affects Versions: 2.1.1 >Reporter: BahaaEddin AlAila > > Correct me if I am wrong > I think there are two places where the pagerank calculation is incorrect > 1st) in the VertexProgram (line 318 of PageRank.scala in spark 2.1.1) > val newPR = oldPR + (1.0 - resetProb) * msgSum > it should be > val newPR = resetProb + (1.0 - resetProb) * msgSum > 2nd) in the message sending part (line 336 of the same file) > Iterator((edge.dstId, edge.srcAttr._2 * edge.attr)) > should be > Iterator((edge.dstId, edge.srcAttr._1 * edge.attr)) > as we should be sending the edge weight multiplied by the current pagerank of > the source vertex (not the vertex's delta) -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
