[
https://issues.apache.org/jira/browse/GIRAPH-37?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271245#comment-13271245
]
Avery Ching commented on GIRAPH-37:
-----------------------------------
Thanks Claudio.
Here are more results with a scaled up 10 worker setup:
Hadoop RPC:
hadoop jar ~/giraph-0.2-SNAPSHOT-jar-with-dependencies.jar
org.apache.giraph.benchmark.PageRankBenchmark -Dgiraph.useNetty=false -w 10 -V
10000000 -s 5 -e 2 -v
12/05/09 02:32:05 INFO mapred.JobClient: Giraph Timers
12/05/09 02:32:05 INFO mapred.JobClient: Total (milliseconds)=149880
12/05/09 02:32:05 INFO mapred.JobClient: Superstep 3 (milliseconds)=21575
12/05/09 02:32:05 INFO mapred.JobClient: Setup (milliseconds)=7428
12/05/09 02:32:05 INFO mapred.JobClient: Shutdown (milliseconds)=174
12/05/09 02:32:05 INFO mapred.JobClient: Vertex input superstep
(milliseconds)=39558
12/05/09 02:32:05 INFO mapred.JobClient: Superstep 0 (milliseconds)=16887
12/05/09 02:32:05 INFO mapred.JobClient: Superstep 4 (milliseconds)=18613
12/05/09 02:32:05 INFO mapred.JobClient: Superstep 5 (milliseconds)=3292
12/05/09 02:32:05 INFO mapred.JobClient: Superstep 2 (milliseconds)=21313
12/05/09 02:32:05 INFO mapred.JobClient: Superstep 1 (milliseconds)=21035
Netty:
hadoop jar ~/giraph-0.2-SNAPSHOT-jar-with-dependencies.jar
org.apache.giraph.benchmark.PageRankBenchmark -Dgiraph.useNetty=true -w 10 -V
10000000 -s 5 -e 2 -v
12/05/09 02:35:06 INFO mapred.JobClient: Giraph Timers
12/05/09 02:35:06 INFO mapred.JobClient: Total (milliseconds)=59270
12/05/09 02:35:06 INFO mapred.JobClient: Superstep 3 (milliseconds)=11827
12/05/09 02:35:06 INFO mapred.JobClient: Setup (milliseconds)=3196
12/05/09 02:35:06 INFO mapred.JobClient: Shutdown (milliseconds)=124
12/05/09 02:35:06 INFO mapred.JobClient: Vertex input superstep
(milliseconds)=13130
12/05/09 02:35:06 INFO mapred.JobClient: Superstep 0 (milliseconds)=8564
12/05/09 02:35:06 INFO mapred.JobClient: Superstep 4 (milliseconds)=5540
12/05/09 02:35:06 INFO mapred.JobClient: Superstep 5 (milliseconds)=2012
12/05/09 02:35:06 INFO mapred.JobClient: Superstep 2 (milliseconds)=8601
12/05/09 02:35:06 INFO mapred.JobClient: Superstep 1 (milliseconds)=6271
These results are fairly similar to the first set (even though there are more
workers). I'm pretty sure we can squeeze more performance from Netty in the
future in future patches (i.e. local send optimization is missing, tuning TCP
parameters, exposing more knobs to the user, etc.).
> Implement Netty-backed rpc solution
> -----------------------------------
>
> Key: GIRAPH-37
> URL: https://issues.apache.org/jira/browse/GIRAPH-37
> Project: Giraph
> Issue Type: New Feature
> Reporter: Jakob Homan
> Assignee: Jakob Homan
> Attachments: GIRAPH-37-wip.patch, GIRAPH-37.patch
>
>
> GIRAPH-12 considered replacing the current Hadoop based rpc method with
> Netty, but didn't went in another direction. I think there is still value in
> this approach, and will also look at Finagle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
