[
https://issues.apache.org/jira/browse/GIRAPH-12?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyunsik Choi updated GIRAPH-12:
-------------------------------
Attachment: GIRAPH-12_2.patch
I attach the second patch.
I have benchmarked this patch via GIRAPH-32. The results are shown as the
below. In the results, the improved version is slightly better than current
implementation. As Avery mentioned, the improved one makes threads
controllable, so it is an improve.
Users can adjust the number of core threads and max threads by using
GiraphJob's constants, such as MSG_FLUSHER_CORE_SIZE and MSG_FLUSHER_MAX_SIZE.
This setting can affect the performance. So, we may need to guide users to find
the best parameters.
But, this experiment may be not enough to evaluate this approach because this
experiment is conducted in small cluster.
*the result of original version*
{noformat}
org.apache.giraph.benchmark.RandomMessageBenchmark -e 2 -s 3 -w 6 -b 4 -n 150
-V 300000 -v
= 1st =
11/09/22 00:55:06 INFO mapred.JobClient: Total (milliseconds)=63096
11/09/22 00:55:06 INFO mapred.JobClient: Superstep 3 (milliseconds)=551
11/09/22 00:55:06 INFO mapred.JobClient: Setup (milliseconds)=1331
11/09/22 00:55:06 INFO mapred.JobClient: Shutdown (milliseconds)=1008
11/09/22 00:55:06 INFO mapred.JobClient: Vertex input superstep
(milliseconds)=516
11/09/22 00:55:06 INFO mapred.JobClient: Superstep 0 (milliseconds)=16079
11/09/22 00:55:06 INFO mapred.JobClient: Superstep 2 (milliseconds)=25657
11/09/22 00:55:06 INFO mapred.JobClient: Superstep 1 (milliseconds)=17950
= 2rd =
11/09/22 00:58:13 INFO mapred.JobClient: Total (milliseconds)=62771
11/09/22 00:58:13 INFO mapred.JobClient: Superstep 3 (milliseconds)=600
11/09/22 00:58:13 INFO mapred.JobClient: Setup (milliseconds)=1290
11/09/22 00:58:13 INFO mapred.JobClient: Shutdown (milliseconds)=950
11/09/22 00:58:13 INFO mapred.JobClient: Vertex input superstep
(milliseconds)=614
11/09/22 00:58:13 INFO mapred.JobClient: Superstep 0 (milliseconds)=15654
11/09/22 00:58:13 INFO mapred.JobClient: Superstep 2 (milliseconds)=25157
11/09/22 00:58:13 INFO mapred.JobClient: Superstep 1 (milliseconds)=18499
{noformat}
*the result of patched version*
{noformat}
= 1st =
11/09/22 00:59:41 INFO mapred.JobClient: Total (milliseconds)=60068
11/09/22 00:59:41 INFO mapred.JobClient: Superstep 3 (milliseconds)=542
11/09/22 00:59:41 INFO mapred.JobClient: Setup (milliseconds)=1219
11/09/22 00:59:41 INFO mapred.JobClient: Shutdown (milliseconds)=1025
11/09/22 00:59:41 INFO mapred.JobClient: Vertex input superstep
(milliseconds)=616
11/09/22 00:59:41 INFO mapred.JobClient: Superstep 0 (milliseconds)=15887
11/09/22 00:59:41 INFO mapred.JobClient: Superstep 2 (milliseconds)=23149
11/09/22 00:59:41 INFO mapred.JobClient: Superstep 1 (milliseconds)=17626
= 2rd =
11/09/22 01:01:05 INFO mapred.JobClient: Total (milliseconds)=60359
11/09/22 01:01:05 INFO mapred.JobClient: Superstep 3 (milliseconds)=510
11/09/22 01:01:05 INFO mapred.JobClient: Setup (milliseconds)=1399
11/09/22 01:01:05 INFO mapred.JobClient: Shutdown (milliseconds)=956
11/09/22 01:01:05 INFO mapred.JobClient: Vertex input superstep
(milliseconds)=550
11/09/22 01:01:05 INFO mapred.JobClient: Superstep 0 (milliseconds)=16054
11/09/22 01:01:05 INFO mapred.JobClient: Superstep 2 (milliseconds)=23049
11/09/22 01:01:05 INFO mapred.JobClient: Superstep 1 (milliseconds)=17835
{noformat}
> Investigate communication improvements
> --------------------------------------
>
> Key: GIRAPH-12
> URL: https://issues.apache.org/jira/browse/GIRAPH-12
> Project: Giraph
> Issue Type: Improvement
> Components: bsp
> Reporter: Avery Ching
> Assignee: Hyunsik Choi
> Priority: Minor
> Attachments: GIRAPH-12_1.patch, GIRAPH-12_2.patch
>
>
> Currently every worker will start up a thread to communicate with every other
> workers. Hadoop RPC is used for communication. For instance if there are
> 400 workers, each worker will create 400 threads. This ends up using a lot
> of memory, even with the option
> -Dmapred.child.java.opts="-Xss64k".
> It would be good to investigate using frameworks like Netty or custom roll
> our own to improve this situation. By moving away from Hadoop RPC, we would
> also make compatibility of different Hadoop versions easier.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
