[
https://issues.apache.org/jira/browse/HAMA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Jungblut updated HAMA-554:
---------------------------------
Component/s: messaging
bsp
Labels: (was: features)
> Implement a new message manager which writes messages to HDFS rather than
> sending them over RPC.
> ------------------------------------------------------------------------------------------------
>
> Key: HAMA-554
> URL: https://issues.apache.org/jira/browse/HAMA-554
> Project: Hama
> Issue Type: New Feature
> Components: bsp, messaging
> Reporter: Apurv Verma
>
> The current message managers send all messages over RPC. In this issue we
> would like to create a DFSMessageManager which writes all messages to HDFS
> and store its PATH. Later in the sync() phase these PATHs may be send to
> other peers which can then read the messages from the HDFS.
> Pros:
> This can handle the case when the number of messages sent in a single
> superstep are too many to be all contained in memory. HAMA-521 already
> provides the implementation of Disk based queue. We can abstract this
> DiskQueue in the util section and use it here.
> Cons:
> Reading data from HDFS might be slow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
