Implement a new message manager which writes messages to HDFS rather than
sending them over RPC.
------------------------------------------------------------------------------------------------
Key: HAMA-554
URL: https://issues.apache.org/jira/browse/HAMA-554
Project: Hama
Issue Type: New Feature
Reporter: Apurv Verma
The current message managers send all messages over RPC. In this issue we would
like to create a DFSMessageManager which writes all messages to HDFS and store
its PATH. Later in the sync() phase these PATHs may be send to other peers
which can then read the messages from the HDFS.
Pros:
This can handle the case when the number of messages sent in a single superstep
are too many to be all contained in memory. HAMA-521 already provides the
implementation of Disk based queue. We can abstract this DiskQueue in the util
section and use it here.
Cons:
Reading data from HDFS might be slow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
