Hassan Eslami created GIRAPH-1025:
-------------------------------------
Summary: Flow-control mechanism improving out-of-core feature
Key: GIRAPH-1025
URL: https://issues.apache.org/jira/browse/GIRAPH-1025
Project: Giraph
Issue Type: Improvement
Components: graph
Reporter: Hassan Eslami
Assignee: Hassan Eslami
Current implementation of out-of-core mechanism may fail due to message
flooding in applications without message combiner. In such applications the
rate at which messages arrive may be much more than the rate of read/write to
disk in OOC mechanism. In this scenario, the OOC mechanism will get OOM as it
cannot keep up with storing incoming messages in-memory.
One might say this problem may also appear in input-superstep as we have input
vertices and edges going around as messages, but note that in input superstep
the rate at which a worker receive messages (vertices and edges) are almost the
same as the rate of reading from disk, and that is almost the same as the rate
at which offloading data to disk happens. However, this flow-control mechanism
can be generalized for input superstep too. At this point, this generalization
does not seem necessary though.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
