Dionysios Logothetis created GIRAPH-1190:
--------------------------------------------
Summary: Improve memory usage of SendWorkerOneMessageToManyRequest
Key: GIRAPH-1190
URL: https://issues.apache.org/jira/browse/GIRAPH-1190
Project: Giraph
Issue Type: Improvement
Reporter: Dionysios Logothetis
The current implementation takes incoming messages stored as
ByteArrayOneMessageToManyIds and prepares them as a map from partition id to a
ByteArrayVertexIdMessages, which holds the messages for the corresponding
partition. It then adds these to the message store.
However, it is possible that these intermediate lists of message get big before
they are added to the message store. If they reach the capacity of the
underlying buffers, the job fails. This can be avoided if we push these lists
to the message store before the get big. This is mostly beneficial when we use
a combiner in which case the message store keeps only one value per vertex.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
