[
https://issues.apache.org/jira/browse/GIRAPH-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643243#comment-14643243
]
Avery Ching commented on GIRAPH-1022:
-------------------------------------
Thanks for this [~heslami] and for the review [~dlogothetis]!
> Out-of-core mechanism for input superstep and graph data
> --------------------------------------------------------
>
> Key: GIRAPH-1022
> URL: https://issues.apache.org/jira/browse/GIRAPH-1022
> Project: Giraph
> Issue Type: New Feature
> Components: graph
> Reporter: Hassan Eslami
> Assignee: Hassan Eslami
>
> Currently the out-of-core mechanism has several issues:
> - It does not work properly in input superstep
> - It has several bugs (correctness, and performance) for graph data structure
> - It is not considering specialized message classes for out-of-core messages.
> This diff addresses some of these problems, and provides a cohesive mechanism
> for input superstep and graph data. With this way of doing out-of-core,
> efficiently handling out-of-core messages is straightforward (will be
> provided in a separate diff).
> This diff provides an adaptive mechanism to remove users out of the loop for
> using out-of-core. If the graph can be fit in memory, all the computation
> remains in memory. If memory is very limited and GC is killing the
> performance, or the graph cannot fit in memory, out-of-core mechanism kicks
> in and helps improve performance in many cases, and avoids application
> failure due to OutOfMemory exception or it's related exceptions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
