[
https://issues.apache.org/jira/browse/GIRAPH-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643247#comment-14643247
]
Hudson commented on GIRAPH-1022:
--------------------------------
FAILURE: Integrated in Giraph-trunk-Commit #1560 (See
[https://builds.apache.org/job/Giraph-trunk-Commit/1560/])
[GIRAPH-1022] Adaptive out-of-core mechanism for input superstep and graph
(aching:
http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=03ade425dd5a65d3a713d5e7d85aa7605956fbd2)
* giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java
* giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java
* giraph-core/src/main/java/org/apache/giraph/ooc/MemoryEstimator.java
* giraph-examples/src/test/java/org/apache/giraph/TestOutOfCore.java
*
giraph-core/src/main/java/org/apache/giraph/partition/DiskBackedPartitionStore.java
* findbugs-exclude.xml
* giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java
*
giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerVerticesRequest.java
* giraph-core/src/main/java/org/apache/giraph/ooc/CheckMemoryCallable.java
* giraph-core/src/test/java/org/apache/giraph/partition/TestPartitionStores.java
*
giraph-core/src/test/java/org/apache/giraph/comm/messages/TestIntFloatPrimitiveMessageStores.java
* giraph-core/src/main/java/org/apache/giraph/graph/GraphTaskManager.java
* giraph-core/src/main/java/org/apache/giraph/edge/primitives/LongEdgeStore.java
* giraph-core/src/main/java/org/apache/giraph/ooc/package-info.java
* giraph-core/src/main/java/org/apache/giraph/graph/ComputeCallable.java
*
giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerEdgesRequest.java
* giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java
* giraph-core/src/main/java/org/apache/giraph/ooc/JVMMemoryEstimator.java
* giraph-core/src/main/java/org/apache/giraph/ooc/AdaptiveOutOfCoreEngine.java
* giraph-core/src/main/java/org/apache/giraph/edge/AbstractEdgeStore.java
* giraph-core/src/main/java/org/apache/giraph/edge/primitives/IntEdgeStore.java
* giraph-core/src/main/java/org/apache/giraph/partition/PartitionStore.java
* giraph-core/src/main/java/org/apache/giraph/ooc/OutOfCoreEngine.java
* giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java
*
giraph-core/src/test/java/org/apache/giraph/comm/messages/TestLongDoublePrimitiveMessageStores.java
* giraph-core/src/main/java/org/apache/giraph/edge/SimpleEdgeStore.java
* CHANGELOG
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
*
giraph-core/src/main/java/org/apache/giraph/partition/SimplePartitionStore.java
*
giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/long_id/LongAbstractListMessageStore.java
* giraph-core/src/main/java/org/apache/giraph/ooc/DiskBackedPartitionStore.java
*
giraph-core/src/main/java/org/apache/giraph/ooc/OutOfCoreProcessorCallable.java
> Out-of-core mechanism for input superstep and graph data
> --------------------------------------------------------
>
> Key: GIRAPH-1022
> URL: https://issues.apache.org/jira/browse/GIRAPH-1022
> Project: Giraph
> Issue Type: New Feature
> Components: graph
> Reporter: Hassan Eslami
> Assignee: Hassan Eslami
>
> Currently the out-of-core mechanism has several issues:
> - It does not work properly in input superstep
> - It has several bugs (correctness, and performance) for graph data structure
> - It is not considering specialized message classes for out-of-core messages.
> This diff addresses some of these problems, and provides a cohesive mechanism
> for input superstep and graph data. With this way of doing out-of-core,
> efficiently handling out-of-core messages is straightforward (will be
> provided in a separate diff).
> This diff provides an adaptive mechanism to remove users out of the loop for
> using out-of-core. If the graph can be fit in memory, all the computation
> remains in memory. If memory is very limited and GC is killing the
> performance, or the graph cannot fit in memory, out-of-core mechanism kicks
> in and helps improve performance in many cases, and avoids application
> failure due to OutOfMemory exception or it's related exceptions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
