[ https://issues.apache.org/jira/browse/GIRAPH-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643247#comment-14643247 ]
Hudson commented on GIRAPH-1022: -------------------------------- FAILURE: Integrated in Giraph-trunk-Commit #1560 (See [https://builds.apache.org/job/Giraph-trunk-Commit/1560/]) [GIRAPH-1022] Adaptive out-of-core mechanism for input superstep and graph (aching: http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=03ade425dd5a65d3a713d5e7d85aa7605956fbd2) * giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java * giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java * giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java * giraph-core/src/main/java/org/apache/giraph/ooc/MemoryEstimator.java * giraph-examples/src/test/java/org/apache/giraph/TestOutOfCore.java * giraph-core/src/main/java/org/apache/giraph/partition/DiskBackedPartitionStore.java * findbugs-exclude.xml * giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java * giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerVerticesRequest.java * giraph-core/src/main/java/org/apache/giraph/ooc/CheckMemoryCallable.java * giraph-core/src/test/java/org/apache/giraph/partition/TestPartitionStores.java * giraph-core/src/test/java/org/apache/giraph/comm/messages/TestIntFloatPrimitiveMessageStores.java * giraph-core/src/main/java/org/apache/giraph/graph/GraphTaskManager.java * giraph-core/src/main/java/org/apache/giraph/edge/primitives/LongEdgeStore.java * giraph-core/src/main/java/org/apache/giraph/ooc/package-info.java * giraph-core/src/main/java/org/apache/giraph/graph/ComputeCallable.java * giraph-core/src/main/java/org/apache/giraph/comm/requests/SendWorkerEdgesRequest.java * giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java * giraph-core/src/main/java/org/apache/giraph/ooc/JVMMemoryEstimator.java * giraph-core/src/main/java/org/apache/giraph/ooc/AdaptiveOutOfCoreEngine.java * giraph-core/src/main/java/org/apache/giraph/edge/AbstractEdgeStore.java * giraph-core/src/main/java/org/apache/giraph/edge/primitives/IntEdgeStore.java * giraph-core/src/main/java/org/apache/giraph/partition/PartitionStore.java * giraph-core/src/main/java/org/apache/giraph/ooc/OutOfCoreEngine.java * giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java * giraph-core/src/test/java/org/apache/giraph/comm/messages/TestLongDoublePrimitiveMessageStores.java * giraph-core/src/main/java/org/apache/giraph/edge/SimpleEdgeStore.java * CHANGELOG * giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java * giraph-core/src/main/java/org/apache/giraph/partition/SimplePartitionStore.java * giraph-core/src/main/java/org/apache/giraph/comm/messages/primitives/long_id/LongAbstractListMessageStore.java * giraph-core/src/main/java/org/apache/giraph/ooc/DiskBackedPartitionStore.java * giraph-core/src/main/java/org/apache/giraph/ooc/OutOfCoreProcessorCallable.java > Out-of-core mechanism for input superstep and graph data > -------------------------------------------------------- > > Key: GIRAPH-1022 > URL: https://issues.apache.org/jira/browse/GIRAPH-1022 > Project: Giraph > Issue Type: New Feature > Components: graph > Reporter: Hassan Eslami > Assignee: Hassan Eslami > > Currently the out-of-core mechanism has several issues: > - It does not work properly in input superstep > - It has several bugs (correctness, and performance) for graph data structure > - It is not considering specialized message classes for out-of-core messages. > This diff addresses some of these problems, and provides a cohesive mechanism > for input superstep and graph data. With this way of doing out-of-core, > efficiently handling out-of-core messages is straightforward (will be > provided in a separate diff). > This diff provides an adaptive mechanism to remove users out of the loop for > using out-of-core. If the graph can be fit in memory, all the computation > remains in memory. If memory is very limited and GC is killing the > performance, or the graph cannot fit in memory, out-of-core mechanism kicks > in and helps improve performance in many cases, and avoids application > failure due to OutOfMemory exception or it's related exceptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)