[
https://issues.apache.org/jira/browse/GIRAPH-462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294416#comment-15294416
]
Hassan Eslami commented on GIRAPH-462:
--------------------------------------
The new out-of-core feature has been pushed to the trunk (GIRAPH-1066). This
issue should be resolved, although out-of-core optimization and its integration
with checkpointing is still on-going.
> Multithreading breaks out-of-core graph
> ---------------------------------------
>
> Key: GIRAPH-462
> URL: https://issues.apache.org/jira/browse/GIRAPH-462
> Project: Giraph
> Issue Type: Bug
> Reporter: Alessandro Presta
> Priority: Critical
> Attachments: GIRAPH-461.patch
>
>
> [~cmartella] pointed out this issue: when using multithreaded computation in
> conjunction with out-of-core graph, we incur in a race condition. The compute
> threads share the same DiskBackedPartitionStore, whose getPartition() method
> is not meant to be thread-safe. When two threads request two out-of-core
> partitions concurrently, they both try to load it to the same slot.
> The result is that we can lose the reference to one of the two partitions
> (which will not be written back to disk) and we can incur in a
> NullPointerException when both threads are trying to offload the currently
> loaded partition to disk.
> I ran this test to confirm the issue:
> https://gist.github.com/4429628
> All tests pass except the one that uses both out-of-core graph and multiple
> compute threads.
> The error is the following:
> https://gist.github.com/4429650
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
