Hi, I'm still trying to get Giraph to work on a graph that requires more memory that is available. The problem is that when the Workers try to offload partitions, the offloading fails. The DiskBackedPartitionStore fails to create the directory _bsp/_partitions/job-xxxx/part-vertices-xxx (roughly from recall).
The input or computation will then continue for a while, which I believe is because it is still managing to hold everything in memory - but at some point it reaches the limit where there simply is no more heap space, and it crashes with OOM. Has anybody had this problem with giraph failing to make HDFS directories?
