Hi Hassan,
I found the source of problem with KryoException: BufferUnderflow.
There is a code snippet In metod
org.apache.giraph.ooc.data.DiskBackedPartitionStore#loadInMemoryPartitionData:
for (int i = 0; i < numVertices; ++i) {
readOutEdges(dataInput, partition);
}
Variable numV
Hi Hassan,
Thank you for your help!
I'll try to tune GC as you suggested and play with different
ThresholdBasedOracle settings.
As for now I was able to run computation without OutOfMemory but with
really strict option giraph.maxPartitionsInMemory=1 (haven't tried
other values since it is really
Hi Denis,
Thanks for trying this out without the "isStaticGraph" option. I'll soon
create a JIRA and fix this issue.
It is worth mentioning that the new OOC is designed in a way that it works
even with a very low available memory. But, there are subtleties that
should be considered. First, the Gi
Hi Hai,
I think we saw something like this in our environment.
Interesting row is this one:
2016-10-27 19:04:00,000 INFO [SessionTracker]
org.apache.zookeeper.server.ZooKeeperServer: Expiring session
0x158084f5b2100b8, timeout of 60ms exceeded
I think that one of workers due to some reason d
Hi Guys
The OutOfMemoryError might be solved be adding
"-Dmapreduce.map.memory.mb=14848". But in my tests, I found some more
problems during running out of core graph.
I did two tests with 150G 10^10 vertices input in 1.2 version, and it seems
like it not necessary to add like "giraph.userPartiti
Hi Xenia,
Thank you! I'll check the thread you mentioned.
Best Regards,
Denis Dudinski
2016-11-08 14:16 GMT+03:00 Xenia Demetriou :
> Hi Denis,
>
> For the "java.lang.OutOfMemoryError: GC overhead limit exceeded" error I
> hope that the conversation in below link can help you.
> www.mail-arch
Hi Denis,
For the "java.lang.OutOfMemoryError: GC overhead limit exceeded" error I
hope that the conversation in below link can help you.
www.mail-archive.com/user@giraph.apache.org/msg02938.html
Regards,
Xenia
2016-11-08 12:25 GMT+02:00 Denis Dudinski :
> Hi Hassan,
>
> Thank you for really
Hi Hassan,
Thank you for really quick response!
I changed "giraph.isStaticGraph" to false and the error disappeared.
As expected iteration became slow and wrote to disk edges once again
in superstep 1.
However, the computation failed at superstep 2 with error
"java.lang.OutOfMemoryError: GC over
Hi Denis,
Thanks for bringing up the issue. In the previous conversation thread, the
similar problem is reported even with a simpler example connected component
calculation. Although, back then, we were developing other
performance-critical components of OOC.
Let's debug this issue together to ma
Hello,
We are trying to calculate PageRank on huge graph, which does not fit
into memory. For calculation to succeed we tried to turn on OutOfCore
feature of Giraph, but every launch we tried resulted in
com.esotericsoftware.kryo.KryoException: Buffer underflow.
Each time it happens on different s
10 matches
Mail list logo
