ever reason, it may not be
necessary to instantiate them all.
From: Claudio Martella [claudio.marte...@gmail.com]
Sent: Thursday, June 07, 2012 1:35 AM
To: user@giraph.apache.org
Subject: Re: Resources or advice on minimising memory usage in Giraph/Hadoop
cod
Hi!
One interesting jvm option I learned about lately is
-XX:+UseCompressedStrings, which will use a byte [] for all strings,
that are fully defined in ASCII. Given that you are working with URIs,
I assume that this is true for most of your strings, so I would give
it a shot.
For more info on JVM
Won't this just postpone the pain?
On Thursday, June 7, 2012, David Garcia wrote:
> Based upon what you have mentioned, o think you are getting heap errors
> because every vertex in your graph will be loaded into memory prior to
> super step one. So if you have a large graph, with lots of state
Based upon what you have mentioned, o think you are getting heap errors because
every vertex in your graph will be loaded into memory prior to super step one.
So if you have a large graph, with lots of state, you probably have memory
issues from the very beginning. A simple way to mitigate the
No article or book, but here's a few tips.
1) Use aggregators! This can drastically can reduce the amount of
memory use by combining messages on the server side.
2) -Dmapred.child.java.opts="-Xss128k" or some other value (should
affect the RPC threads or netty threads)
3) You'll want to minimi
