Hi,
I m facing an OOM Exception when running a Giraph Job. Worker logs gives
following exception
014-12-09 21:57:43,451 INFO [communication thread]
org.apache.hadoop.mapred.Task: Communication exception:
java.lang.OutOfMemoryError: Java heap space
at
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:231)
at $Proxy6.ping(Unknown Source)
at org.apache.hadoop.mapred.Task$TaskReporter.run(Task.java:736)
at java.lang.Thread.run(Thread.java:722)
I tried increase the heap size by passing -ca
mapred.child.java.opts="-Xmx1024m" to the Giraph application. But when I do
that I am getting following error
2014-12-09 22:08:13,103 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics
report from attempt_1415143619219_0076_m_000006_0: Container
[pid=13551,containerID=container_1415143619219_0076_01_000008] is running
beyond virtual memory limits. Current usage: 223.3 MB of 1 GB physical
memory used; 2.7 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1415143619219_0076_01_000008 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
I am using Giraph 1.1.0-SNAPSHOT which was taken a week before the release.
It will be great if some one could give me some tips to overcome this
problem.
Regards,
Charith
--
Charith Dhanushka Wickramaarachchi
Tel +1 213 447 4253
Blog http://charith.wickramaarachchi.org/
<http://charithwiki.blogspot.com/>
Twitter @charithwiki <https://twitter.com/charithwiki>
This communication may contain privileged or other confidential information
and is intended exclusively for the addressee/s. If you are not the
intended recipient/s, or believe that you may have
received this communication in error, please reply to the sender indicating
that fact and delete the copy you received and in addition, you should not
print, copy, retransmit, disseminate, or otherwise use the information
contained in this communication. Internet communications cannot be
guaranteed to be timely, secure, error or virus-free. The sender does not
accept liability for any errors or omissions
