Hi Pavan, For all the intermediate processing there would be a buffer (intermediate memory space) that stores data, messages etc.and then the complete process further. Pls correct me if I am wrong.
I have set Xms and Xmx values properly. The problem is that the task runs for small datasets but as the input data size is increased, it fails. The error that I am getting in sysout logs is- # A fatal error has been detected by the Java Runtime Environment: # # SIGBUS (0x7) at pc=0x00002aaaab404144, pid=10397, tid=1144650048 # # JRE version: 6.0_25-b06 # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode linux-amd64 compressed oops) # Problematic frame: # J sun.nio.ch.SelectorImpl.processDeregisterQueue()V # # An error report file with more information is saved as: # /hadoopTaskTrackerLogsLocation/process_id/s_err_pid10397.log # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp Please suggest what should be done? Am I missing anything? Regards, Agrta Rawat On Wed, Apr 16, 2014 at 12:44 PM, Pavan Kumar A <pava...@outlook.com> wrote: > > What do u mean by buffer size? Just as a note, please ensure that Xmx & > Xms values are properly set for the mapper using mapred.child.java.opts > or mapred.map.child.java.opts > Also what does the error message show: please use pastebin & post the link > here. > ------------------------------ > Date: Wed, 16 Apr 2014 12:13:29 +0530 > Subject: Giraph Buffer Size > From: agrta.ra...@gmail.com > To: user@giraph.apache.org > > > Hi All, > > I am trying to run a job in Giraph-1.0.0 on Hadoop-1.0.0 cluster with 3 > nodes. > Each node has 32gb RAM. > > In superstep-8 of my algorithm approximately 2M messages are being sent > where size of each message is more than 20 kb. But the process sticks here > and task gets failed. > > In the sysout logs, it shows Fatal Error. > > Is this error because Buffer is getting full? > How can I increase the buffer size for giraph application? > > Please suggest. > > Regards, > Agrta Rawat > >
