Hello Artie;
I also had such problems when working on large datasets. It probably
needs a better hadoop/mapreduce configuration. For example did you try
increasing the memory of task tracker child processes in mapred-site.xml
such as:
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx3000m</value>
</property>
Regards;
Sertug
On 08-11-2013 01:26, Artie Pesh-Imam wrote:
Hey
It does work against a smaller data set and Im running 1.0.0 for hadoop 2.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
------------------------------------------------------------------------
From: Mohammad Islam Mohammad Islam <mailto:misla...@yahoo.com>
Reply: user@giraph.apache.org user@giraph.apache.org
<mailto:user@giraph.apache.org>, Mohammad Islam misla...@yahoo.com
<mailto:misla...@yahoo.com>
Date: November 7, 2013 at 6:20:53 PM
To: user@giraph.apache.org user@giraph.apache.org
<mailto:user@giraph.apache.org>
Subject: Re: Null Pointer Exception when running a job
Hi Artie,
Hopefully someone would give you a concrete answer.
My rambling comments:
Did you try with a smaller number of vertices? I want to know if the
problem is due to increased load.
What version of Giraph are you using?
Regards,
Mohammad
On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam
<artie.pesh-i...@tapad.com> wrote:
Please let me know if any additional context would be useful. With no
detail in the error message, I have no idea what the root issue could be.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
------------------------------------------------------------------------
From: Artie Pesh-Imam Artie Pesh-Imam <mailto:artie.pesh-i...@tapad.com>
Reply: Artie Pesh-Imam artie.pesh-i...@tapad.com
<mailto:artie.pesh-i...@tapad.com>
Date: November 7, 2013 at 4:25:30 PM
To: user@giraph.apache.org user@giraph.apache.org
<mailto:user@giraph.apache.org>
Subject: Null Pointer Exception when running a job
Hi all,
Im running a simple connected components algorithm against a graph of
almost 200 million vertices.
I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at
org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at
org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at
org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at
org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
