Is there no other suggestions on what to do here? I really don’t have any idea
about what’s going on? I actually ran this against a single extract from our
graph.
Here are some more logs.
Again this is running giraph 1.0 against hadoop 2.
Thanks,
2013-11-19 17:06:04,097 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at
org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at
org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at
org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-i...@tapad.com
Date: November 11, 2013 at 10:55:54 AM
To: Sertuğ Kaya sertug.k...@agmlab.com, Mohammad Islam misla...@yahoo.com,
user@giraph.apache.org user@giraph.apache.org
Subject: Re: Null Pointer Exception when running a job
I can certainly try this. Are there any other configurations which are worth
exploring?
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Sertuğ Kaya Sertuğ Kaya
Reply: user@giraph.apache.org user@giraph.apache.org
Date: November 9, 2013 at 3:58:39 PM
To: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam
misla...@yahoo.com
Subject: Re: Null Pointer Exception when running a job
Hello Artie;
I also had such problems when working on large datasets. It probably needs a
better hadoop/mapreduce configuration. For example did you try increasing the
memory of task tracker child processes in mapred-site.xml such as:
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx3000m</value>
</property>
Regards;
Sertug
On 08-11-2013 01:26, Artie Pesh-Imam wrote:
Hey
It does work against a smaller data set and Im running 1.0.0 for hadoop 2.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Mohammad Islam Mohammad Islam
Reply: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam
misla...@yahoo.com
Date: November 7, 2013 at 6:20:53 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Re: Null Pointer Exception when running a job
Hi Artie,
Hopefully someone would give you a concrete answer.
My rambling comments:
Did you try with a smaller number of vertices? I want to know if the problem is
due to increased load.
What version of Giraph are you using?
Regards,
Mohammad
On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam
<artie.pesh-i...@tapad.com> wrote:
Please let me know if any additional context would be useful. With no detail in
the error message, I have no idea what the root issue could be.
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-i...@tapad.com
Date: November 7, 2013 at 4:25:30 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject: Null Pointer Exception when running a job
Hi all,
Im running a simple connected components algorithm against a graph of almost
200 million vertices.
I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
at
org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
at
org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
at
org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
--
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
