Hi All, I am trying to run a giraph job on multiple number of workers. But it fails with IllegalStateException. While the same job runs fine when no. of worker is 1.
I am running giraph job using- hadoop jar TryGiraph.jar org.apache.giraph.GiraphRunner vertexClass -mc MyMasterCompute -wc MyWorkerContext -eif EdgeClass -eip EdgeInputPath -vif VertexClass -vip VertexPath -of OutputFormaClass -w 12 and the exception is- java.lang.IllegalStateException: run: Caught an unrecoverable exception connectAllAddresses: Too many failures (1000). at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.IllegalStateException: connectAllAddresses: Too many failures (1000). at org.apache.giraph.comm.netty.NettyClient.connectAllAddresses(NettyClient.java:445) at org.apache.giraph.comm.netty.NettyWorkerClient.openConnections(NettyWorkerClient.java:129) at org.apache.giraph.comm.netty.NettyWorkerClient.setup(NettyWorkerClient.java:165) at org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:496) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:230) at org.apache. java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:366) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:282) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) 2014-08-18 12:33:52,982 WARN org.apache.giraph.comm.netty.NettyClient: connectAllAddresses: Future failed to connect with iLabHadoop3/172.21.208.178:30000 with 2 failures because of java.net.ConnectException: Connection refused 2014-08-18 12:33:52,982 WARN org.apache.giraph.comm.netty.handler.ResponseClientHandler: exceptionCaught: Channel failed with remote address null java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:366) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:282) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) 2014-08-18 12:33:52,983 INFO org.apache.giraph.comm.netty.NettyClient: Using Netty without authentication. 2014-08-18 12:33:52,984 INFO org.apache.giraph.comm.netty.NettyClient: connectAllAddresses: Successfully added 0 connections, (1 total connected) 1 failed, 3 failures total. 2014-08-18 12:33:52,985 WARN org.apache.giraph.comm.netty.handler.ResponseClientHandler: exceptionCaught: Channel failed with remote address null java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:404) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:366) at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:282) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) I have installed hadoop-1.0.0 in distributed mode and giraph version is 1.0.0.