Hi, I have a problem in launching a large Giraph job. The same code works for a small number of workers, but when I try to get more I get the stack trace attached. I assume it must be some kind of inter machine communication problem. Is it a know issue by any chance?
Thanks, -- Gianmarco
2012-05-24 17:31:46,742 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2012-05-24 17:31:46,752 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@121b59a 2012-05-24 17:31:46,894 INFO org.apache.giraph.graph.GraphMapper: Distributed cache is empty. Assuming fatjar. 2012-05-24 17:31:46,894 INFO org.apache.giraph.graph.GraphMapper: setup: classpath @ /grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/jars/job.jar 2012-05-24 17:31:46,908 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Made the directory _bsp/_defaultZkManagerDir/job_201205070659_153878 2012-05-24 17:31:46,920 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Creating my filestamp _bsp/_defaultZkManagerDir/job_201205070659_153878/_task/gsta30753.tan.ygrid.yahoo.com 0 2012-05-24 17:31:46,935 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: Got [gsta30753.tan.ygrid.yahoo.com] 1 hosts from 1 candidates when 1 required (polling period is 3000) on attempt 0 2012-05-24 17:31:46,935 INFO org.apache.giraph.zk.ZooKeeperManager: createZooKeeperServerList: Creating the final ZooKeeper file '_bsp/_defaultZkManagerDir/job_201205070659_153878/zkServerList_gsta30753.tan.ygrid.yahoo.com 0 ' 2012-05-24 17:31:46,950 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: For task 0, got file 'zkServerList_gsta30753.tan.ygrid.yahoo.com 0 ' (polling period is 3000) 2012-05-24 17:31:46,950 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: Found [gsta30753.tan.ygrid.yahoo.com, 0] 2 hosts in filename 'zkServerList_gsta30753.tan.ygrid.yahoo.com 0 ' 2012-05-24 17:31:46,952 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Trying to delete old directory /grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/work/_bspZooKeeper 2012-05-24 17:31:46,965 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Creating file /grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/work/_bspZooKeeper/zoo.cfg in /grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/work/_bspZooKeeper with base port 22181 2012-05-24 17:31:46,965 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true 2012-05-24 17:31:46,965 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Delete of zoo.cfg = false 2012-05-24 17:31:46,980 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Attempting to start ZooKeeper server with command [/grid/0/Releases/gridjdk-1.6.0_21.1011192346-20110120-000/share/gridjdk-1.6.0_21/jre/bin/java, -Xmx512m, -XX:ParallelGCThreads=4, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=70, -XX:MaxGCPauseMillis=100, -cp, /grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/jars/job.jar, org.apache.zookeeper.server.quorum.QuorumPeerMain, /grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/work/_bspZooKeeper/zoo.cfg] in directory /grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/work/_bspZooKeeper 2012-05-24 17:31:46,988 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect to gsta30753.tan.ygrid.yahoo.com:22181 with poll msecs = 3000 2012-05-24 17:31:46,992 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Got ConnectException java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) at java.net.Socket.connect(Socket.java:529) at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:659) at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:424) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:645) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1082) at org.apache.hadoop.mapred.Child.main(Child.java:249) 2012-05-24 17:31:49,994 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect to gsta30753.tan.ygrid.yahoo.com:22181 with poll msecs = 3000 2012-05-24 17:31:49,995 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connected to gsta30753.tan.ygrid.yahoo.com/10.216.115.59:22181! 2012-05-24 17:31:49,995 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Creating my filestamp _bsp/_defaultZkManagerDir/job_201205070659_153878/_zkServer/gsta30753.tan.ygrid.yahoo.com 0 2012-05-24 17:31:50,016 INFO org.apache.giraph.graph.GraphMapper: setup: Starting up BspServiceMaster (master thread)... 2012-05-24 17:31:50,030 INFO org.apache.giraph.graph.BspService: BspService: Connecting to ZooKeeper with job job_201205070659_153878, 0 on gsta30753.tan.ygrid.yahoo.com:22181 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=gsta30753.tan.ygrid.yahoo.com 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_21 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/grid/0/Releases/gridjdk-1.6.0_21.1011192346-20110120-000/share/gridjdk-1.6.0_21/jre 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/jars/classes:/grid/1/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/jars:/grid/3/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/attempt_201205070659_153878_m_000000_0/work:/home/gs/conf/hadoop/tasktracker:/home/gs/java/jdk/lib/tools.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/..:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../hadoop-core-0.20.205.0.3.1.1204152302.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/asm-3.2.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/aspectjrt-1.6.5.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/aspectjtools-1.6.5.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/axis.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/bouncer_auth_java-0.5.12.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/BouncerFilterAuth-1.1.4.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-beanutils-1.7.0.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-beanutils-core-1.8.0.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-cli-1.2.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-codec-1.4.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-collections-3.2.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-configuration-1.6.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-daemon-1.0.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-digester-1.8.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-el-1.0.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-httpclient-3.0.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-lang-2.4.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-logging-1.1.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-logging-api-1.0.4.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-math-2.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/commons-net-1.4.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/core-3.1.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/hadoop-capacity-scheduler-0.20.205.0.3.1.1204152302.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/hadoop-fairscheduler-0.20.205.0.3.1.1204152302.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/hadoop-gpl-compression-0.1.0-1204152302.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/hadoop-simon-0.20.201.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/hadoop-thriftfs-0.20.205.0.3.1.1204152302.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/hadoop-ymon-0.20.201.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/hsqldb-1.8.0.10.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jackson-core-asl-1.0.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jackson-mapper-asl-1.0.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jasper-compiler-5.5.12.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jasper-runtime-5.5.12.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jaxrpc.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jdeb-0.8.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jersey-core-1.8.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jersey-json-1.8.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jersey-server-1.8.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jets3t-0.6.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jetty-6.1.26.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jetty-util-6.1.26.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jsch-0.1.42.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/junit-4.5.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/kfs-0.2.2.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/log4j-1.2.15.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/mockito-all-1.8.5.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/oro-2.0.8.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/saaj.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/servlet-api-2.5-20081211.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/SimonTool.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/slf4j-api-1.4.3.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/slf4j-log4j12-1.4.3.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/wsdl4j-1.5.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/xmlenc-0.52.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/YahooDNSToSwitchMapping-0.20.205.0.3.1.1204152302.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/yjava_byauth-0.5.6.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/yjava_servlet_filters-0.4.2-0.4.2.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/yjava_ysecure-1.3.2.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/yjava_ysecure_native-1.5.25.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/yjava_ysecure_native.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jsp-2.1/jsp-2.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/jsp-2.1/jsp-api-2.1.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../hadoop-capacity-scheduler-*.jar:/home/gs/hadoopviewfs/current/*.jar:/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../hadoop-capacity-scheduler-*.jar:/home/gs/hadoopviewfs/current/*.jar 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/grid/0/Releases/hadoop-0.20.205.0.3.1.1204152302-20120420-000/share/hadoop-0.20.205.0.3.1.1204152302/libexec/../lib/native/Linux-i386-32:/grid/3/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/attempt_201205070659_153878_m_000000_0/work 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/grid/3/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/attempt_201205070659_153878_m_000000_0/work/tmp 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=i386 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.18-164.15.1.el5.YAHOO.20101019 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=gdfm 2012-05-24 17:31:50,039 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/homes/gdfm 2012-05-24 17:31:50,040 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/grid/3/tmp/mapred-local/taskTracker/gdfm/jobcache/job_201205070659_153878/attempt_201205070659_153878_m_000000_0/work 2012-05-24 17:31:50,041 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=gsta30753.tan.ygrid.yahoo.com:22181 sessionTimeout=60000 watcher=org.apache.giraph.graph.BspServiceMaster@e7f6eb 2012-05-24 17:31:50,055 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server gsta30753.tan.ygrid.yahoo.com/10.216.115.59:22181 2012-05-24 17:31:50,057 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to gsta30753.tan.ygrid.yahoo.com/10.216.115.59:22181, initiating session 2012-05-24 17:31:50,113 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server gsta30753.tan.ygrid.yahoo.com/10.216.115.59:22181, sessionid = 0x1377fe780c00000, negotiated timeout = 300000 2012-05-24 17:31:50,115 INFO org.apache.giraph.graph.BspService: process: Asynchronous connection complete. 2012-05-24 17:31:50,118 INFO org.apache.giraph.graph.GraphMapper: map: No need to do anything when not a worker 2012-05-24 17:31:50,125 INFO org.apache.giraph.graph.GraphMapper: cleanup: Starting for MASTER_ZOOKEEPER_ONLY 2012-05-24 17:31:50,198 INFO org.apache.giraph.graph.BspServiceMaster: becomeMaster: First child is '/_hadoopBsp/job_201205070659_153878/_masterElectionDir/gsta30753.tan.ygrid.yahoo.com_00000000000' and my bid is '/_hadoopBsp/job_201205070659_153878/_masterElectionDir/gsta30753.tan.ygrid.yahoo.com_00000000000' 2012-05-24 17:31:50,208 INFO org.apache.giraph.graph.BspServiceMaster: becomeMaster: I am now the master! 2012-05-24 17:31:50,228 INFO org.apache.giraph.graph.BspService: process: applicationAttemptChanged signaled 2012-05-24 17:31:50,234 INFO org.apache.giraph.graph.BspService: getApplicationAttempt: Node /_hadoopBsp/job_201205070659_153878/_applicationAttemptsDir already exists! 2012-05-24 17:31:50,253 WARN org.apache.giraph.graph.BspService: process: Unknown and unprocessed event (path=/_hadoopBsp/job_201205070659_153878/_applicationAttemptsDir/0/_superstepDir, type=NodeChildrenChanged, state=SyncConnected) 2012-05-24 17:31:53,972 FATAL org.apache.giraph.graph.GraphMapper: uncaughtException: OverrideExceptionHandler on thread org.apache.giraph.graph.MasterThread, msg = generateInputSplits: Got IOException, exiting... java.lang.IllegalStateException: generateInputSplits: Got IOException at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:249) at org.apache.giraph.graph.BspServiceMaster.createInputSplits(BspServiceMaster.java:506) at org.apache.giraph.graph.MasterThread.run(MasterThread.java:98) Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://tiberiumtan-nn1.tan.ygrid.yahoo.com:8020/user/gdfm/implicit/implicit/flow-all-giraph at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235) at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252) at org.apache.giraph.lib.TextVertexInputFormat.getSplits(TextVertexInputFormat.java:120) at org.apache.giraph.graph.BspServiceMaster.generateInputSplits(BspServiceMaster.java:229) ... 2 more 2012-05-24 17:31:53,973 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Forced a shutdown hook kill of the ZooKeeper process.