Dear Giraph users, I am trying to get my Giraph installation to work but I am stuck.
Below I paste my command and job tracker log entry. I appreciate your help. hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/fredrik/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/fredrik/output/shortestpaths -w 1 Task Logs: 'attempt_201501281117_0016_m_000000_0' *stdout logs* ------------------------------ *stderr logs* SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [file:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] ------------------------------ *syslog logs* 2015-01-28 14:49:28,422 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library 2015-01-28 14:49:30,078 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists! 2015-01-28 14:49:30,745 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2015-01-28 14:49:30,781 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2bf8e35d 2015-01-28 14:49:31,022 INFO org.apache.hadoop.mapred.MapTask: Processing split: 'org.apache.giraph.bsp.BspInputSplit, index=-1, num=-1 2015-01-28 14:49:31,270 INFO org.apache.giraph.graph.GraphTaskManager: setup: Log level remains at info 2015-01-28 14:49:31,852 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Made the directory _bsp/_defaultZkManagerDir/job_201501281117_0016 2015-01-28 14:49:31,854 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Made the directory _bsp/_defaultZkManagerDir/job_201501281117_0016/_zkServer 2015-01-28 14:49:31,857 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Creating my filestamp _bsp/_defaultZkManagerDir/job_201501281117_0016/_task/fredrik-virtualbox 0 2015-01-28 14:49:31,925 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: Got [fredrik-virtualbox] 1 hosts from 1 candidates when 1 required (polling period is 3000) on attempt 0 2015-01-28 14:49:31,926 INFO org.apache.giraph.zk.ZooKeeperManager: createZooKeeperServerList: Creating the final ZooKeeper file '_bsp/_defaultZkManagerDir/job_201501281117_0016/zkServerList_fredrik-virtualbox 0 ' 2015-01-28 14:49:31,955 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: For task 0, got file 'zkServerList_fredrik-virtualbox 0 ' (polling period is 3000) 2015-01-28 14:49:31,955 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: Found [fredrik-virtualbox, 0] 2 hosts in filename 'zkServerList_fredrik-virtualbox 0 ' 2015-01-28 14:49:31,956 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Trying to delete old directory /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper 2015-01-28 14:49:31,969 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Creating file /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg in /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper with base port 22181 2015-01-28 14:49:31,970 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true 2015-01-28 14:49:31,970 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Delete of zoo.cfg = false 2015-01-28 14:49:31,989 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect to fredrik-virtualbox:22181 with poll msecs = 3000 2015-01-28 14:49:32,000 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Got ConnectException java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:601) at org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:433) at org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:231) at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:55) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.Child.main(Child.java:249) 2015-01-28 14:49:32,018 INFO org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading configuration from: /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg 2015-01-28 14:49:32,029 INFO org.apache.zookeeper.server.DatadirCleanupManager: autopurge.snapRetainCount set to 3 2015-01-28 14:49:32,029 INFO org.apache.zookeeper.server.DatadirCleanupManager: autopurge.purgeInterval set to 0 2015-01-28 14:49:32,029 INFO org.apache.zookeeper.server.DatadirCleanupManager: Purge task is not scheduled. 2015-01-28 14:49:32,035 WARN org.apache.giraph.zk.InProcessZooKeeperRunner: Either no config or no quorum defined in config, running in standalone mode 2015-01-28 14:49:32,154 INFO org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading configuration from: /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg 2015-01-28 14:49:32,154 INFO org.apache.zookeeper.server.ZooKeeperServerMain: Starting server 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:host.name=fredrik-VirtualBox 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.version=1.7.0_65 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.vendor=Oracle Corporation 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.class.path=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/classes:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work:/usr/local/hadoop/libexec/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/libexec/..:/usr/local/hadoop/libexec/../hadoop-core-1.2.1.jar:/usr/local/hadoop/libexec/../lib/asm-3.2.jar:/usr/local/hadoop/libexec/../lib/aspectjrt-1.6.11.jar:/usr/local/hadoop/libexec/../lib/aspectjtools-1.6.11.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-net-3.1.jar:/usr/local/hadoop/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-capacity-scheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-fairscheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-thriftfs-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop/libexec/../lib/junit-4.5.jar:/usr/local/hadoop/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-api-2.1.jar 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.library.path=/usr/local/hadoop/libexec/../lib/native/Linux-amd64-64:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.io.tmpdir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work/tmp 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.compiler=<NA> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:os.name=Linux 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:os.arch=amd64 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:os.version=3.16.0-29-generic 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:user.name=fredrik 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:user.home=/home/fredrik 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:user.dir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work 2015-01-28 14:49:32,215 INFO org.apache.zookeeper.server.ZooKeeperServer: tickTime set to 6000 2015-01-28 14:49:32,215 INFO org.apache.zookeeper.server.ZooKeeperServer: minSessionTimeout set to 600000 2015-01-28 14:49:32,216 INFO org.apache.zookeeper.server.ZooKeeperServer: maxSessionTimeout set to 900000 2015-01-28 14:49:32,266 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: binding to port 0.0.0.0/0.0.0.0:22181 2015-01-28 14:49:32,337 INFO org.apache.zookeeper.server.persistence.FileTxnSnapLog: Snapshotting: 0x0 to /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/version-2/snapshot.0 2015-01-28 14:49:32,414 INFO org.apache.zookeeper.server.PrepRequestProcessor: zookeeper.skipACL=="yes", ACL checks will be skipped 2015-01-28 14:49:35,030 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect to fredrik-virtualbox:22181 with poll msecs = 3000 2015-01-28 14:49:35,030 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connected to fredrik-VirtualBox/192.168.56.10:22181! 2015-01-28 14:49:35,031 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Creating my filestamp _bsp/_defaultZkManagerDir/job_201501281117_0016/_zkServer/fredrik-virtualbox 0 2015-01-28 14:49:35,031 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.10:34203 2015-01-28 14:49:35,798 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.10:34204 2015-01-28 14:49:35,804 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.10:34204 2015-01-28 14:49:35,807 INFO org.apache.zookeeper.server.persistence.FileTxnLog: Creating new log file: log.1 2015-01-28 14:49:35,825 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x14b30cddb450000 with negotiated timeout 600000 for client /192.168.56.10:34204 2015-01-28 14:49:35,989 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x1 zxid:0x2 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NoNode for /_hadoopBsp/job_201501281117_0016 2015-01-28 14:49:36,012 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x8 zxid:0x7 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0 Error:KeeperErrorCode = NoNode for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0 2015-01-28 14:49:36,013 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x9 zxid:0x8 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp 2015-01-28 14:49:36,013 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0xa zxid:0x9 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016 2015-01-28 14:49:36,014 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0xb zxid:0xa txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir 2015-01-28 14:49:36,023 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0xf zxid:0xd txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir Error:KeeperErrorCode = NoNode for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir 2015-01-28 14:49:36,025 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x10 zxid:0xe txntype:-1 reqpath:n/a Error Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp 2015-01-28 14:49:36,027 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x11 zxid:0xf txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016 2015-01-28 14:49:36,028 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x12 zxid:0x10 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir 2015-01-28 14:49:36,029 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x13 zxid:0x11 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0 Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0 2015-01-28 14:49:36,030 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x14 zxid:0x12 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir 2015-01-28 14:49:36,415 WARN org.apache.zookeeper.server.NIOServerCnxn: caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) at java.lang.Thread.run(Thread.java:745) 2015-01-28 14:49:36,415 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.168.56.10:34203 (no session established for client) 2015-01-28 14:49:37,361 INFO org.apache.giraph.graph.GraphTaskManager: setup: Chosen to run ZooKeeper... 2015-01-28 14:49:37,362 INFO org.apache.giraph.graph.GraphTaskManager: setup: Starting up BspServiceMaster (master thread)... 2015-01-28 14:49:37,374 INFO org.apache.giraph.bsp.BspService: BspService: Path to create to halt is /_hadoopBsp/job_201501281117_0016/_haltComputation 2015-01-28 14:49:37,374 INFO org.apache.giraph.bsp.BspService: BspService: Connecting to ZooKeeper with job job_201501281117_0016, 0 on fredrik-virtualbox:22181 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=fredrik-VirtualBox 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.7.0_65 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/classes:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work:/usr/local/hadoop/libexec/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/libexec/..:/usr/local/hadoop/libexec/../hadoop-core-1.2.1.jar:/usr/local/hadoop/libexec/../lib/asm-3.2.jar:/usr/local/hadoop/libexec/../lib/aspectjrt-1.6.11.jar:/usr/local/hadoop/libexec/../lib/aspectjtools-1.6.11.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-net-3.1.jar:/usr/local/hadoop/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-capacity-scheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-fairscheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-thriftfs-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop/libexec/../lib/junit-4.5.jar:/usr/local/hadoop/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-api-2.1.jar 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/local/hadoop/libexec/../lib/native/Linux-amd64-64:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work/tmp 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=3.16.0-29-generic 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=fredrik 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/fredrik 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=fredrik-virtualbox:22181 sessionTimeout=60000 watcher=org.apache.giraph.master.BspServiceMaster@f9aac17 2015-01-28 14:49:37,393 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server fredrik-VirtualBox/192.168.56.10:22181. Will not attempt to authenticate using SASL (unknown error) 2015-01-28 14:49:37,393 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.10:34205 2015-01-28 14:49:37,394 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to fredrik-VirtualBox/192.168.56.10:22181, initiating session 2015-01-28 14:49:37,394 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.10:34205 2015-01-28 14:49:37,395 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x14b30cddb450001 with negotiated timeout 600000 for client /192.168.56.10:34205 2015-01-28 14:49:37,395 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server fredrik-VirtualBox/192.168.56.10:22181, sessionid = 0x14b30cddb450001, negotiated timeout = 600000 2015-01-28 14:49:37,396 INFO org.apache.giraph.bsp.BspService: process: Asynchronous connection complete. 2015-01-28 14:49:37,402 INFO org.apache.giraph.graph.GraphTaskManager: map: No need to do anything when not a worker 2015-01-28 14:49:37,402 INFO org.apache.giraph.graph.GraphTaskManager: cleanup: Starting for MASTER_ZOOKEEPER_ONLY 2015-01-28 14:49:37,404 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x1 zxid:0x17 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_masterElectionDir Error:KeeperErrorCode = NoNode for /_hadoopBsp/job_201501281117_0016/_masterElectionDir 2015-01-28 14:49:37,405 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x2 zxid:0x18 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp 2015-01-28 14:49:37,406 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x3 zxid:0x19 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016 2015-01-28 14:49:37,409 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x6 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_masterJobState Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_masterJobState 2015-01-28 14:49:37,409 INFO org.apache.giraph.bsp.BspService: getJobState: Job state already exists (/_hadoopBsp/job_201501281117_0016/_masterJobState) 2015-01-28 14:49:37,411 INFO org.apache.giraph.master.BspServiceMaster: becomeMaster: First child is '/_hadoopBsp/job_201501281117_0016/_masterElectionDir/fredrik-virtualbox_00000000000' and my bid is '/_hadoopBsp/job_201501281117_0016/_masterElectionDir/fredrik-virtualbox_00000000000' 2015-01-28 14:49:37,414 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x9 zxid:0x1d txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir 2015-01-28 14:49:37,414 INFO org.apache.giraph.bsp.BspService: getApplicationAttempt: Node /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir already exists! 2015-01-28 14:49:37,477 INFO org.apache.giraph.comm.netty.NettyServer: NettyServer: Using execution group with 8 threads for requestFrameDecoder. 2015-01-28 14:49:37,537 INFO org.apache.giraph.comm.netty.NettyServer: start: Started server communication server: fredrik-VirtualBox/192.168.56.10:30000 with up to 16 threads on bind attempt 0 with sendBufferSize = 32768 receiveBufferSize = 524288 2015-01-28 14:49:37,545 INFO org.apache.giraph.comm.netty.NettyClient: NettyClient: Using execution handler with 8 threads after request-encoder. 2015-01-28 14:49:37,548 INFO org.apache.giraph.master.BspServiceMaster: becomeMaster: I am now the master! 2015-01-28 14:49:37,549 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0xb zxid:0x1e txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir 2015-01-28 14:49:37,549 INFO org.apache.giraph.bsp.BspService: getApplicationAttempt: Node /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir already exists! 2015-01-28 14:49:37,550 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0xd zxid:0x1f txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir 2015-01-28 14:49:37,559 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x13 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir 2015-01-28 14:49:37,560 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x14 zxid:0x22 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir 2015-01-28 14:49:37,568 INFO org.apache.giraph.io.formats.GiraphFileInputFormat: Total input paths to process : 1 2015-01-28 14:49:37,580 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library not loaded 2015-01-28 14:49:37,586 INFO org.apache.giraph.master.BspServiceMaster: generateVertexInputSplits: Got 1 input splits for 1 input threads 2015-01-28 14:49:37,586 INFO org.apache.giraph.master.BspServiceMaster: createVertexInputSplits: Starting to write input split data to zookeeper with 1 threads 2015-01-28 14:49:37,590 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x18 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_vertexInputSplitDir Error:KeeperErrorCode = NoNode for /_hadoopBsp/job_201501281117_0016/_vertexInputSplitDir 2015-01-28 14:49:37,591 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x19 zxid:0x24 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp 2015-01-28 14:49:37,591 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x1a zxid:0x25 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016 2015-01-28 14:49:37,594 INFO org.apache.giraph.master.BspServiceMaster: createVertexInputSplits: Done writing input split data to zookeeper 2015-01-28 14:49:37,595 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x1e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir 2015-01-28 14:49:37,596 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x1f zxid:0x2a txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir 2015-01-28 14:49:37,605 INFO org.apache.giraph.comm.netty.NettyClient: Using Netty without authentication. 2015-01-28 14:49:37,614 INFO org.apache.giraph.comm.netty.NettyClient: connectAllAddresses: Successfully added 1 connections, (1 total connected) 0 failed, 0 failures total. 2015-01-28 14:49:37,615 INFO org.apache.giraph.partition.PartitionUtils: computePartitionCount: Creating 1, default would have been 1 partitions. 2015-01-28 14:49:37,679 INFO org.apache.giraph.master.BspServiceMaster: barrierOnWorkerList: 0 out of 1 workers finished on superstep -1 on path /_hadoopBsp/job_201501281117_0016/_vertexInputSplitDoneDir 2015-01-28 14:49:37,681 INFO org.apache.giraph.comm.netty.NettyServer: start: Using Netty without authentication. 2015-01-28 14:49:37,683 INFO org.apache.giraph.master.BspServiceMaster: barrierOnWorkerList: Waiting on [fredrik-virtualbox_1] 2015-01-28 14:49:37,779 INFO org.apache.giraph.master.BspServiceMaster: setJobState: {"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1} on superstep -1 2015-01-28 14:49:37,782 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x24 zxid:0x32 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_masterJobState Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_masterJobState 2015-01-28 14:49:37,785 INFO org.apache.giraph.master.BspServiceMaster: setJobState: {"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1} 2015-01-28 14:49:37,785 FATAL org.apache.giraph.master.BspServiceMaster: failJob: Killing job job_201501281117_0016 2015-01-28 14:49:37,785 FATAL org.apache.giraph.master.BspServiceMaster: failJob: exception java.lang.IllegalStateException: ******* WORKERS [Worker(hostname=fredrik-virtualbox, MRtaskID=1, port=30001)] FAILED ******* 2015-01-28 14:49:37,830 ERROR org.apache.giraph.master.MasterThread: masterThread: Master algorithm failed with IllegalStateException java.lang.IllegalStateException: coordinateVertexInputSplits: Worker failed during input split (currently not supported) at org.apache.giraph.master.BspServiceMaster.coordinateInputSplits(BspServiceMaster.java:1499) at org.apache.giraph.master.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1666) at org.apache.giraph.master.MasterThread.run(MasterThread.java:124) 2015-01-28 14:49:37,831 FATAL org.apache.giraph.graph.GraphTaskManager: uncaughtException: OverrideExceptionHandler on thread org.apache.giraph.master.MasterThread, msg = java.lang.IllegalStateException: coordinateVertexInputSplits: Worker failed during input split (currently not supported), exiting... java.lang.IllegalStateException: java.lang.IllegalStateException: coordinateVertexInputSplits: Worker failed during input split (currently not supported) at org.apache.giraph.master.MasterThread.run(MasterThread.java:194) Caused by: java.lang.IllegalStateException: coordinateVertexInputSplits: Worker failed during input split (currently not supported) at org.apache.giraph.master.BspServiceMaster.coordinateInputSplits(BspServiceMaster.java:1499) at org.apache.giraph.master.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1666) at org.apache.giraph.master.MasterThread.run(MasterThread.java:124) *Kinds Regards* Fredrik
