Hi, I've noticied that you used only one worker in your command ( -w 1). Have you tried to set the option giraph.SplitMasterWorker to false ? I am pretty new to Giraph but from what I know, you have to do that when you have only 1 worker.
Yours Sincerely, Flavian 2015-01-28 14:57 GMT+01:00 Fredrik Einarsson <einarsso...@gmail.com>: > Dear Giraph users, > > I am trying to get my Giraph installation to work but I am stuck. > > Below I paste my command and job tracker log entry. I appreciate your > help. > > hadoop jar > /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > -vip /user/fredrik/input/tiny_graph.txt -vof > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > /user/fredrik/output/shortestpaths -w 1 > > Task Logs: 'attempt_201501281117_0016_m_000000_0' > > *stdout logs* > ------------------------------ > > > *stderr logs* > > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [file:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/usr/local/hadoop/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > > ------------------------------ > > > *syslog logs* > > 2015-01-28 14:49:28,422 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded > the native-hadoop library > 2015-01-28 14:49:30,078 WARN > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already > exists! > 2015-01-28 14:49:30,745 INFO org.apache.hadoop.util.ProcessTree: setsid > exited with exit code 0 > 2015-01-28 14:49:30,781 INFO org.apache.hadoop.mapred.Task: Using > ResourceCalculatorPlugin : > org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2bf8e35d > 2015-01-28 14:49:31,022 INFO org.apache.hadoop.mapred.MapTask: Processing > split: 'org.apache.giraph.bsp.BspInputSplit, index=-1, num=-1 > 2015-01-28 14:49:31,270 INFO org.apache.giraph.graph.GraphTaskManager: setup: > Log level remains at info > 2015-01-28 14:49:31,852 INFO org.apache.giraph.zk.ZooKeeperManager: > createCandidateStamp: Made the directory > _bsp/_defaultZkManagerDir/job_201501281117_0016 > 2015-01-28 14:49:31,854 INFO org.apache.giraph.zk.ZooKeeperManager: > createCandidateStamp: Made the directory > _bsp/_defaultZkManagerDir/job_201501281117_0016/_zkServer > 2015-01-28 14:49:31,857 INFO org.apache.giraph.zk.ZooKeeperManager: > createCandidateStamp: Creating my filestamp > _bsp/_defaultZkManagerDir/job_201501281117_0016/_task/fredrik-virtualbox 0 > 2015-01-28 14:49:31,925 INFO org.apache.giraph.zk.ZooKeeperManager: > getZooKeeperServerList: Got [fredrik-virtualbox] 1 hosts from 1 candidates > when 1 required (polling period is 3000) on attempt 0 > 2015-01-28 14:49:31,926 INFO org.apache.giraph.zk.ZooKeeperManager: > createZooKeeperServerList: Creating the final ZooKeeper file > '_bsp/_defaultZkManagerDir/job_201501281117_0016/zkServerList_fredrik-virtualbox > 0 ' > 2015-01-28 14:49:31,955 INFO org.apache.giraph.zk.ZooKeeperManager: > getZooKeeperServerList: For task 0, got file 'zkServerList_fredrik-virtualbox > 0 ' (polling period is 3000) > 2015-01-28 14:49:31,955 INFO org.apache.giraph.zk.ZooKeeperManager: > getZooKeeperServerList: Found [fredrik-virtualbox, 0] 2 hosts in filename > 'zkServerList_fredrik-virtualbox 0 ' > 2015-01-28 14:49:31,956 INFO org.apache.giraph.zk.ZooKeeperManager: > onlineZooKeeperServers: Trying to delete old directory > /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper > 2015-01-28 14:49:31,969 INFO org.apache.giraph.zk.ZooKeeperManager: > generateZooKeeperConfigFile: Creating file > /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg > in > /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper > with base port 22181 > 2015-01-28 14:49:31,970 INFO org.apache.giraph.zk.ZooKeeperManager: > generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true > 2015-01-28 14:49:31,970 INFO org.apache.giraph.zk.ZooKeeperManager: > generateZooKeeperConfigFile: Delete of zoo.cfg = false > 2015-01-28 14:49:31,989 INFO org.apache.giraph.zk.ZooKeeperManager: > onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect to > fredrik-virtualbox:22181 with poll msecs = 3000 > 2015-01-28 14:49:32,000 WARN org.apache.giraph.zk.ZooKeeperManager: > onlineZooKeeperServers: Got ConnectException > java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) > at > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at > org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:601) > at > org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:433) > at > org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:231) > at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:55) > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > 2015-01-28 14:49:32,018 INFO > org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading configuration > from: > /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg > 2015-01-28 14:49:32,029 INFO > org.apache.zookeeper.server.DatadirCleanupManager: autopurge.snapRetainCount > set to 3 > 2015-01-28 14:49:32,029 INFO > org.apache.zookeeper.server.DatadirCleanupManager: autopurge.purgeInterval > set to 0 > 2015-01-28 14:49:32,029 INFO > org.apache.zookeeper.server.DatadirCleanupManager: Purge task is not > scheduled. > 2015-01-28 14:49:32,035 WARN org.apache.giraph.zk.InProcessZooKeeperRunner: > Either no config or no quorum defined in config, running in standalone mode > 2015-01-28 14:49:32,154 INFO > org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading configuration > from: > /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg > 2015-01-28 14:49:32,154 INFO org.apache.zookeeper.server.ZooKeeperServerMain: > Starting server > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 > GMT > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:host.name=fredrik-VirtualBox > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:java.version=1.7.0_65 > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:java.vendor=Oracle Corporation > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server > environment:java.class.path=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/classes:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work:/usr/local/hadoop/libexec/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/libexec/..:/usr/local/hadoop/libexec/../hadoop-core-1.2.1.jar:/usr/local/hadoop/libexec/../lib/asm-3.2.jar:/usr/local/hadoop/libexec/../lib/aspectjrt-1.6.11.jar:/usr/local/hadoop/libexec/../lib/aspectjtools-1.6.11.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-net-3.1.jar:/usr/local/hadoop/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-capacity-scheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-fairscheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-thriftfs-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop/libexec/../lib/junit-4.5.jar:/usr/local/hadoop/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-api-2.1.jar > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server > environment:java.library.path=/usr/local/hadoop/libexec/../lib/native/Linux-amd64-64:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server > environment:java.io.tmpdir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work/tmp > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:java.compiler=<NA> > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:os.name=Linux > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:os.arch=amd64 > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:os.version=3.16.0-29-generic > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:user.name=fredrik > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server environment:user.home=/home/fredrik > 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: > Server > environment:user.dir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work > 2015-01-28 14:49:32,215 INFO org.apache.zookeeper.server.ZooKeeperServer: > tickTime set to 6000 > 2015-01-28 14:49:32,215 INFO org.apache.zookeeper.server.ZooKeeperServer: > minSessionTimeout set to 600000 > 2015-01-28 14:49:32,216 INFO org.apache.zookeeper.server.ZooKeeperServer: > maxSessionTimeout set to 900000 > 2015-01-28 14:49:32,266 INFO > org.apache.zookeeper.server.NIOServerCnxnFactory: binding to port > 0.0.0.0/0.0.0.0:22181 > 2015-01-28 <http://0.0.0.0/0.0.0.0:221812015-01-28> 14:49:32,337 INFO > org.apache.zookeeper.server.persistence.FileTxnSnapLog: Snapshotting: 0x0 to > /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/version-2/snapshot.0 > 2015-01-28 14:49:32,414 INFO > org.apache.zookeeper.server.PrepRequestProcessor: zookeeper.skipACL=="yes", > ACL checks will be skipped > 2015-01-28 14:49:35,030 INFO org.apache.giraph.zk.ZooKeeperManager: > onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect to > fredrik-virtualbox:22181 with poll msecs = 3000 > 2015-01-28 14:49:35,030 INFO org.apache.giraph.zk.ZooKeeperManager: > onlineZooKeeperServers: Connected to fredrik-VirtualBox/192.168.56.10:22181! > 2015-01-28 14:49:35,031 INFO org.apache.giraph.zk.ZooKeeperManager: > onlineZooKeeperServers: Creating my filestamp > _bsp/_defaultZkManagerDir/job_201501281117_0016/_zkServer/fredrik-virtualbox 0 > 2015-01-28 14:49:35,031 INFO > org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection > from /192.168.56.10:34203 > 2015-01-28 14:49:35,798 INFO > org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection > from /192.168.56.10:34204 > 2015-01-28 14:49:35,804 INFO org.apache.zookeeper.server.ZooKeeperServer: > Client attempting to establish new session at /192.168.56.10:34204 > 2015-01-28 14:49:35,807 INFO > org.apache.zookeeper.server.persistence.FileTxnLog: Creating new log file: > log.1 > 2015-01-28 14:49:35,825 INFO org.apache.zookeeper.server.ZooKeeperServer: > Established session 0x14b30cddb450000 with negotiated timeout 600000 for > client /192.168.56.10:34204 > 2015-01-28 14:49:35,989 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x1 zxid:0x2 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NoNode for > /_hadoopBsp/job_201501281117_0016 > 2015-01-28 14:49:36,012 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x8 zxid:0x7 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0 > Error:KeeperErrorCode = NoNode for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0 > 2015-01-28 14:49:36,013 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x9 zxid:0x8 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp > Error:KeeperErrorCode = NodeExists for /_hadoopBsp > 2015-01-28 14:49:36,013 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0xa zxid:0x9 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016 > 2015-01-28 14:49:36,014 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0xb zxid:0xa txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir > 2015-01-28 14:49:36,023 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0xf zxid:0xd txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir > Error:KeeperErrorCode = NoNode for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir > 2015-01-28 14:49:36,025 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x10 zxid:0xe txntype:-1 reqpath:n/a Error Path:/_hadoopBsp > Error:KeeperErrorCode = NodeExists for /_hadoopBsp > 2015-01-28 14:49:36,027 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x11 zxid:0xf txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016 > 2015-01-28 14:49:36,028 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x12 zxid:0x10 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir > 2015-01-28 14:49:36,029 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x13 zxid:0x11 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0 > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0 > 2015-01-28 14:49:36,030 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x14 zxid:0x12 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir > 2015-01-28 14:49:36,415 WARN org.apache.zookeeper.server.NIOServerCnxn: > caught end of stream exception > EndOfStreamException: Unable to read additional data from client sessionid > 0x0, likely client has closed socket > at > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220) > at > org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) > at java.lang.Thread.run(Thread.java:745) > 2015-01-28 14:49:36,415 INFO org.apache.zookeeper.server.NIOServerCnxn: > Closed socket connection for client /192.168.56.10:34203 (no session > established for client) > 2015-01-28 14:49:37,361 INFO org.apache.giraph.graph.GraphTaskManager: setup: > Chosen to run ZooKeeper... > 2015-01-28 14:49:37,362 INFO org.apache.giraph.graph.GraphTaskManager: setup: > Starting up BspServiceMaster (master thread)... > 2015-01-28 14:49:37,374 INFO org.apache.giraph.bsp.BspService: BspService: > Path to create to halt is /_hadoopBsp/job_201501281117_0016/_haltComputation > 2015-01-28 14:49:37,374 INFO org.apache.giraph.bsp.BspService: BspService: > Connecting to ZooKeeper with job job_201501281117_0016, 0 on > fredrik-virtualbox:22181 > 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client > environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT > 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client > environment:host.name=fredrik-VirtualBox > 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.version=1.7.0_65 > 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.vendor=Oracle Corporation > 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre > 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.class.path=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/classes:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work:/usr/local/hadoop/libexec/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/libexec/..:/usr/local/hadoop/libexec/../hadoop-core-1.2.1.jar:/usr/local/hadoop/libexec/../lib/asm-3.2.jar:/usr/local/hadoop/libexec/../lib/aspectjrt-1.6.11.jar:/usr/local/hadoop/libexec/../lib/aspectjtools-1.6.11.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-net-3.1.jar:/usr/local/hadoop/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-capacity-scheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-fairscheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-thriftfs-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop/libexec/../lib/junit-4.5.jar:/usr/local/hadoop/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-api-2.1.jar > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.library.path=/usr/local/hadoop/libexec/../lib/native/Linux-amd64-64:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.io.tmpdir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work/tmp > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.compiler=<NA> > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.name=Linux > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.arch=amd64 > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.version=3.16.0-29-generic > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.name=fredrik > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.home=/home/fredrik > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.dir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work > 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Initiating > client connection, connectString=fredrik-virtualbox:22181 > sessionTimeout=60000 watcher=org.apache.giraph.master.BspServiceMaster@f9aac17 > 2015-01-28 14:49:37,393 INFO org.apache.zookeeper.ClientCnxn: Opening socket > connection to server fredrik-VirtualBox/192.168.56.10:22181. Will not attempt > to authenticate using SASL (unknown error) > 2015-01-28 14:49:37,393 INFO > org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection > from /192.168.56.10:34205 > 2015-01-28 14:49:37,394 INFO org.apache.zookeeper.ClientCnxn: Socket > connection established to fredrik-VirtualBox/192.168.56.10:22181, initiating > session > 2015-01-28 14:49:37,394 INFO org.apache.zookeeper.server.ZooKeeperServer: > Client attempting to establish new session at /192.168.56.10:34205 > 2015-01-28 14:49:37,395 INFO org.apache.zookeeper.server.ZooKeeperServer: > Established session 0x14b30cddb450001 with negotiated timeout 600000 for > client /192.168.56.10:34205 > 2015-01-28 14:49:37,395 INFO org.apache.zookeeper.ClientCnxn: Session > establishment complete on server fredrik-VirtualBox/192.168.56.10:22181, > sessionid = 0x14b30cddb450001, negotiated timeout = 600000 > 2015-01-28 14:49:37,396 INFO org.apache.giraph.bsp.BspService: process: > Asynchronous connection complete. > 2015-01-28 14:49:37,402 INFO org.apache.giraph.graph.GraphTaskManager: map: > No need to do anything when not a worker > 2015-01-28 14:49:37,402 INFO org.apache.giraph.graph.GraphTaskManager: > cleanup: Starting for MASTER_ZOOKEEPER_ONLY > 2015-01-28 14:49:37,404 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x1 zxid:0x17 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_masterElectionDir > Error:KeeperErrorCode = NoNode for > /_hadoopBsp/job_201501281117_0016/_masterElectionDir > 2015-01-28 14:49:37,405 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x2 zxid:0x18 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp > Error:KeeperErrorCode = NodeExists for /_hadoopBsp > 2015-01-28 14:49:37,406 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x3 zxid:0x19 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016 > 2015-01-28 14:49:37,409 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x6 zxid:0x1c txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_masterJobState Error:KeeperErrorCode > = NodeExists for /_hadoopBsp/job_201501281117_0016/_masterJobState > 2015-01-28 14:49:37,409 INFO org.apache.giraph.bsp.BspService: getJobState: > Job state already exists (/_hadoopBsp/job_201501281117_0016/_masterJobState) > 2015-01-28 14:49:37,411 INFO org.apache.giraph.master.BspServiceMaster: > becomeMaster: First child is > '/_hadoopBsp/job_201501281117_0016/_masterElectionDir/fredrik-virtualbox_00000000000' > and my bid is > '/_hadoopBsp/job_201501281117_0016/_masterElectionDir/fredrik-virtualbox_00000000000' > 2015-01-28 14:49:37,414 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x9 zxid:0x1d txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir > 2015-01-28 14:49:37,414 INFO org.apache.giraph.bsp.BspService: > getApplicationAttempt: Node > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir already exists! > 2015-01-28 14:49:37,477 INFO org.apache.giraph.comm.netty.NettyServer: > NettyServer: Using execution group with 8 threads for requestFrameDecoder. > 2015-01-28 14:49:37,537 INFO org.apache.giraph.comm.netty.NettyServer: start: > Started server communication server: fredrik-VirtualBox/192.168.56.10:30000 > with up to 16 threads on bind attempt 0 with sendBufferSize = 32768 > receiveBufferSize = 524288 > 2015-01-28 14:49:37,545 INFO org.apache.giraph.comm.netty.NettyClient: > NettyClient: Using execution handler with 8 threads after request-encoder. > 2015-01-28 14:49:37,548 INFO org.apache.giraph.master.BspServiceMaster: > becomeMaster: I am now the master! > 2015-01-28 14:49:37,549 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0xb zxid:0x1e txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir > 2015-01-28 14:49:37,549 INFO org.apache.giraph.bsp.BspService: > getApplicationAttempt: Node > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir already exists! > 2015-01-28 14:49:37,550 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0xd zxid:0x1f txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir > 2015-01-28 14:49:37,559 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x13 zxid:0x21 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir > 2015-01-28 14:49:37,560 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x14 zxid:0x22 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir > 2015-01-28 14:49:37,568 INFO > org.apache.giraph.io.formats.GiraphFileInputFormat: Total input paths to > process : 1 > 2015-01-28 14:49:37,580 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: > Snappy native library not loaded > 2015-01-28 14:49:37,586 INFO org.apache.giraph.master.BspServiceMaster: > generateVertexInputSplits: Got 1 input splits for 1 input threads > 2015-01-28 14:49:37,586 INFO org.apache.giraph.master.BspServiceMaster: > createVertexInputSplits: Starting to write input split data to zookeeper with > 1 threads > 2015-01-28 14:49:37,590 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x18 zxid:0x23 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_vertexInputSplitDir > Error:KeeperErrorCode = NoNode for > /_hadoopBsp/job_201501281117_0016/_vertexInputSplitDir > 2015-01-28 14:49:37,591 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x19 zxid:0x24 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp > Error:KeeperErrorCode = NodeExists for /_hadoopBsp > 2015-01-28 14:49:37,591 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x1a zxid:0x25 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016 > 2015-01-28 14:49:37,594 INFO org.apache.giraph.master.BspServiceMaster: > createVertexInputSplits: Done writing input split data to zookeeper > 2015-01-28 14:49:37,595 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x1e zxid:0x29 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir > 2015-01-28 14:49:37,596 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450001 type:create > cxid:0x1f zxid:0x2a txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir > Error:KeeperErrorCode = NodeExists for > /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir > 2015-01-28 14:49:37,605 INFO org.apache.giraph.comm.netty.NettyClient: Using > Netty without authentication. > 2015-01-28 14:49:37,614 INFO org.apache.giraph.comm.netty.NettyClient: > connectAllAddresses: Successfully added 1 connections, (1 total connected) 0 > failed, 0 failures total. > 2015-01-28 14:49:37,615 INFO org.apache.giraph.partition.PartitionUtils: > computePartitionCount: Creating 1, default would have been 1 partitions. > 2015-01-28 14:49:37,679 INFO org.apache.giraph.master.BspServiceMaster: > barrierOnWorkerList: 0 out of 1 workers finished on superstep -1 on path > /_hadoopBsp/job_201501281117_0016/_vertexInputSplitDoneDir > 2015-01-28 14:49:37,681 INFO org.apache.giraph.comm.netty.NettyServer: start: > Using Netty without authentication. > 2015-01-28 14:49:37,683 INFO org.apache.giraph.master.BspServiceMaster: > barrierOnWorkerList: Waiting on [fredrik-virtualbox_1] > 2015-01-28 14:49:37,779 INFO org.apache.giraph.master.BspServiceMaster: > setJobState: > {"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1} on > superstep -1 > 2015-01-28 14:49:37,782 INFO > org.apache.zookeeper.server.PrepRequestProcessor: Got user-level > KeeperException when processing sessionid:0x14b30cddb450000 type:create > cxid:0x24 zxid:0x32 txntype:-1 reqpath:n/a Error > Path:/_hadoopBsp/job_201501281117_0016/_masterJobState Error:KeeperErrorCode > = NodeExists for /_hadoopBsp/job_201501281117_0016/_masterJobState > 2015-01-28 14:49:37,785 INFO org.apache.giraph.master.BspServiceMaster: > setJobState: > {"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1} > 2015-01-28 14:49:37,785 FATAL org.apache.giraph.master.BspServiceMaster: > failJob: Killing job job_201501281117_0016 > 2015-01-28 14:49:37,785 FATAL org.apache.giraph.master.BspServiceMaster: > failJob: exception java.lang.IllegalStateException: ******* WORKERS > [Worker(hostname=fredrik-virtualbox, MRtaskID=1, port=30001)] FAILED ******* > 2015-01-28 14:49:37,830 ERROR org.apache.giraph.master.MasterThread: > masterThread: Master algorithm failed with IllegalStateException > java.lang.IllegalStateException: coordinateVertexInputSplits: Worker failed > during input split (currently not supported) > at > org.apache.giraph.master.BspServiceMaster.coordinateInputSplits(BspServiceMaster.java:1499) > at > org.apache.giraph.master.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1666) > at org.apache.giraph.master.MasterThread.run(MasterThread.java:124) > 2015-01-28 14:49:37,831 FATAL org.apache.giraph.graph.GraphTaskManager: > uncaughtException: OverrideExceptionHandler on thread > org.apache.giraph.master.MasterThread, msg = java.lang.IllegalStateException: > coordinateVertexInputSplits: Worker failed during input split (currently not > supported), exiting... > java.lang.IllegalStateException: java.lang.IllegalStateException: > coordinateVertexInputSplits: Worker failed during input split (currently not > supported) > at org.apache.giraph.master.MasterThread.run(MasterThread.java:194) > Caused by: java.lang.IllegalStateException: coordinateVertexInputSplits: > Worker failed during input split (currently not supported) > at > org.apache.giraph.master.BspServiceMaster.coordinateInputSplits(BspServiceMaster.java:1499) > at > org.apache.giraph.master.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1666) > at org.apache.giraph.master.MasterThread.run(MasterThread.java:124) > > > > *Kinds Regards* > Fredrik >
