I have tried to perform my job but after some updates in Hama I have received the following message:
Messages must never be behind the vertex in ID! Current Message ID: 13064 vs. 13079 I'm using the Hama-Trunk. The complete log is below. hduser @ node100: ~ / hama-2 $ time bin / hama jar FastGDC_4.jar 40001 / hdfs/disco/dados/v40mila200mil / hdfs/disco/v40mila200_out 18 30 Mortbay.log 04/13/17 09:02:19 INFO: Logging to org.slf4j.impl.Log4jLoggerAdapter (org.mortbay.log) via org.mortbay.log.Slf4jLog Bsp.FileInputFormat 04/13/17 09:02:20 INFO: Total input paths to process: 1 04/13/17 09:02:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform ... using builtin-java classes where applicable 04/13/17 09:02:20 WARN snappy.LoadSnappy: Snappy native library not loaded Bsp.FileInputFormat 04/13/17 09:02:20 INFO: Total input paths to process: 1 Bsp.BSPJobClient 04/13/17 09:02:27 INFO: Running job: job_201304170858_0002 04/13/17 09:02:30 INFO bsp.BSPJobClient: Current supersteps number: 0 04/13/17 09:02:39 INFO bsp.BSPJobClient: Current supersteps number: 2 04/13/17 09:02:42 INFO bsp.BSPJobClient: The overall number of supersteps: 2 Bsp.BSPJobClient 04/13/17 09:02:42 INFO: Counters: 6 04/13/17 09:02:42 INFO bsp.BSPJobClient: $ org.apache.hama.bsp.JobInProgress JobCounter 04/13/17 09:02:42 INFO bsp.BSPJobClient: SUPERSTEPS = 2 04/13/17 09:02:42 INFO bsp.BSPJobClient: LAUNCHED_TASKS = 1 04/13/17 09:02:42 INFO bsp.BSPJobClient: $ org.apache.hama.bsp.BSPPeerImpl PeerCounter 04/13/17 09:02:42 INFO bsp.BSPJobClient: SUPERSTEP_SUM = 2 04/13/17 09:02:42 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS = 68 04/13/17 09:02:42 INFO bsp.BSPJobClient: IO_BYTES_READ = 1411919 04/13/17 09:02:42 INFO bsp.BSPJobClient: TASK_INPUT_RECORDS = 39688 Bsp.FileInputFormat 04/13/17 09:02:42 INFO: Total input paths to process: 18 Bsp.BSPJobClient 04/13/17 09:02:45 INFO: Running job: job_201304170858_0001 04/13/17 09:02:48 INFO bsp.BSPJobClient: Current supersteps number: 0 04/13/17 09:03:15 INFO bsp.BSPJobClient: Current supersteps number: 1 04/13/17 09:05:34 INFO bsp.BSPJobClient: Current supersteps number: 2 attempt_201304170858_0001_000003_0: SLF4J: Class path contains multiple SLF4J bindings. attempt_201304170858_0001_000003_0: SLF4J: Found binding in [jar: file :/ home/hduser/hama-2/lib/slf4j-log4j12-1.5.8.jar! / org/slf4j/impl/StaticLoggerBinder.class] attempt_201304170858_0001_000003_0: SLF4J: Found binding in attempt_201304170858_0001_000003_0: SLF4J: See http://www.slf4j.org/codes.html # multiple_bindings for an explanation. attempt_201304170858_0001_000003_0: mortbay.log 04/13/17 08:35:42 INFO: Logging to org.slf4j.impl.Log4jLoggerAdapter (org.mortbay.log) via org.mortbay.log.Slf4jLog attempt_201304170858_0001_000003_0: sync.ZKSyncClient 04/13/17 08:35:51 INFO: Initializing ZK Sync Client attempt_201304170858_0001_000003_0: 04/13/17 08:35:51 INFO sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At node109/ 10.16.5.109: 61001 attempt_201304170858_0001_000003_0: 04/13/17 08:35:52 ERROR sync.ZooKeeperSyncClientImpl: $ org.apache.zookeeper.KeeperException NodeExistsException: KeeperErrorCode = NodeExists for / bsp/job_201304170858_0001/peers attempt_201304170858_0001_000003_0: ipc.Server 04/13/17 08:35:54 INFO: Starting SocketReader attempt_201304170858_0001_000003_0: 04/13/17 08:35:54 INFO ipc.Server: IPC Server Reply: starting attempt_201304170858_0001_000003_0: 04/13/17 08:35:54 INFO ipc.Server: IPC Server listener on 61001: starting attempt_201304170858_0001_000003_0: 04/13/17 08:35:54 INFO message.HadoopMessageManagerImpl: BSPPeer address: port node109: 61001 attempt_201304170858_0001_000003_0: 04/13/17 08:35:54 INFO ipc.Server: IPC Server handler 0 on 61001: starting attempt_201304170858_0001_000003_0: 04/13/17 08:36:06 INFO graph.GraphJobRunner: 2200 vertices are loaded into node109: 61001 attempt_201304170858_0001_000003_0: bsp.BSPTask 04/13/17 08:38:23 ERROR: Error running bsp bsp setup and function. attempt_201304170858_0001_000003_0: java.lang.IllegalArgumentException: Messages must never be behind the vertex in ID! Current Message ID: 13064 vs. 13079 attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.iterate (GraphJobRunner.java: 280) attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.doSuperstep (GraphJobRunner.java: 226) attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.bsp (GraphJobRunner.java: 130) attempt_201304170858_0001_000003_0: at org.apache.hama.bsp.BSPTask.runBSP (BSPTask.java: 177) attempt_201304170858_0001_000003_0: at org.apache.hama.bsp.BSPTask.run (BSPTask.java: 146) attempt_201304170858_0001_000003_0: org.apache.hama.bsp.GroomServer at $ BSPPeerChild.main (GroomServer.java: 1262) attempt_201304170858_0001_000003_0: ipc.Server 04/13/17 08:38:29 INFO: Stopping server on 61001 attempt_201304170858_0001_000003_0: ipc.Server 04/13/17 08:38:29 INFO: Stopping IPC Server listener on 61001 attempt_201304170858_0001_000003_0: 04/13/17 08:38:29 INFO ipc.Server: IPC Server handler 0 on 61001: exiting attempt_201304170858_0001_000003_0: 04/13/17 08:38:29 INFO metrics.RpcInstrumentation: shut down attempt_201304170858_0001_000003_0: 04/13/17 08:38:29 ERROR bsp.BSPTask: Shutting down ping service. attempt_201304170858_0001_000003_0: 04/13/17 08:38:29 bsp.GroomServer FATAL: Error running child attempt_201304170858_0001_000003_0: java.lang.IllegalArgumentException: Messages must never be behind the vertex in ID! Current Message ID: 13064 vs. 13079 attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.iterate (GraphJobRunner.java: 280) attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.doSuperstep (GraphJobRunner.java: 226) attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.bsp (GraphJobRunner.java: 130) attempt_201304170858_0001_000003_0: at org.apache.hama.bsp.BSPTask.runBSP (BSPTask.java: 177) attempt_201304170858_0001_000003_0: at org.apache.hama.bsp.BSPTask.run (BSPTask.java: 146) attempt_201304170858_0001_000003_0: org.apache.hama.bsp.GroomServer at $ BSPPeerChild.main (GroomServer.java: 1262) attempt_201304170858_0001_000003_0: ipc.Server 04/13/17 08:38:29 INFO: Stopping IPC Server Reply attempt_201304170858_0001_000003_0: java.lang.IllegalArgumentException: Messages must never be behind the vertex in ID! Current Message ID: 13064 vs. 13079 attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.iterate (GraphJobRunner.java: 280) attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.doSuperstep (GraphJobRunner.java: 226) attempt_201304170858_0001_000003_0: at org.apache.hama.graph.GraphJobRunner.bsp (GraphJobRunner.java: 130) attempt_201304170858_0001_000003_0: at org.apache.hama.bsp.BSPTask.runBSP (BSPTask.java: 177) attempt_201304170858_0001_000003_0: at org.apache.hama.bsp.BSPTask.run (BSPTask.java: 146) attempt_201304170858_0001_000003_0: org.apache.hama.bsp.GroomServer at $ BSPPeerChild.main (GroomServer.java: 1262) 04/13/17 09:06:42 INFO bsp.BSPJobClient: Job failed. real 4m28.022s -- Francisco Sanches
