Please use Hadoop RPC instead of Avro. On Fri, Oct 19, 2012 at 4:37 PM, Sandy Ding <[email protected]> wrote: > As follows, still no specific error info > > STARTUP_MSG: Starting BSPMaster > STARTUP_MSG: host = debian-sandy/127.0.1.1 > STARTUP_MSG: args = [] > STARTUP_MSG: version = 1.0.0 > STARTUP_MSG: build = > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r > 1214675; compiled by 'hortonfo' on Fri Dec 16 20:01:27 UTC 2011 > ************************************************************/ > 2012-10-19 13:51:21,855 INFO org.apache.hama.bsp.BSPMaster: RPC BSPMaster: > host localhost port 40000 > 2012-10-19 13:51:22,078 INFO org.apache.hadoop.ipc.Server: Starting > SocketReader > 2012-10-19 13:51:22,148 INFO org.mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 2012-10-19 13:51:22,188 INFO org.apache.hama.http.HttpServer: Port returned > by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening > the listener on 40013 > 2012-10-19 13:51:22,189 INFO org.apache.hama.http.HttpServer: > listener.getLocalPort() returned 40013 > webServer.getConnectors()[0].getLocalPort() returned 40013 > 2012-10-19 13:51:22,189 INFO org.apache.hama.http.HttpServer: Jetty bound > to port 40013 > 2012-10-19 13:51:22,189 INFO org.mortbay.log: jetty-6.1.14 > 2012-10-19 13:51:22,416 INFO org.mortbay.log: Started > SelectChannelConnector@localhost:40013 > 2012-10-19 13:51:22,565 INFO org.apache.hama.bsp.BSPMaster: Cleaning up the > system directory > 2012-10-19 13:51:22,565 INFO org.apache.hama.bsp.BSPMaster: > hdfs://localhost:54310/tmp/bsp/system > 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 GMT > 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:host.name=debian-sandy.localdomain > 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.version=1.7.0_07 > 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.vendor=Oracle Corporation > 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.home=/usr/lib/sun-jdk1.7/jre > 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.class.path=/home/sandy/workspace/hama-trunk/bin/../conf:/usr/lib/sun-jdk1.7/lib/tools.jar:/home/sandy/workspace/hama-trunk/bin/../core/target/classes:/home/sandy/workspace/hama-trunk/bin/../graph/target/classes:/home/sandy/workspace/hama-trunk/bin/../hama-**.jar:/home/sandy/workspace/hama-trunk/bin/../lib/ant-1.7.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/ant-launcher-1.7.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/avro-1.6.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/avro-ipc-1.6.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-cli-1.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-configuration-1.7.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-lang-2.6.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-logging-1.1.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-math3-3.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/guava-10.0.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/hadoop-core-1.0.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/hadoop-test-1.0.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jackson-core-asl-1.9.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jackson-mapper-asl-1.9.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-annotations-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-util-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jsp-2.1-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jsp-api-2.1-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/junit-4.8.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/log4j-1.2.16.jar:/home/sandy/workspace/hama-trunk/bin/../lib/netty-3.2.6.Final.jar:/home/sandy/workspace/hama-trunk/bin/../lib/servlet-api-6.0.32.jar:/home/sandy/workspace/hama-trunk/bin/../lib/slf4j-api-1.5.8.jar:/home/sandy/workspace/hama-trunk/bin/../lib/slf4j-log4j12-1.5.8.jar:/home/sandy/workspace/hama-trunk/bin/../lib/snappy-java-1.0.4.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/zookeeper-3.3.3.jar > 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > 2012-10-19 13:51:22,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.io.tmpdir=/tmp > 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.compiler=<NA> > 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.name=Linux > 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.arch=amd64 > 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.version=2.6.32-5-amd64 > 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.name=sandy > 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.home=/home/sandy > 2012-10-19 13:51:22,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.dir=/home/sandy/workspace/hama-trunk > 2012-10-19 13:51:22,810 INFO org.apache.zookeeper.ZooKeeper: Initiating > client connection, connectString=localhost:21810 sessionTimeout=24000000 > watcher=org.apache.hama.bsp.BSPMaster@3c75339f > 2012-10-19 13:51:22,819 INFO org.apache.zookeeper.ClientCnxn: Opening > socket connection to server localhost/127.0.0.1:21810 > 2012-10-19 13:51:22,819 INFO org.apache.zookeeper.ClientCnxn: Socket > connection established to localhost/127.0.0.1:21810, initiating session > 2012-10-19 13:51:22,837 INFO org.apache.zookeeper.ClientCnxn: Session > establishment complete on server localhost/127.0.0.1:21810, sessionid = > 0x13a77936b710000, negotiated timeout = 24000000 > 2012-10-19 13:51:22,870 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > 2012-10-19 13:51:22,870 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 40000: starting > 2012-10-19 13:51:22,871 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 0 on 40000: starting > 2012-10-19 13:51:22,873 INFO org.apache.hama.bsp.BSPMaster: Starting RUNNING > 2012-10-19 13:51:31,107 INFO org.apache.hama.bsp.BSPMaster: > groomd_debian-sandy.localdomain_50000 is added. > 2012-10-19 13:51:31,753 WARN org.apache.hadoop.util.NativeCodeLoader: > Unable to load native-hadoop library for your platform... using > builtin-java classes where applicable > 2012-10-19 13:51:31,910 INFO org.apache.hama.bsp.JobInProgress: num > BSPTasks: 3 > 2012-10-19 13:51:31,934 INFO org.apache.hama.bsp.JobInProgress: Job is > initialized. > 2012-10-19 13:51:33,176 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0001_000001_0' has failed. > 2012-10-19 13:51:33,176 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0001_000001_0' has failed. > 2012-10-19 13:51:33,177 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0001_000001' has failed. > 2012-10-19 13:51:33,326 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 13:51:33,332 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0001_000000_0' has failed. > 2012-10-19 13:51:33,332 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0001_000000' has failed. > 2012-10-19 13:51:33,338 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 13:51:33,348 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0001_000002_0' has failed. > 2012-10-19 13:51:33,348 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0001_000002' has failed. > 2012-10-19 13:51:33,349 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 13:52:07,216 INFO org.apache.hama.bsp.JobInProgress: num > BSPTasks: 3 > 2012-10-19 13:52:07,219 INFO org.apache.hama.bsp.JobInProgress: Job is > initialized. > 2012-10-19 13:52:37,322 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0002_000002_0' has failed. > 2012-10-19 13:52:37,334 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0002_000002' has failed. > 2012-10-19 13:52:38,040 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 13:52:38,507 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0002_000001_0' has failed. > 2012-10-19 13:52:38,507 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0002_000001' has failed. > 2012-10-19 13:52:39,276 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 14:01:32,449 INFO org.apache.hama.bsp.JobInProgress: num > BSPTasks: 3 > 2012-10-19 14:01:32,733 INFO org.apache.hama.bsp.JobInProgress: Job is > initialized. > 2012-10-19 14:01:42,770 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0003_000002_0' has failed. > 2012-10-19 14:01:42,770 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0003_000002' has failed. > 2012-10-19 14:01:43,047 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 14:01:43,137 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0003_000000_0' has failed. > 2012-10-19 14:01:43,137 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0003_000000' has failed. > 2012-10-19 14:01:43,146 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 14:01:43,159 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0003_000001_0' has failed. > 2012-10-19 14:01:43,159 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0003_000001' has failed. > 2012-10-19 14:01:43,167 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 14:07:11,209 INFO org.apache.hama.bsp.JobInProgress: num > BSPTasks: 3 > 2012-10-19 14:07:11,311 INFO org.apache.hama.bsp.JobInProgress: Job is > initialized. > 2012-10-19 14:07:23,655 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0004_000002_0' has failed. > 2012-10-19 14:07:23,670 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0004_000002' has failed. > 2012-10-19 14:07:23,706 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 14:07:23,728 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0004_000000_0' has failed. > 2012-10-19 14:07:23,728 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0004_000000' has failed. > 2012-10-19 14:07:23,729 INFO org.apache.hama.bsp.JobInProgress: Job failed. > 2012-10-19 14:07:23,737 INFO org.apache.hama.bsp.JobInProgress: Taskid > 'attempt_201210191351_0004_000001_0' has failed. > 2012-10-19 14:07:23,737 INFO org.apache.hama.bsp.TaskInProgress: Task > 'task_201210191351_0004_000001' has failed. > 2012-10-19 14:07:23,738 INFO org.apache.hama.bsp.JobInProgress: Job failed. > > > 2012/10/19 Thomas Jungblut <[email protected]> > >> Can you show the bsp master log? >> >> 2012/10/19 Sandy Ding <[email protected]> >> >> > Hi, all >> > >> > I run pagerank under the pseudo distributed mode with the web-google.txt >> > input. >> > But the job kept failed without error log infomation. The tasklog is as >> > follows: >> > What might be wrong? >> > >> > >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:zookeeper.version=3.3.3-1073969, built on 02/23/2011 22:27 >> GMT >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client environment:host.name >> > =debian-sandy.localdomain >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:java.version=1.7.0_07 >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:java.vendor=Oracle Corporation >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:java.home=/usr/lib/sun-jdk1.7/jre >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > >> > >> environment:java.class.path=/home/sandy/workspace/hama-trunk/bin/../conf:/usr/lib/sun-jdk1.7/lib/tools.jar:/home/sandy/workspace/hama-trunk/bin/../core/target/classes:/home/sandy/workspace/hama-trunk/bin/../graph/target/classes:/home/sandy/workspace/hama-trunk/bin/../hama-**.jar:/home/sandy/workspace/hama-trunk/bin/../lib/ant-1.7.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/ant-launcher-1.7.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/avro-1.6.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/avro-ipc-1.6.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-cli-1.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-configuration-1.7.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-lang-2.6.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-logging-1.1.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/commons-math3-3.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/guava-10.0.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/hadoop-core-1.0.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/hadoop-test-1.0.0.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jackson-core-asl-1.9.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jackson-mapper-asl-1.9.2.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-annotations-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jetty-util-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jsp-2.1-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/jsp-api-2.1-6.1.14.jar:/home/sandy/workspace/hama-trunk/bin/../lib/junit-4.8.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/log4j-1.2.16.jar:/home/sandy/workspace/hama-trunk/bin/../lib/netty-3.2.6.Final.jar:/home/sandy/workspace/hama-trunk/bin/../lib/servlet-api-6.0.32.jar:/home/sandy/workspace/hama-trunk/bin/../lib/slf4j-api-1.5.8.jar:/home/sandy/workspace/hama-trunk/bin/../lib/slf4j-log4j12-1.5.8.jar:/home/sandy/workspace/hama-trunk/bin/../lib/snappy-java-1.0.4.1.jar:/home/sandy/workspace/hama-trunk/bin/../lib/zookeeper-3.3.3.jar::/tmp/bsp-local/groomServer/attempt_201210191351_0003_000001_0/work/classes:/tmp/bsp-local/groomServer/attempt_201210191351_0003_000001_0/work >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > >> > >> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:java.io.tmpdir=/tmp >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:java.compiler=<NA> >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client environment:os.name >> > =Linux >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:os.arch=amd64 >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:os.version=2.6.32-5-amd64 >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client environment:user.name >> > =sandy >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > environment:user.home=/home/sandy >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Client >> > >> > >> environment:user.dir=/tmp/bsp-local/groomServer/attempt_201210191351_0003_000001_0/work >> > 12/10/19 14:01:33 INFO zookeeper.ZooKeeper: Initiating client connection, >> > connectString=localhost:21810 sessionTimeout=24000000 >> > watcher=org.apache.hama.bsp.sync.ZooKeeperSyncClientImpl@67af0d15 >> > 12/10/19 14:01:34 INFO zookeeper.ClientCnxn: Opening socket connection to >> > server localhost/127.0.0.1:21810 >> > 12/10/19 14:01:34 INFO zookeeper.ClientCnxn: Socket connection >> established >> > to localhost/127.0.0.1:21810, initiating session >> > 12/10/19 14:01:34 INFO sync.ZooKeeperSyncClientImpl: Start connecting to >> > Zookeeper! At debian-sandy.localdomain/127.0.1.1:61001 >> > 12/10/19 14:01:34 INFO zookeeper.ClientCnxn: Session establishment >> complete >> > on server localhost/127.0.0.1:21810, sessionid = 0x13a77936b710005, >> > negotiated timeout = 24000000 >> > 12/10/19 14:01:35 INFO bsp.BSPTask: [DX ping]: ping period is 30000going >> to >> > ping groom >> > 12/10/19 14:01:35 ERROR bsp.BSPTask: Started pinging to groom >> > 12/10/19 14:01:35 INFO graph.GraphJobRunner: [DX]: starting to load >> > vertices >> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: Connecting to >> > debian-sandy.localdomain/127.0.1.1:61003 >> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c] OPEN >> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c, / >> > 127.0.1.1:55129 => debian-sandy.localdomain/127.0.1.1:61003] BOUND: / >> > 127.0.1.1:55129 >> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c, / >> > 127.0.1.1:55129 => debian-sandy.localdomain/127.0.1.1:61003] CONNECTED: >> > debian-sandy.localdomain/127.0.1.1:61003 >> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c, / >> > 127.0.1.1:55129 => debian-sandy.localdomain/127.0.1.1:61003] >> > INTEREST_CHANGED >> > 12/10/19 14:01:41 INFO ipc.NettyTransceiver: [id: 0x39ce757c, / >> > 127.0.1.1:55129 => debian-sandy.localdomain/127.0.1.1:61003] >> > INTEREST_CHANGED >> > >>
-- Best Regards, Edward J. Yoon @eddieyoon
