Please use 0.6.1 and try your application with small data again. See also http://hama.apache.org/run_examples.html
On Wed, Apr 17, 2013 at 10:27 AM, Lyu Xuedong <[email protected]> wrote: > hama: 0.6.0 > hadoop : 1.0.4 > JDK : 1.6 > OS : ubuntu 12.04 > > > On 04/17/2013 05:47 AM, Edward J. Yoon wrote: >> >> Your version? >> >> On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <[email protected]> wrote: >>> >>> Hi, Edward, thank you. But your suggestion seems do not work, do you have >>> some other advice ? >>> >>> Is java.lang.NullPointerException related with heap size ? >>> >>> >>> >>> 在 2013-4-16，22:13，"Edward J. Yoon" <[email protected]> 写道： >>> >>>> I guess you need to increase the child processor JVM heap size. >>>> >>>> - conf/hama-site.xml: >>>> >>>> <property> >>>> <name>bsp.child.java.opts</name> >>>> <value>-Xmx2048m</value> >>>> </property> >>>> >>>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <[email protected]> >>>> wrote: >>>>> >>>>> My project can work well in eclipse, but when I export it as a jar file >>>>> and >>>>> submit it to a two-node hama cluster, some errors and fatal occur if >>>>> the >>>>> input file is over 64M. >>>>> I suspect that my configure files of hadoop or hama is somewhere not >>>>> right, >>>>> but a pi estimator can run normally on my cluster and >>>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole >>>>> day, >>>>> nothing improved. >>>>> What in my input file are a large number of RDF triples: "<subject> >>>>> <predicate> <object> ." My task is to create vertices for subjects and >>>>> objects. Predicates are subjects' edges. >>>>> What should I do ? >>>>> >>>>> Terminal output: >>>>> >>>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain. >>>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to >>>>> process : 2 >>>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: >>>>> job_201304161357_0015 >>>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0 >>>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2 >>>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3 >>>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4 >>>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5 >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>> sync.ZKSyncClient: Initializing ZK Sync Client >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At >>>>> hadoop1/1.2.3.4:61002 >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: >>>>> Starting SocketReader >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: >>>>> IPC >>>>> Server Responder: starting >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: >>>>> IPC >>>>> Server handler 0 on 61002: starting >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002 >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: >>>>> IPC >>>>> Server listener on 61002: starting >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR >>>>> bsp.BSPTask: >>>>> Error running bsp setup and bsp function. >>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>>>> Stopping server on 61002 >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>>>> IPC >>>>> Server handler 0 on 61002: exiting >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>>>> Stopping IPC Server listener on 61002 >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>>>> Stopping IPC Server Responder >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO >>>>> metrics.RpcInstrumentation: shut down >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR >>>>> bsp.BSPTask: >>>>> Shutting down ping service. >>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL >>>>> bsp.GroomServer: >>>>> Error running child >>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed. >>>>> >>>>> tasklogs: >>>>> attempt_201304161357_0015_000000_0.log >>>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client >>>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting >>>>> to >>>>> Zookeeper! At hadoop1/1.2.3.4:61002 >>>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader >>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting >>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: >>>>> starting >>>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer >>>>> address:hadoop1 port:61002 >>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: >>>>> starting >>>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp >>>>> function. >>>>> java.lang.NullPointerException >>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002 >>>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: >>>>> exiting >>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on >>>>> 61002 >>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder >>>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down >>>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service. >>>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child >>>>> java.lang.NullPointerException >>>>> java.lang.NullPointerException >>>>> >>>>> attempt_201304161357_0015_000001_0.log >>>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client >>>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting >>>>> to >>>>> Zookeeper! At hadoop1/1.2.3.4:61001 >>>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl: >>>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = >>>>> NoNode for /bsp/job_201304161357_0015/peers >>>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader >>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting >>>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer >>>>> address:hadoop1 port:61001 >>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: >>>>> starting >>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: >>>>> starting >>>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages >>>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local >>>>> exception: java.io.EOFException >>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103) >>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071) >>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225) >>>>> at $Proxy3.put(Unknown Source) >>>>> at >>>>> >>>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108) >>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410) >>>>> at >>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118) >>>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166) >>>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143) >>>>> at >>>>> >>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266) >>>>> Caused by: java.io.EOFException >>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375) >>>>> at >>>>> >>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800) >>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745) >>>> >>>> >>>> >>>> -- >>>> Best Regards, Edward J. Yoon >>>> @eddieyoon >> >> >> > -- Best Regards, Edward J. Yoon @eddieyoon
