Yes, the repair function has deleted.
On Sat, Apr 20, 2013 at 7:26 PM, Lyu Xuedong <[email protected]> wrote: > Hi, Edward, > > Thank you. > I updated the version. There comes a new problem. > > In the /Apache Hama BSP Programming Model > (http://people.apache.org/~tjungblut/downloads/hamadocs/ApacheHamaBSPProgrammingmodel_06.pdf)/ > there is a paragraph describes 'Graph repair' : "Hama requires a graph to be > completed before feeding it to an algorithm. By complete we mean that every > vertex that is referenced by an edge must somewhere be a vertex in the > graph. In many cases of leafs this is not always the case, therefore we have > added a repair functionality which is traversing the whole graph for leafs > and adding them to the vertex structure to prevent algorithms from breaking > with NullPointerExceptions when it does not find a referenced vertex. You > can turn this feature on by setting it in your configuration like this: > conf.setBoolean(GraphJobRunner.GRAPH_REPAIR, true);" > > I followed the guidance but got a hint 'GRAPH_REPAIR cannot be resolved or > is not a field' programming in Eclipse. I read the source code later and > there is really no variable named GRAPH_REPAIR in GraphJobRunner.java which > exists in the version 0.5.0. Is this function not supported any longer ? How > can I make a 'repaired graph'? > Thank you. > > > On 04/17/2013 12:51 PM, Edward J. Yoon wrote: >> >> Please use 0.6.1 and try your application with small data again. >> >> See also http://hama.apache.org/run_examples.html >> >> On Wed, Apr 17, 2013 at 10:27 AM, Lyu Xuedong <[email protected]> wrote: >>> >>> hama: 0.6.0 >>> hadoop : 1.0.4 >>> JDK : 1.6 >>> OS : ubuntu 12.04 >>> >>> >>> On 04/17/2013 05:47 AM, Edward J. Yoon wrote: >>>> >>>> Your version? >>>> >>>> On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong <[email protected]> wrote: >>>>> >>>>> Hi, Edward, thank you. But your suggestion seems do not work, do you >>>>> have >>>>> some other advice ? >>>>> >>>>> Is java.lang.NullPointerException related with heap size ? >>>>> >>>>> >>>>> >>>>> 在 2013-4-16，22:13，"Edward J. Yoon" <[email protected]> 写道： >>>>> >>>>>> I guess you need to increase the child processor JVM heap size. >>>>>> >>>>>> - conf/hama-site.xml: >>>>>> >>>>>> <property> >>>>>> <name>bsp.child.java.opts</name> >>>>>> <value>-Xmx2048m</value> >>>>>> </property> >>>>>> >>>>>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> My project can work well in eclipse, but when I export it as a jar >>>>>>> file >>>>>>> and >>>>>>> submit it to a two-node hama cluster, some errors and fatal occur if >>>>>>> the >>>>>>> input file is over 64M. >>>>>>> I suspect that my configure files of hadoop or hama is somewhere not >>>>>>> right, >>>>>>> but a pi estimator can run normally on my cluster and >>>>>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a >>>>>>> whole >>>>>>> day, >>>>>>> nothing improved. >>>>>>> What in my input file are a large number of RDF triples: "<subject> >>>>>>> <predicate> <object> ." My task is to create vertices for subjects >>>>>>> and >>>>>>> objects. Predicates are subjects' edges. >>>>>>> What should I do ? >>>>>>> >>>>>>> Terminal output: >>>>>>> >>>>>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain. >>>>>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to >>>>>>> process : 2 >>>>>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: >>>>>>> job_201304161357_0015 >>>>>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0 >>>>>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2 >>>>>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3 >>>>>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4 >>>>>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5 >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>>>> sync.ZKSyncClient: Initializing ZK Sync Client >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At >>>>>>> hadoop1/1.2.3.4:61002 >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>>>> ipc.Server: >>>>>>> Starting SocketReader >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>>>> ipc.Server: >>>>>>> IPC >>>>>>> Server Responder: starting >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>>>> ipc.Server: >>>>>>> IPC >>>>>>> Server handler 0 on 61002: starting >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002 >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>>>>> ipc.Server: >>>>>>> IPC >>>>>>> Server listener on 61002: starting >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR >>>>>>> bsp.BSPTask: >>>>>>> Error running bsp setup and bsp function. >>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO >>>>>>> ipc.Server: >>>>>>> Stopping server on 61002 >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO >>>>>>> ipc.Server: >>>>>>> IPC >>>>>>> Server handler 0 on 61002: exiting >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO >>>>>>> ipc.Server: >>>>>>> Stopping IPC Server listener on 61002 >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO >>>>>>> ipc.Server: >>>>>>> Stopping IPC Server Responder >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO >>>>>>> metrics.RpcInstrumentation: shut down >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR >>>>>>> bsp.BSPTask: >>>>>>> Shutting down ping service. >>>>>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL >>>>>>> bsp.GroomServer: >>>>>>> Error running child >>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>>>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>>>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed. >>>>>>> >>>>>>> tasklogs: >>>>>>> attempt_201304161357_0015_000000_0.log >>>>>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client >>>>>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting >>>>>>> to >>>>>>> Zookeeper! At hadoop1/1.2.3.4:61002 >>>>>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader >>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting >>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: >>>>>>> starting >>>>>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer >>>>>>> address:hadoop1 port:61002 >>>>>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: >>>>>>> starting >>>>>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp >>>>>>> function. >>>>>>> java.lang.NullPointerException >>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002 >>>>>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: >>>>>>> exiting >>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on >>>>>>> 61002 >>>>>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder >>>>>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down >>>>>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service. >>>>>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child >>>>>>> java.lang.NullPointerException >>>>>>> java.lang.NullPointerException >>>>>>> >>>>>>> attempt_201304161357_0015_000001_0.log >>>>>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client >>>>>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting >>>>>>> to >>>>>>> Zookeeper! At hadoop1/1.2.3.4:61001 >>>>>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl: >>>>>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode >>>>>>> = >>>>>>> NoNode for /bsp/job_201304161357_0015/peers >>>>>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader >>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting >>>>>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer >>>>>>> address:hadoop1 port:61001 >>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: >>>>>>> starting >>>>>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: >>>>>>> starting >>>>>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages >>>>>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local >>>>>>> exception: java.io.EOFException >>>>>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103) >>>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071) >>>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225) >>>>>>> at $Proxy3.put(Unknown Source) >>>>>>> at >>>>>>> >>>>>>> >>>>>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108) >>>>>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410) >>>>>>> at >>>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118) >>>>>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166) >>>>>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143) >>>>>>> at >>>>>>> >>>>>>> >>>>>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266) >>>>>>> Caused by: java.io.EOFException >>>>>>> at java.io.DataInputStream.readInt(DataInputStream.java:375) >>>>>>> at >>>>>>> >>>>>>> >>>>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800) >>>>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745) >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards, Edward J. Yoon >>>>>> @eddieyoon >>>> >>>> >>>> >> >> > -- Best Regards, Edward J. Yoon @eddieyoon
