I didn't try the original pagerank job on the cluster because my graph data is not in sequence file format. However, both the original pagerank job and my pagerank job works in local mode.
Yingyi On Fri, Sep 27, 2013 at 1:01 PM, Yingyi Bu <[email protected]> wrote: > This is the property: > > <name>bsp.child.java.opts</name> > <value>-Xmx4096m</value> > > Yingyi > > > > On Fri, Sep 27, 2013 at 3:56 AM, Edward J. Yoon <[email protected]>wrote: > >> That's strange ... I couldn't find any errors in tasks logs. How did >> you set the value of "bsp.child.java.opts" property? >> >> > at org.apache.hama.examples.PageRank2.main(PageRank2.java:162) >> >> The original Hama PageRank example also fails? >> >> On Fri, Sep 27, 2013 at 7:06 PM, Yingyi Bu <[email protected]> wrote: >> > It fails again: >> > >> > 13/09/27 09:49:12 INFO mortbay.log: Logging to >> > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> > org.mortbay.log.Slf4jLog >> > 13/09/27 09:49:12 INFO bsp.FileInputFormat: Total input paths to >> process : 2 >> > 13/09/27 09:49:12 WARN util.NativeCodeLoader: Unable to load >> native-hadoop >> > library for your platform... using builtin-java classes where applicable >> > 13/09/27 09:49:12 WARN snappy.LoadSnappy: Snappy native library not >> loaded >> > 13/09/27 09:49:12 INFO bsp.FileInputFormat: Total input paths to >> process : 2 >> > 13/09/27 09:49:13 INFO bsp.BSPJobClient: Running job: >> job_201309270937_0004 >> > 13/09/27 09:49:16 INFO bsp.BSPJobClient: Current supersteps number: 0 >> > 13/09/27 09:53:09 INFO bsp.BSPJobClient: Job failed. >> > 13/09/27 09:53:09 ERROR bsp.BSPJobClient: Error partitioning the input >> path. >> > >> > java.io.IOException: Runtime partition failed for the job. >> > at >> org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465) >> > at >> > >> org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333) >> > at >> org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293) >> > at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:229) >> > at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203) >> > at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:236) >> > at org.apache.hama.examples.PageRank2.main(PageRank2.java:162) >> > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> > at >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:601) >> > at >> > >> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) >> > at >> > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) >> > at >> > org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:43) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> > at >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:601) >> > at org.apache.hama.util.RunJar.main(RunJar.java:146) >> > >> > I set the HAMA_LOG_DIR in conf/hama-env.sh to avoid logs going to NFS. >> > Just write a script to dump all tasklogs for the job. >> > Attached is the logs. But I cannot find any clue from the logs. >> > >> > Any help is appreciated. Thanks!! >> > >> > Yingyi >> > >> > >> > >> > On Fri, Sep 27, 2013 at 2:44 AM, Edward J. Yoon <[email protected]> >> > wrote: >> >> >> >> Hi, >> >> >> >> The task logs are in ${HAMA_HOME}/logs/tasklogs/. >> >> >> >> >> >> >> >> On Fri, Sep 27, 2013 at 6:12 PM, Yingyi Bu <[email protected]> wrote: >> >> > All the groomsevers' tasklogs? >> >> > >> >> > Yingyi >> >> > >> >> > >> >> > On Fri, Sep 27, 2013 at 2:11 AM, Edward J. Yoon >> >> > <[email protected]>wrote: >> >> > >> >> >> Could you please check the tasklogs? >> >> >> >> >> >> On Fri, Sep 27, 2013 at 6:03 PM, Yingyi Bu <[email protected]> >> wrote: >> >> >> > Hi, >> >> >> > >> >> >> > I got a runtime partition exception as follows: >> >> >> > >> >> >> > 13/09/27 08:52:41 INFO mortbay.log: Logging to >> >> >> > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> >> >> > org.mortbay.log.Slf4jLog >> >> >> > 13/09/27 08:52:42 INFO bsp.FileInputFormat: Total input paths to >> >> >> > process >> >> >> : 2 >> >> >> > 13/09/27 08:52:42 WARN util.NativeCodeLoader: Unable to load >> >> >> native-hadoop >> >> >> > library for your platform... using builtin-java classes where >> >> >> > applicable >> >> >> > 13/09/27 08:52:42 WARN snappy.LoadSnappy: Snappy native library >> not >> >> >> loaded >> >> >> > 13/09/27 08:52:42 INFO bsp.FileInputFormat: Total input paths to >> >> >> > process >> >> >> : 2 >> >> >> > 13/09/27 08:52:42 INFO bsp.BSPJobClient: Running job: >> >> >> job_201309270852_0002 >> >> >> > 13/09/27 08:52:45 INFO bsp.BSPJobClient: Current supersteps >> number: 0 >> >> >> > 13/09/27 08:58:10 INFO bsp.BSPJobClient: Current supersteps >> number: 1 >> >> >> > 13/09/27 08:58:16 INFO bsp.BSPJobClient: Current supersteps >> number: 2 >> >> >> > 13/09/27 09:00:04 INFO bsp.BSPJobClient: Job failed. >> >> >> > 13/09/27 09:00:04 ERROR bsp.BSPJobClient: Error partitioning the >> >> >> > input >> >> >> path. >> >> >> > java.io.IOException: Runtime partition failed for the job. >> >> >> > at >> >> >> org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465) >> >> >> > at >> >> >> > >> >> >> > >> org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333) >> >> >> > at >> >> >> org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293) >> >> >> > at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:229) >> >> >> > at >> org.apache.hama.graph.GraphJob.submit(GraphJob.java:203) >> >> >> > at >> >> >> > org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:236) >> >> >> > at >> >> >> > org.apache.hama.examples.PageRank2.main(PageRank2.java:164) >> >> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> >> >> > Method) >> >> >> > at >> >> >> > >> >> >> >> >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> >> >> > at >> >> >> > >> >> >> >> >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> >> > at java.lang.reflect.Method.invoke(Method.java:601) >> >> >> > at >> >> >> > >> >> >> >> >> >> >> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) >> >> >> > at >> >> >> > >> org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) >> >> >> > at >> >> >> > org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:43) >> >> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> >> >> > Method) >> >> >> > at >> >> >> > >> >> >> >> >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> >> >> > at >> >> >> > >> >> >> >> >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> >> > at java.lang.reflect.Method.invoke(Method.java:601) >> >> >> > at org.apache.hama.util.RunJar.main(RunJar.java:146) >> >> >> > >> >> >> > Does anyone know what's the possible reason of this exception? >> >> >> > Thanks! >> >> >> > >> >> >> > Yingyi >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Best Regards, Edward J. Yoon >> >> >> @eddieyoon >> >> >> >> >> >> >> >> >> >> >> -- >> >> Best Regards, Edward J. Yoon >> >> @eddieyoon >> > >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon >> > >
