Have you tried to run in pseudo-distributed mode? On Fri, Sep 27, 2013 at 5:47 AM, Roman Shapovalov <[email protected]> wrote: > Martin, > > Thanks for such verbose instructions. > >> You can find all Hama configuration files in the *conf* folder. > > OK, I thought Edward meant Hadoop configs specifically. > I have only added JAVA_HOME variable there, otherwise they are default. > >> You should also find task logs in your *temp* folder. > > I found the folder, but there were no .log files in the attempt* > folders (in both modes). > >> Normally you should find it in *hama/logs/tasklogs*. > > They appear in the pseudo-distributed mode only (which also fails). > See the attached file. > >> By the way do you have python3.2 installed? :-) > > Yes. "python" links to Python 2.6, but I pass "python3.2" as an > interpreter, which links to the correct version. > > > Roman > > On Thu, Sep 26, 2013 at 4:03 PM, Martin Illecker <[email protected]> wrote: >> Hi Roman, >> >> if you are running Hama in local mode, it will not use HDFS anyway. >> >> You can find all Hama configuration files in the *conf* folder. >> >> $ll hama/conf/ >> total 56 >> -rwxr-xr-x groomservers* >> -rwxr-xr-x hama-default.xml* >> -rwxr-xr-x hama-env.sh* >> -rwxr-xr-x hama-site.xml* >> -rwxr-xr-x log4j.properties* >> >> Probably you should setup the Pseudo Distributed Mode [1] in hama-site.xml. >> >> But the task log would be very interesting. >> >> Normally you should find it in *hama/logs/tasklogs*. >> e.g., >> hama/logs/tasklogs/job_201309262134_0001/attempt_201309262134_0001_000000_0.log >> >> You should also find task logs in your *temp* folder. >> But this location will depend on your operation system. >> e.g., in OSX >> /private/tmp/hadoop-YOURUSER/bsp/local/groomServer/attempt_201309262134_0001_000000_0/work/tasklogs/ >> >> By the way do you have python3.2 installed? :-) >> $ python --version >> Python 3.2.5 >> $ python3.2 --version >> Python 3.2.5 >> >> May I ask which operation system do you use? >> >> Martin >> >> [1] http://wiki.apache.org/hama/GettingStarted#Pseudo_Distributed_Mode >> >> >> >> 2013/9/26 Roman Shapovalov <[email protected]> >> >>> Hi Edward, >>> >>> Could you please be more specific? (Sorry, I am new to this stuff) >>> >>> I run Hama in local mode. The logs/ directory is empty, and I did not >>> find any logs in HDFS as well. >>> >>> And where can I find the Hadoop configuration? >>> >>> Thank you, >>> Roman >>> >>> On Thu, Sep 26, 2013 at 12:05 PM, Edward J. Yoon <[email protected]> >>> wrote: >>> > Hi, >>> > >>> > That's strange. Can you attach your namenode logs and hadoop >>> configurations? >>> > >>> > On Thu, Sep 26, 2013 at 11:03 PM, Roman Shapovalov >>> > <[email protected]> wrote: >>> >> Hi again, >>> >> >>> >> I have updated both Hama (from the trunk) and Streaming (from Martin's >>> >> github), and checked that patches have been applied, but I keep >>> >> getting the same error (full log for local configuration is attached). >>> >> >>> >> Another thing may be relevant: I keep the default Hadoop libraries in >>> >> lib/. If I replace them as the tutorial says, some classes cannot be >>> >> found even if I run pure Hama (which works perfectly with default >>> >> libs). I don't know if it is important. >>> >> >>> >> Thanks, >>> >> Roman >>> >> >>> >> On Tue, Sep 24, 2013 at 9:22 AM, Martin Illecker <[email protected]> >>> wrote: >>> >>> Hi Roman, >>> >>> >>> >>> sorry for inconvenience! >>> >>> The problem has been reported [1] and will be fixed shortly to the >>> trunk. >>> >>> >>> >>> [1] https://issues.apache.org/jira/browse/HAMA-805 >>> >>> >>> >>> >>> >>> 2013/9/23 Edward J. Yoon <[email protected]> >>> >>> >>> >>>> This looks like a bug of DistCacheUtils. >>> >>>> >>> >>>> Thanks for your report. I'll look at it tomorrow. >>> >>>> >>> >>>> On Mon, Sep 23, 2013 at 11:52 PM, Roman Shapovalov >>> >>>> <[email protected]> wrote: >>> >>>> > Hello all, >>> >>>> > >>> >>>> > I try to use Hama Streaming. >>> >>>> > I have successfully installed Hama (the Pi example works). >>> >>>> > I follow this tutorial: >>> >>>> > http://wiki.apache.org/hama/HamaStreaming >>> >>>> > >>> >>>> > When I try to run the distributed HelloWorld in the local >>> >>>> > configuration, I get the following error: >>> >>>> > >>> >>>> > $ bin/hama pipes -streaming true -bspTasks 3 -interpreter python3.2 >>> >>>> > -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/ >>> -program >>> >>>> > /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP >>> >>>> > >>> >>>> > 13/09/23 18:03:50 INFO pipes.Submitter: Streaming enabled! >>> >>>> > 13/09/23 18:03:50 WARN util.NativeCodeLoader: Unable to load >>> >>>> > native-hadoop library for your platform... using builtin-java >>> classes >>> >>>> > where applicable >>> >>>> > 13/09/23 18:03:50 WARN bsp.BSPJobClient: No job jar file set. User >>> >>>> > classes may not be found. See BSPJob#setJar(String) or check Your >>> jar >>> >>>> > file. >>> >>>> > 13/09/23 18:03:50 INFO bsp.BSPJobClient: Running job: >>> >>>> job_localrunner_0001 >>> >>>> > 13/09/23 18:03:50 INFO bsp.LocalBSPRunner: Setting up a new barrier >>> for >>> >>>> 3 tasks! >>> >>>> > 13/09/23 18:03:50 ERROR bsp.LocalBSPRunner: Exception during BSP >>> >>>> execution! >>> >>>> > java.lang.NullPointerException >>> >>>> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:44) >>> >>>> > at >>> >>>> >>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255) >>> >>>> > at >>> >>>> >>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286) >>> >>>> > at >>> >>>> >>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211) >>> >>>> > at >>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>> >>>> > at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>> >>>> > at >>> >>>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) >>> >>>> > at >>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>> >>>> > at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>> >>>> > at >>> >>>> >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >>> >>>> > at >>> >>>> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >>> >>>> > at java.lang.Thread.run(Thread.java:662) >>> >>>> > [output cropped] >>> >>>> > >>> >>>> > When I turn to the pseudo-distributed mode, job fails too (after a >>> >>>> > minute of execution): >>> >>>> > >>> >>>> > 13/09/23 18:46:34 INFO pipes.Submitter: Streaming enabled! >>> >>>> > 13/09/23 18:46:34 WARN util.NativeCodeLoader: Unable to load >>> >>>> > native-hadoop library for your platform... using builtin-java >>> classes >>> >>>> > where applicable >>> >>>> > 13/09/23 18:46:34 WARN bsp.BSPJobClient: No job jar file set. User >>> >>>> > classes may not be found. See BSPJob#setJar(String) or check Your >>> jar >>> >>>> > file. >>> >>>> > 13/09/23 18:46:34 INFO bsp.BSPJobClient: Running job: >>> >>>> job_201309231846_0001 >>> >>>> > 13/09/23 18:47:40 INFO bsp.BSPJobClient: Job failed. >>> >>>> > >>> >>>> > Task log contains errors: >>> >>>> > >>> >>>> > 13/09/23 18:46:37 INFO ipc.Server: Starting Socket Reader #1 for >>> port >>> >>>> 43475 >>> >>>> > 13/09/23 18:46:37 INFO ipc.Server: IPC Server Responder: starting >>> >>>> > 13/09/23 18:46:37 INFO ipc.Server: IPC Server listener on 43475: >>> starting >>> >>>> > 13/09/23 18:46:37 INFO message.HadoopMessageManagerImpl: BSPPeer >>> >>>> > address:localhost.localdomain port:43475 >>> >>>> > 13/09/23 18:46:37 INFO ipc.Server: IPC Server handler 0 on 43475: >>> >>>> starting >>> >>>> > 13/09/23 18:46:37 WARN util.NativeCodeLoader: Unable to load >>> >>>> > native-hadoop library for your platform... using builtin-java >>> classes >>> >>>> > where applicable >>> >>>> > 13/09/23 18:46:37 INFO sync.ZKSyncClient: Initializing ZK Sync >>> Client >>> >>>> > 13/09/23 18:46:37 INFO sync.ZooKeeperSyncClientImpl: Start >>> connecting >>> >>>> > to Zookeeper! At localhost.localdomain/127.0.0.1:43475 >>> >>>> > 13/09/23 18:46:37 ERROR bsp.BSPTask: Error running bsp setup and bsp >>> >>>> function. >>> >>>> > java.lang.NullPointerException >>> >>>> > at java.io.File.<init>(File.java:222) >>> >>>> > at >>> >>>> >>> org.apache.hama.pipes.PipesApplication.setupCommand(PipesApplication.java:130) >>> >>>> > at >>> >>>> >>> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:257) >>> >>>> > at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:44) >>> >>>> > at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176) >>> >>>> > at org.apache.hama.bsp.BSPTask.run(BSPTask.java:146) >>> >>>> > at >>> >>>> >>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246) >>> >>>> > [output cropped] >>> >>>> > >>> >>>> > I use the latest trunk version of Hama, Python 3.2.5 and Hadoop >>> >>>> 2.0.0-cdh4.1.1. >>> >>>> > >>> >>>> > Please help me to figure out the problem. >>> >>>> > >>> >>>> > Thanks in advance, >>> >>>> > Roman >>> >>>> >>> >>>> >>> >>>> >>> >>>> -- >>> >>>> Best Regards, Edward J. Yoon >>> >>>> @eddieyoon >>> >>>> >>> > >>> > >>> > >>> > -- >>> > Best Regards, Edward J. Yoon >>> > @eddieyoon >>>
-- Best Regards, Edward J. Yoon @eddieyoon
