>
> Hi Roman!
>
Did you setup the default filesystem in hama-site.xml?
>
Please submit your hama-site.xml configuration.
>
Martin
>
hama-site.xml - pseudo-distributed mode
>
<configuration>
>
<property>
<name>bsp.master.address</name>
<value>localhost:40000</value>
<description>The address of the bsp master server. Either the
literal string "local" or a host:port for distributed mode
</description>
</property>
>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost/</value>
<description>
The name of the default file system. Either the literal string
"local" or a host:port for HDFS.
</description>
</property>
>
<property>
<name>hama.zookeeper.quorum</name>
<value>localhost</value>
<description>Comma separated list of servers in the ZooKeeper
Quorum.
For example, "host1.mydomain.com,host2.mydomain.com,
host3.mydomain.com".
By default this is set to localhost for local and
pseudo-distributed modes
of operation. For a fully-distributed setup, this should be set
to a full
list of ZooKeeper quorum servers. If HAMA_MANAGES_ZK is set in
hama-env.sh
this is the list of servers which we will start/stop zookeeper
on.
</description>
</property>
>
</configuration>
>
>
> Am 27.09.2013 um 09:32 schrieb Roman Shapovalov <
> [email protected]>:
>
> > Edward,
> >
> > Yes, I did. See the logs in my previous message.
> >
> > Roman
> >
> > On Fri, Sep 27, 2013 at 7:15 AM, Edward J. Yoon <[email protected]>
> wrote:
> >> Have you tried to run in pseudo-distributed mode?
> >>
> >> On Fri, Sep 27, 2013 at 5:47 AM, Roman Shapovalov
> >> <[email protected]> wrote:
> >>> Martin,
> >>>
> >>> Thanks for such verbose instructions.
> >>>
> >>>> You can find all Hama configuration files in the *conf* folder.
> >>>
> >>> OK, I thought Edward meant Hadoop configs specifically.
> >>> I have only added JAVA_HOME variable there, otherwise they are default.
> >>>
> >>>> You should also find task logs in your *temp* folder.
> >>>
> >>> I found the folder, but there were no .log files in the attempt*
> >>> folders (in both modes).
> >>>
> >>>> Normally you should find it in *hama/logs/tasklogs*.
> >>>
> >>> They appear in the pseudo-distributed mode only (which also fails).
> >>> See the attached file.
> >>>
> >>>> By the way do you have python3.2 installed? :-)
> >>>
> >>> Yes. "python" links to Python 2.6, but I pass "python3.2" as an
> >>> interpreter, which links to the correct version.
> >>>
> >>>
> >>> Roman
> >>>
> >>> On Thu, Sep 26, 2013 at 4:03 PM, Martin Illecker <[email protected]>
> wrote:
> >>>> Hi Roman,
> >>>>
> >>>> if you are running Hama in local mode, it will not use HDFS anyway.
> >>>>
> >>>> You can find all Hama configuration files in the *conf* folder.
> >>>>
> >>>> $ll hama/conf/
> >>>> total 56
> >>>> -rwxr-xr-x groomservers*
> >>>> -rwxr-xr-x hama-default.xml*
> >>>> -rwxr-xr-x hama-env.sh*
> >>>> -rwxr-xr-x hama-site.xml*
> >>>> -rwxr-xr-x log4j.properties*
> >>>>
> >>>> Probably you should setup the Pseudo Distributed Mode [1] in
> hama-site.xml.
> >>>>
> >>>> But the task log would be very interesting.
> >>>>
> >>>> Normally you should find it in *hama/logs/tasklogs*.
> >>>> e.g.,
> hama/logs/tasklogs/job_201309262134_0001/attempt_201309262134_0001_000000_0.log
> >>>>
> >>>> You should also find task logs in your *temp* folder.
> >>>> But this location will depend on your operation system.
> >>>> e.g., in OSX
> >>>>
> /private/tmp/hadoop-YOURUSER/bsp/local/groomServer/attempt_201309262134_0001_000000_0/work/tasklogs/
> >>>>
> >>>> By the way do you have python3.2 installed? :-)
> >>>> $ python --version
> >>>> Python 3.2.5
> >>>> $ python3.2 --version
> >>>> Python 3.2.5
> >>>>
> >>>> May I ask which operation system do you use?
> >>>>
> >>>> Martin
> >>>>
> >>>> [1]
> http://wiki.apache.org/hama/GettingStarted#Pseudo_Distributed_Mode
> >>>>
> >>>>
> >>>>
> >>>> 2013/9/26 Roman Shapovalov <[email protected]>
> >>>>
> >>>>> Hi Edward,
> >>>>>
> >>>>> Could you please be more specific? (Sorry, I am new to this stuff)
> >>>>>
> >>>>> I run Hama in local mode. The logs/ directory is empty, and I did not
> >>>>> find any logs in HDFS as well.
> >>>>>
> >>>>> And where can I find the Hadoop configuration?
> >>>>>
> >>>>> Thank you,
> >>>>> Roman
> >>>>>
> >>>>> On Thu, Sep 26, 2013 at 12:05 PM, Edward J. Yoon <
> [email protected]>
> >>>>> wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> That's strange. Can you attach your namenode logs and hadoop
> >>>>> configurations?
> >>>>>>
> >>>>>> On Thu, Sep 26, 2013 at 11:03 PM, Roman Shapovalov
> >>>>>> <[email protected]> wrote:
> >>>>>>> Hi again,
> >>>>>>>
> >>>>>>> I have updated both Hama (from the trunk) and Streaming (from
> Martin's
> >>>>>>> github), and checked that patches have been applied, but I keep
> >>>>>>> getting the same error (full log for local configuration is
> attached).
> >>>>>>>
> >>>>>>> Another thing may be relevant: I keep the default Hadoop libraries
> in
> >>>>>>> lib/. If I replace them as the tutorial says, some classes cannot
> be
> >>>>>>> found even if I run pure Hama (which works perfectly with default
> >>>>>>> libs). I don't know if it is important.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Roman
> >>>>>>>
> >>>>>>> On Tue, Sep 24, 2013 at 9:22 AM, Martin Illecker <
> [email protected]>
> >>>>> wrote:
> >>>>>>>> Hi Roman,
> >>>>>>>>
> >>>>>>>> sorry for inconvenience!
> >>>>>>>> The problem has been reported [1] and will be fixed shortly to the
> >>>>> trunk.
> >>>>>>>>
> >>>>>>>> [1] https://issues.apache.org/jira/browse/HAMA-805
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 2013/9/23 Edward J. Yoon <[email protected]>
> >>>>>>>>
> >>>>>>>>> This looks like a bug of DistCacheUtils.
> >>>>>>>>>
> >>>>>>>>> Thanks for your report. I'll look at it tomorrow.
> >>>>>>>>>
> >>>>>>>>> On Mon, Sep 23, 2013 at 11:52 PM, Roman Shapovalov
> >>>>>>>>> <[email protected]> wrote:
> >>>>>>>>>> Hello all,
> >>>>>>>>>>
> >>>>>>>>>> I try to use Hama Streaming.
> >>>>>>>>>> I have successfully installed Hama (the Pi example works).
> >>>>>>>>>> I follow this tutorial:
> >>>>>>>>>> http://wiki.apache.org/hama/HamaStreaming
> >>>>>>>>>>
> >>>>>>>>>> When I try to run the distributed HelloWorld in the local
> >>>>>>>>>> configuration, I get the following error:
> >>>>>>>>>>
> >>>>>>>>>> $ bin/hama pipes -streaming true -bspTasks 3 -interpreter
> python3.2
> >>>>>>>>>> -cachefiles /tmp/PyStreaming/*.py -output /tmp/pystream-out/
> >>>>> -program
> >>>>>>>>>> /tmp/PyStreaming/BSPRunner.py -programArgs HelloWorldBSP
> >>>>>>>>>>
> >>>>>>>>>> 13/09/23 18:03:50 INFO pipes.Submitter: Streaming enabled!
> >>>>>>>>>> 13/09/23 18:03:50 WARN util.NativeCodeLoader: Unable to load
> >>>>>>>>>> native-hadoop library for your platform... using builtin-java
> >>>>> classes
> >>>>>>>>>> where applicable
> >>>>>>>>>> 13/09/23 18:03:50 WARN bsp.BSPJobClient: No job jar file set.
> User
> >>>>>>>>>> classes may not be found. See BSPJob#setJar(String) or check
> Your
> >>>>> jar
> >>>>>>>>>> file.
> >>>>>>>>>> 13/09/23 18:03:50 INFO bsp.BSPJobClient: Running job:
> >>>>>>>>> job_localrunner_0001
> >>>>>>>>>> 13/09/23 18:03:50 INFO bsp.LocalBSPRunner: Setting up a new
> barrier
> >>>>> for
> >>>>>>>>> 3 tasks!
> >>>>>>>>>> 13/09/23 18:03:50 ERROR bsp.LocalBSPRunner: Exception during BSP
> >>>>>>>>> execution!
> >>>>>>>>>> java.lang.NullPointerException
> >>>>>>>>>> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:44)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:255)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:211)
> >>>>>>>>>> at
> >>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> >>>>>>>>>> at
> >>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> >>>>>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> >>>>>>>>>> at java.lang.Thread.run(Thread.java:662)
> >>>>>>>>>> [output cropped]
> >>>>>>>>>>
> >>>>>>>>>> When I turn to the pseudo-distributed mode, job fails too
> (after a
> >>>>>>>>>> minute of execution):
> >>>>>>>>>>
> >>>>>>>>>> 13/09/23 18:46:34 INFO pipes.Submitter: Streaming enabled!
> >>>>>>>>>> 13/09/23 18:46:34 WARN util.NativeCodeLoader: Unable to load
> >>>>>>>>>> native-hadoop library for your platform... using builtin-java
> >>>>> classes
> >>>>>>>>>> where applicable
> >>>>>>>>>> 13/09/23 18:46:34 WARN bsp.BSPJobClient: No job jar file set.
> User
> >>>>>>>>>> classes may not be found. See BSPJob#setJar(String) or check
> Your
> >>>>> jar
> >>>>>>>>>> file.
> >>>>>>>>>> 13/09/23 18:46:34 INFO bsp.BSPJobClient: Running job:
> >>>>>>>>> job_201309231846_0001
> >>>>>>>>>> 13/09/23 18:47:40 INFO bsp.BSPJobClient: Job failed.
> >>>>>>>>>>
> >>>>>>>>>> Task log contains errors:
> >>>>>>>>>>
> >>>>>>>>>> 13/09/23 18:46:37 INFO ipc.Server: Starting Socket Reader #1 for
> >>>>> port
> >>>>>>>>> 43475
> >>>>>>>>>> 13/09/23 18:46:37 INFO ipc.Server: IPC Server Responder:
> starting
> >>>>>>>>>> 13/09/23 18:46:37 INFO ipc.Server: IPC Server listener on 43475:
> >>>>> starting
> >>>>>>>>>> 13/09/23 18:46:37 INFO message.HadoopMessageManagerImpl:
> BSPPeer
> >>>>>>>>>> address:localhost.localdomain port:43475
> >>>>>>>>>> 13/09/23 18:46:37 INFO ipc.Server: IPC Server handler 0 on
> 43475:
> >>>>>>>>> starting
> >>>>>>>>>> 13/09/23 18:46:37 WARN util.NativeCodeLoader: Unable to load
> >>>>>>>>>> native-hadoop library for your platform... using builtin-java
> >>>>> classes
> >>>>>>>>>> where applicable
> >>>>>>>>>> 13/09/23 18:46:37 INFO sync.ZKSyncClient: Initializing ZK Sync
> >>>>> Client
> >>>>>>>>>> 13/09/23 18:46:37 INFO sync.ZooKeeperSyncClientImpl: Start
> >>>>> connecting
> >>>>>>>>>> to Zookeeper! At localhost.localdomain/127.0.0.1:43475
> >>>>>>>>>> 13/09/23 18:46:37 ERROR bsp.BSPTask: Error running bsp setup
> and bsp
> >>>>>>>>> function.
> >>>>>>>>>> java.lang.NullPointerException
> >>>>>>>>>> at java.io.File.<init>(File.java:222)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> org.apache.hama.pipes.PipesApplication.setupCommand(PipesApplication.java:130)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> org.apache.hama.pipes.PipesApplication.start(PipesApplication.java:257)
> >>>>>>>>>> at org.apache.hama.pipes.PipesBSP.setup(PipesBSP.java:44)
> >>>>>>>>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:176)
> >>>>>>>>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
> >>>>>>>>>> at
> >>>>>>>>>
> >>>>>
> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1246)
> >>>>>>>>>> [output cropped]
> >>>>>>>>>>
> >>>>>>>>>> I use the latest trunk version of Hama, Python 3.2.5 and Hadoop
> >>>>>>>>> 2.0.0-cdh4.1.1.
> >>>>>>>>>>
> >>>>>>>>>> Please help me to figure out the problem.
> >>>>>>>>>>
> >>>>>>>>>> Thanks in advance,
> >>>>>>>>>> Roman
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best Regards, Edward J. Yoon
> >>>>>>>>> @eddieyoon
> >>>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best Regards, Edward J. Yoon
> >>>>>> @eddieyoon
> >>>>>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
>
>
