In hama_[user]_bspmaster_.....log file I get the following exception. But
this occurs in both cases when I run my job with 3 tasks or with 4 tasks
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /bsp
at
org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1069)
at
org.apache.hama.bsp.sync.ZKSyncBSPMasterClient.init(ZKSyncBSPMasterClient.java:62)
at org.apache.hama.bsp.BSPMaster.initZK(BSPMaster.java:509)
at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:492)
at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:475)
at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:46)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:56)
2015-06-26 23:18:41,140 ERROR
org.apache.hama.bsp.sync.ZKSyncBSPMasterClient:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
NodeExists for /bsp
On Sat, Jun 27, 2015 at 1:03 AM, Behroz Sikander <[email protected]> wrote:
> Hi,
> In the current thread, I mentioned 3 issues. Issue 1 and 3 are resolved
> but issue number 2 is still giving me headaches.
>
> My problem:
> My cluster now consists of 3 machines. Each one of them properly
> configured (Apparently). From my master machine when I start Hadoop and
> Hama, I can see the processes started on other 2 machines. If I check the
> maximum tasks that my cluster can support then I get 9 (3 tasks on each
> machine).
>
> When I run the PI example, it uses 9 tasks and runs fine. When I run my
> program with 3 tasks, everything runs fine. But when I increase the tasks
> (to 4) by using "setNumBspTask". Hama freezes. I do not understand what can
> go wrong.
>
> I checked the logs files and things look fine. I just sometimes get an
> exception that hama was not able to delete the sytem directory
> (bsp.system.dir) defined in the hama-site.xml.
>
> Any help or clue would be great.
>
> Regards,
> Behroz Sikander
>
> On Thu, Jun 25, 2015 at 1:13 PM, Behroz Sikander <[email protected]>
> wrote:
>
>> Thank you :)
>>
>> On Thu, Jun 25, 2015 at 12:14 AM, Edward J. Yoon <[email protected]>
>> wrote:
>>
>>> Hi,
>>>
>>> You can get the maximum number of available tasks like following code:
>>>
>>> BSPJobClient jobClient = new BSPJobClient(conf);
>>> ClusterStatus cluster = jobClient.getClusterStatus(true);
>>>
>>> // Set to maximum
>>> bsp.setNumBspTask(cluster.getMaxTasks());
>>>
>>>
>>> On Wed, Jun 24, 2015 at 11:20 PM, Behroz Sikander <[email protected]>
>>> wrote:
>>> > Hi,
>>> > 1) Thank you for this.
>>> > 2) Here are the images. I will look into the log files of PI example
>>> >
>>> > *Result of JPS command on slave*
>>> >
>>> http://s17.postimg.org/gpwe2bbfj/Screen_Shot_2015_06_22_at_7_23_31_PM.png
>>> >
>>> > *Result of JPS command on Master*
>>> >
>>> http://s14.postimg.org/s9922em5p/Screen_Shot_2015_06_22_at_7_23_42_PM.png
>>> >
>>> > 3) In my current case, I do not have any input submitted to the job.
>>> During
>>> > run time, I directly fetch data from HDFS. So, I am looking for
>>> something
>>> > like BSPJob.set*Max*NumBspTask().
>>> >
>>> > Regards,
>>> > Behroz
>>> >
>>> >
>>> >
>>> > On Tue, Jun 23, 2015 at 12:57 AM, Edward J. Yoon <
>>> [email protected]>
>>> > wrote:
>>> >
>>> >> Hello,
>>> >>
>>> >> 1) You can get the filesystem URI from a configuration using
>>> >> "FileSystem fs = FileSystem.get(conf);". Of course, the fs.defaultFS
>>> >> property should be in hama-site.xml
>>> >>
>>> >> <property>
>>> >> <name>fs.defaultFS</name>
>>> >> <value>hdfs://host1.mydomain.com:9000/</value>
>>> >> <description>
>>> >> The name of the default file system. Either the literal string
>>> >> "local" or a host:port for HDFS.
>>> >> </description>
>>> >> </property>
>>> >>
>>> >> 2) The 'bsp.tasks.maximum' is the number of tasks per node. It looks
>>> >> cluster configuration issue. Please run Pi example and look at the
>>> >> logs for more details. NOTE: you can not attach the images to mailing
>>> >> list so I can't see it.
>>> >>
>>> >> 3) You can use the BSPJob.setNumBspTask(int) method. If input is
>>> >> provided, the number of BSP tasks is basically driven by the number of
>>> >> DFS blocks. I'll fix it to be more flexible on HAMA-956.
>>> >>
>>> >> Thanks!
>>> >>
>>> >>
>>> >> On Tue, Jun 23, 2015 at 2:33 AM, Behroz Sikander <[email protected]>
>>> >> wrote:
>>> >> > Hi,
>>> >> > Recently, I moved from a single machine setup to a 2 machine setup.
>>> I was
>>> >> > successfully able to run my job that uses the HDFS to get data. I
>>> have 3
>>> >> > trivial questions
>>> >> >
>>> >> > 1- To access HDFS, I have to manually give the IP address of server
>>> >> running
>>> >> > HDFS. I thought that Hama will automatically pick from the
>>> configurations
>>> >> > but it does not. I am probably doing something wrong. Right now my
>>> code
>>> >> work
>>> >> > by using the following.
>>> >> >
>>> >> > FileSystem fs = FileSystem.get(new URI("hdfs://server_ip:port/"),
>>> conf);
>>> >> >
>>> >> > 2- On my master server, when I start hama it automatically starts
>>> hama in
>>> >> > the slave machine (all good). Both master and slave are set as
>>> >> groomservers.
>>> >> > This means that I have 2 servers to run my job which means that I
>>> can
>>> >> open
>>> >> > more BSPPeerChild processes. And if I submit my jar with 3 bsp
>>> tasks then
>>> >> > everything works fine. But when I move to 4 tasks, Hama freezes.
>>> Here is
>>> >> the
>>> >> > result of JPS command on slave.
>>> >> >
>>> >> >
>>> >> > Result of JPS command on Master
>>> >> >
>>> >> >
>>> >> >
>>> >> > You can see that it is only opening tasks on slaves but not on
>>> master.
>>> >> >
>>> >> > Note: I tried to change the bsp.tasks.maximum property in
>>> >> hama-default.xml
>>> >> > to 4 but still same result.
>>> >> >
>>> >> > 3- I want my cluster to open as many BSPPeerChild processes as
>>> possible.
>>> >> Is
>>> >> > there any setting that can I do to achieve that ? Or hama picks up
>>> the
>>> >> > values from hama-default.xml to open tasks ?
>>> >> >
>>> >> >
>>> >> > Regards,
>>> >> >
>>> >> > Behroz Sikander
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Best Regards, Edward J. Yoon
>>> >>
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>>
>>
>>
>
