Re: Job Manager HA manual setup

[Welly Tambunan]

Hi Ufuk,

Thanks, it's working fine now with your suggestion.

Cheers

On Sun, Feb 28, 2016 at 10:10 PM, Ufuk Celebi  wrote:

> Hey Welly!
>
> Yes, it is possible to do manually via the jobmanager.sh and
> taskmanager.sh script like this:
>
> jobmanager.sh start cluster $HOST $WEB-UI-PORT
> taskmanager.sh start
>
> The start-cluster.sh script is just a wrapper around these scripts.
>
> From experience, it's often forgotten to sync the configuration files.
> Make sure to have the same configuration file on each host (both job
> and task managers), because that is relevant to parse the ZooKeeper
> quorum etc.
>
> The task managers retrieve the currently leading job manager via
> ZooKeeper. If job manager failover works as expected, but the task
> managers don't connect to the new job manager, I would suspect that
> the task manager configuration is out of sync. Could you check this
> please?
>
> Moreover, it will be helpful to have a look at the jobmanager and task
> manager logs to further investigate this. Can you share these?
> (Privately works as well of course.)
>
> – Ufuk
>
>
>
>
> On Sat, Feb 27, 2016 at 1:28 AM, Welly Tambunan  wrote:
> > typos
> >
> >
> > We have tried this one the job manager can failover, but the task manager
> > CAN'T be relocated to the new task manager. Is there some settings for
> this
> > one ? Or is the task manager also can be relocate to the new job manager
> ?
> >
> > Cheers
> >
> > On Sat, Feb 27, 2016 at 7:27 AM, Welly Tambunan 
> wrote:
> >>
> >> Hi All,
> >>
> >> We have already try to setup the Job Manager HA based on the
> documentation
> >> and using script and provided zookeeper. It works.
> >>
> >> However currently everything is done using start-cluster script that I
> >> believe will require passwordlress ssh between node. We are restricted
> with
> >> our environment so this one is not possible.
> >>
> >> Is it possible to setup the Job Manager HA manually ? By starting each
> job
> >> manager with in each node and task manager. We have our zookeeper and
> hdfs
> >> cluster already.
> >>
> >> We have tried this one the job manager can failover, but the task
> manager
> >> can be relocated to the new task manager. Is there some settings for
> this
> >> one ? Or is the task manager also can be relocate to the new job
> manager ?
> >>
> >> Any more details on the mechanism used on Job Manager HA and interaction
> >> with Zookeeper ?
> >>
> >> Is task manager also registered on Zookeeper ? How they find the right
> job
> >> manager master ?
> >>
> >>
> >> Thanks for your help.
> >>
> >> Cheers
> >> --
> >> Welly Tambunan
> >> Triplelands
> >>
> >> http://weltam.wordpress.com
> >> http://www.triplelands.com
> >
> >
> >
> >
> > --
> > Welly Tambunan
> > Triplelands
> >
> > http://weltam.wordpress.com
> > http://www.triplelands.com
>



-- 
Welly Tambunan
Triplelands

http://weltam.wordpress.com
http://www.triplelands.com 

Re: Job Manager HA manual setup

[Ufuk Celebi]

Hey Welly!

Yes, it is possible to do manually via the jobmanager.sh and
taskmanager.sh script like this:

jobmanager.sh start cluster $HOST $WEB-UI-PORT
taskmanager.sh start

The start-cluster.sh script is just a wrapper around these scripts.

>From experience, it's often forgotten to sync the configuration files.
Make sure to have the same configuration file on each host (both job
and task managers), because that is relevant to parse the ZooKeeper
quorum etc.

The task managers retrieve the currently leading job manager via
ZooKeeper. If job manager failover works as expected, but the task
managers don't connect to the new job manager, I would suspect that
the task manager configuration is out of sync. Could you check this
please?

Moreover, it will be helpful to have a look at the jobmanager and task
manager logs to further investigate this. Can you share these?
(Privately works as well of course.)

– Ufuk




On Sat, Feb 27, 2016 at 1:28 AM, Welly Tambunan  wrote:
> typos
>
>
> We have tried this one the job manager can failover, but the task manager
> CAN'T be relocated to the new task manager. Is there some settings for this
> one ? Or is the task manager also can be relocate to the new job manager ?
>
> Cheers
>
> On Sat, Feb 27, 2016 at 7:27 AM, Welly Tambunan  wrote:
>>
>> Hi All,
>>
>> We have already try to setup the Job Manager HA based on the documentation
>> and using script and provided zookeeper. It works.
>>
>> However currently everything is done using start-cluster script that I
>> believe will require passwordlress ssh between node. We are restricted with
>> our environment so this one is not possible.
>>
>> Is it possible to setup the Job Manager HA manually ? By starting each job
>> manager with in each node and task manager. We have our zookeeper and hdfs
>> cluster already.
>>
>> We have tried this one the job manager can failover, but the task manager
>> can be relocated to the new task manager. Is there some settings for this
>> one ? Or is the task manager also can be relocate to the new job manager ?
>>
>> Any more details on the mechanism used on Job Manager HA and interaction
>> with Zookeeper ?
>>
>> Is task manager also registered on Zookeeper ? How they find the right job
>> manager master ?
>>
>>
>> Thanks for your help.
>>
>> Cheers
>> --
>> Welly Tambunan
>> Triplelands
>>
>> http://weltam.wordpress.com
>> http://www.triplelands.com
>
>
>
>
> --
> Welly Tambunan
> Triplelands
>
> http://weltam.wordpress.com
> http://www.triplelands.com

Re: Job Manager HA manual setup

[Welly Tambunan]

typos


We have tried this one the job manager can failover, but the task manager
CAN'T be relocated to the new task manager. Is there some settings for this
one ? Or is the task manager also can be relocate to the new job manager ?

Cheers

On Sat, Feb 27, 2016 at 7:27 AM, Welly Tambunan  wrote:

> Hi All,
>
> We have already try to setup the Job Manager HA based on the documentation
> and using script and provided zookeeper. It works.
>
> However currently everything is done using start-cluster script that I
> believe will require passwordlress ssh between node. We are restricted with
> our environment so this one is not possible.
>
> Is it possible to setup the Job Manager HA manually ? By starting each job
> manager with in each node and task manager. We have our zookeeper and hdfs
> cluster already.
>
> We have tried this one the job manager can failover, but the task manager
> can be relocated to the new task manager. Is there some settings for this
> one ? Or is the task manager also can be relocate to the new job manager ?
>
> Any more details on the mechanism used on Job Manager HA and interaction
> with Zookeeper ?
>
> Is task manager also registered on Zookeeper ? How they find the right job
> manager master ?
>
>
> Thanks for your help.
>
> Cheers
> --
> Welly Tambunan
> Triplelands
>
> http://weltam.wordpress.com
> http://www.triplelands.com 
>



-- 
Welly Tambunan
Triplelands

http://weltam.wordpress.com
http://www.triplelands.com 

Job Manager HA manual setup

[Welly Tambunan]

Hi All,

We have already try to setup the Job Manager HA based on the documentation
and using script and provided zookeeper. It works.

However currently everything is done using start-cluster script that I
believe will require passwordlress ssh between node. We are restricted with
our environment so this one is not possible.

Is it possible to setup the Job Manager HA manually ? By starting each job
manager with in each node and task manager. We have our zookeeper and hdfs
cluster already.

We have tried this one the job manager can failover, but the task manager
can be relocated to the new task manager. Is there some settings for this
one ? Or is the task manager also can be relocate to the new job manager ?

Any more details on the mechanism used on Job Manager HA and interaction
with Zookeeper ?

Is task manager also registered on Zookeeper ? How they find the right job
manager master ?


Thanks for your help.

Cheers
-- 
Welly Tambunan
Triplelands

http://weltam.wordpress.com
http://www.triplelands.com