Jon, Ted
Happy new year.
After read more, I understand knowing the host list or single host of
master role in Slider script or template is not supported by either Slider
or YARN.
So what I can do is
a) pre-define a group of hosts with label of master
b) specify label of master for master role in app resource specfication
c) get host list with label of master in template/scripts when deploying
slave role
However I did not find doc how to get host list with label of master in
Slider template/scripts like rs_hosts as follows
{% for host in rs_hosts %}{{host}}
{% endfor %}
Any suggestion?
Thanks,
Yong
On Wed, Dec 31, 2014 at 5:18 PM, Yong Feng <fengyong...@gmail.com> wrote:
> Hi John
>
> I do expect the hosts allocated to master role of my slider application
> are labeled as appname_rolename, but the question is who will add such
> label and how? and how to get the hosts with the label?
>
> Thanks,
>
> Yong
>
> On Wed, Dec 31, 2014 at 3:57 PM, Jon Maron <jma...@hortonworks.com> wrote:
>
>> I’m confused - if certain hosts were labeled as MASTER, wouldn’t you know
>> a priori the list of master hosts?
>>
>>
>> On Dec 31, 2014, at 12:31 PM, Yong Feng <fengyong...@gmail.com> wrote:
>>
>> > Thanks Ted.
>> >
>> >>> knowing the host list of master role in Slider script or template is
>> not
>> > supported by either Slider or YARN.
>> > [Yong] If there is only one instance of master role, at least I could
>> know
>> > its host name by COMPONENT_HOST in slave hosts, right? If so, I could
>> work
>> > it around to just configure one master.
>> >
>> > Yong
>> >
>> > On Tue, Dec 30, 2014 at 10:59 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>> >
>> >> What's the timeframe for this migration ?
>> >>
>> >> If time permits, you should consider rewriting master failover logic in
>> >> your application.
>> >> Using zookeeper is a common technique. HBase, e.g., registers active
>> master
>> >> under a well known zookeeper znode.
>> >>
>> >> This is much more flexible than the way you described.
>> >>
>> >> Again, knowing the host list of master role in Slider script or
>> template is
>> >> not supported by either Slider or YARN.
>> >>
>> >> Cheers
>> >>
>> >> On Tue, Dec 30, 2014 at 7:50 PM, Yong Feng <fengyong...@gmail.com>
>> wrote:
>> >>
>> >>> ah, rs_hosts means all the nodes managed by YARN.
>> >>>
>> >>> I know a little about labels feature of YARN which adds a tag to host
>> and
>> >>> allow yarn API client to filter hosts according to the tag. I don't
>> think
>> >>> it helps my case.
>> >>>
>> >>> Taking a typical master/slave cluster as an example. To support
>> >>> active-passive fail over, we need define multiple instances for master
>> >> role
>> >>> (unfortunately it does not use Zookeeper as the cluster is implemented
>> >> 10+
>> >>> years ago before Zookeeper). In slave node, a list of mast host are
>> >>> specified in conf file, so that slave node could contact master.
>> >>>
>> >>> To deploy such cluster, we need know host list of master and configure
>> >> them
>> >>> in the conf file of slave. Changing membership of master need restart
>> >> whole
>> >>> cluster so we will not support flex up/down role of master.
>> >>>
>> >>> How can I know the host list of master role in Slider script or
>> template?
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Yong
>> >>>
>> >>> On Tue, Dec 30, 2014 at 9:08 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>> >>>
>> >>>> bq. host list of all the instances of a component
>> >>>>
>> >>>> Are you using node labels feature of YARN ?
>> >>>> If not, the list would comprise of all the nodes in your YARN
>> cluster.
>> >>>>
>> >>>> Cheers
>> >>>>
>> >>>> On Tue, Dec 30, 2014 at 4:43 PM, Yong Feng <fengyong...@gmail.com>
>> >>> wrote:
>> >>>>
>> >>>>> Hi Team,
>> >>>>>
>> >>>>> The template of regionserver of HBase application uses following
>> >>> scripts
>> >>>> to
>> >>>>> configure configuraiton file of regionserver of HBase. I did not
>> find
>> >>>> where
>> >>>>> the rs_hosts is defined in any file in Slider HBase package. How a
>> >>>> template
>> >>>>> is designed is mentioned a little at the end of
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> http://slider.incubator.apache.org/docs/slider_specs/application_package.html
>> >>>>> ,
>> >>>>> but cannot help much.
>> >>>>>
>> >>>>> {% for host in rs_hosts %}{{host}}
>> >>>>> {% endfor %}
>> >>>>>
>> >>>>> Does anyone know where rs_hosts is defined? I am now writing a
>> >> package
>> >>>> for
>> >>>>> a HPC applicaiton and need to know host list of all the instances of
>> >> a
>> >>>>> component to prepare configuration file of the application.
>> >>>>>
>> >>>>> Thanks,
>> >>>>>
>> >>>>> Yong
>> >>>>>
>> >>>>
>> >>>
>> >>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
