Hi Konstantin, Custom services added via the API won't automatically show up in the UI if you are using Ambari 1.6.1 or below. Please try with Ambari 1.7.0.
Thanks, Yusaku On Tue, Dec 9, 2014 at 6:42 PM, Konstantin Boudnik <c...@apache.org> wrote: > Thanks Nate - that does the trick. With the command-line upgrade + a curl > call > to add my new service now I can see it among > > http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/DC-2/services > > The issues I still see are: > - the service doesn't appear in "Add Service" wizard > - even after getting added by POST'ing > {"ServiceInfo":{"service_name":"NSNN"}} info, I don't see the service in > the dashboard. It might be caused by some inconsistencies in my service > definition as I can use the same procedure to add Tez service, and being > able to see it in the dashboard. > > Will dig a bit more into this tomorrow and report back if I hit anything > new. > > Thank you all guys for your help - quite appreciated! > Cos > > On Mon, Dec 08, 2014 at 05:11PM, Nate Cole wrote: > > Cos, > > > > There is an ambari server command that should allow you to switch the > version. Here are the steps I took to change the cluster version: > > > > 1. Install a cluster as normal (I used cluster HDP-2.1 as the stack). > > 2. In /var/lib/ambari-server/resources/stacks/HDP, copy 2.1 to > 2.1.TestStack > > 3. Shutdown ambari. > > 4. ran: ambari-server upgradestack HDP-2.1.TestStack > > 5. Start ambari. > > 6. On the cluster admin page, verified at the top of the page that the > cluster says "HDP-2.1.TestStack" > > > > Hope that helps, > > Nate > > > > On Dec 8, 2014, at 4:24 PM, Konstantin Boudnik <c...@apache.org> wrote: > > > > > On Mon, Dec 08, 2014 at 04:14PM, John Speidel wrote: > > >> Cos, > > >> > > >> I don't know why this is occurring. I have not actually ever changed > the > > >> stack of a running cluster, my response was a result of my > understanding > > >> and talking to others that have done this. > > >> > > >> Hopefully somebody that has actually done this can comment. > > >> In the meantime, I will see if I can find somebody that may may know > why > > >> this isn't working for you. > > > > > > Thank you very much - really appreciate your help! In the interest of > full > > > disclosure: I am using 1.6.1 server, not sure if this is of any > significance. > > > > > > Cos > > > > > >> On Fri, Dec 5, 2014 at 8:38 PM, Konstantin Boudnik <c...@apache.org> > wrote: > > >> > > >>> release Thanks for the info John. I think I am getting there, but not > > >>> completely.... > > >>> > > >>> So, here's what I've done: > > >>> - added service, bounced Ambari server > > >>> > > >>> - curl to > http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/stacks/HDP/ > > >>> says that I have new stack > > >>> { > > >>> "href" : " > > >>> > http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/stacks/HDP/versions/2.1.WANdisco > > >>> ", > > >>> "Versions" : { > > >>> "stack_name" : "HDP", > > >>> "stack_version" : "2.1.WANdisco" > > >>> } > > >>> } > > >>> > > >>> - ran update > > >>> curl -u admin:admin -i -X PUT -H 'X-Requested-By: ambari' -d > > >>> '{"Clusters":{"version":"HDP-2.1.WANdisco"}}' > > >>> http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/DC-2 > > >>> > > >>> The result was: > > >>> HTTP/1.1 200 OK > > >>> Set-Cookie: AMBARISESSIONID=vueucwdh8p6mkmcygj803vi9;Path=/ > > >>> Expires: Thu, 01 Jan 1970 00:00:00 GMT > > >>> Content-Type: text/plain > > >>> Content-Length: 0 > > >>> Server: Jetty(7.6.7.v20120910) > > >>> > > >>> However, running > > >>> curl -u admin:admin > > >>> http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/ > > >>> > > >>> Still gives me this.... > > >>> { > > >>> "href" : " > http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/ > > >>> ", > > >>> "items" : [ > > >>> { > > >>> "href" : " > > >>> http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/DC-2";, > > >>> "Clusters" : { > > >>> "cluster_name" : "DC-2", > > >>> "version" : "HDP-2.1" > > >>> } > > >>> } > > >>> ] > > >>> > > >>> Any ideas/hints? Thank you very much for your help! > > >>> Cos > > >>> > > >>> On Thu, Dec 04, 2014 at 08:08PM, John Speidel wrote: > > >>>> Cos, > > >>>> > > >>>> Yes, if you are able to push a new stack to the ambari server hosts > > >>>> filesystem, you can use a REST api call to update the cluster to be > > >>>> associated with the new stack. > > >>>> > > >>>> So, the steps would would look like this: > > >>>> - install cluster with existing stack > > >>>> - create new stack with additional services/components > > >>>> - write the new stack to the filesystem of the ambari server host > > >>>> - bounce ambari server > > >>>> -- after the server restart, the hash of the stacks directory is > > >>>> recalculated which will result in the new updated stack to be > pushed to > > >>> the > > >>>> agents (not immediate, the check of the stack dir hash occurs every > n(not > > >>>> sure of exact value) heartbeats from the agents) > > >>>> - do an update (PUT) on the cluster resource changing the value of > the > > >>>> Clusters/version property to point to the new stack > > >>>> > > >>>> When these steps are completed the new service/components should be > > >>>> recognized by ambari. > > >>>> > > >>>> -John > > >>>> > > >>>> > > >>>> On Thu, Dec 4, 2014 at 6:07 PM, Konstantin Boudnik <c...@apache.org> > > >>> wrote: > > >>>> > > >>>>> Hi John. > > >>>>> > > >>>>> Thanks for more info. I'd like to pry more for a little bit of > > >>> information > > >>>>> (in-lined): > > >>>>> > > >>>>> On Thu, Dec 04, 2014 at 04:19PM, John Speidel wrote: > > >>>>>> Hi Cos, > > >>>>>> As mentioned above, it isn't currently possible to augment an > > >>> existing > > >>>>>> stack at runtime with new services/components. You could possibly > > >>> change > > >>>>>> the stack associated with a running cluster to a new stack which > > >>> contains > > >>>>>> the required services/components, but this new stack definition > would > > >>>>> need > > >>>>>> to be available to ambari so it really isn't very dynamic. We > > >>> understand > > >>>>> > > >>>>> Let me describe this scenario as I understood this. Let's say I > have > > >>> two > > >>>>> stacks: > > >>>>> - StackA (with usual set of components), currently installed on a > > >>> cluster > > >>>>> - StackB == StackA + ComponentX > > >>>>> So, if I add StackB definition to the Ambari server and restart it, > > >>> will I > > >>>>> be > > >>>>> able to switch from StackA to StackB _without_ resintalling the > > >>> cluster, > > >>>>> and > > >>>>> then add/configure ComponentX's service? > > >>>>> > > >>>>> If my depiction is correct then it might be an acceptable > workaround > > >>> in the > > >>>>> absence of dynamic extensibility. > > >>>>> > > >>>>>> that going forward that the stack framework will need to be more > > >>>>> extensible > > >>>>>> and dynamic as additional stack definitions are created. Some of > the > > >>>>>> underlying work to make stack processing more flexible has already > > >>> been > > >>>>>> done in anticipation of new requirements in this area. > > >>>>> > > >>>>> Shivaji Dutta, in an offline email exchange, pointed me to > AMBARI-7175 > > >>> and > > >>>>> AMBARI-7201, which is seemingly going into the right direction, but > > >>> still > > >>>>> would require a stack re-definition, if I read it right. > > >>>>> > > >>>>>> This is actually the second time that this scenario has came up > for > > >>> me > > >>>>>> today so it would be nice to get a Jira filed for an enhancement > > >>> where > > >>>>> the > > >>>>>> community can start to discuss the associated use cases. > > >>>>> > > >>>>> I will start a JIRA and put together the set of requirements. > > >>>>> > > >>>>> Regards, > > >>>>> Cos > > >>>>> > > >>>>>> -John > > >>>>>> > > >>>>>> On Tue, Dec 2, 2014 at 2:00 PM, Konstantin Boudnik < > c...@apache.org> > > >>>>> wrote: > > >>>>>> > > >>>>>>> Thanks for chiming in, Erin! Do you know if there's any plan to > > >>> have > > >>>>>>> dynamic > > >>>>>>> extensibility for services? For instance, Cloudera Manager has > this > > >>>>>>> concept of > > >>>>>>> Custom Services. And while it was done as an added late fix for > > >>>>> parcels, it > > >>>>>>> provides a way of adding new software components into an existing > > >>>>> Hadoop > > >>>>>>> cluster. Which seems to be a pretty handy concept, considering > the > > >>>>> liquid > > >>>>>>> state of the whole Hadoop ecosystem. > > >>>>>>> > > >>>>>>> It seems that this functionality, if desired, is going to be > brand > > >>> new > > >>>>> and > > >>>>>>> as > > >>>>>>> such require not just development but also re-consideration at > the > > >>>>> design > > >>>>>>> level. Am I right? Appreciate the thoughts! > > >>>>>>> > > >>>>>>> The best part of active-active masters paradigm is that a client > > >>> can > > >>>>> work > > >>>>>>> with > > >>>>>>> either of them without a concern of which one is ahead or behind > > >>> of the > > >>>>>>> rest > > >>>>>>> as they are equal. Essentially this solves some major issues that > > >>>>> pester > > >>>>>>> active-standby solutions. > > >>>>>>> > > >>>>>>> And yes, we do install the services to the cluster's nodes as you > > >>>>>>> suggested in > > >>>>>>> the later part of your email - after all they are just Linux > > >>> daemons > > >>>>>>> (although > > >>>>>>> Ambari tries to hide the fact for whatever reason that is). > However > > >>>>> this > > >>>>>>> presents the core of the very issue: Ambari isn't aware about > > >>> services > > >>>>>>> installed outside of its realm, and it doesn't provide any > > >>> monitoring > > >>>>> or > > >>>>>>> life-cycle management for those. And because of that I am trying > to > > >>>>> figure > > >>>>>>> out > > >>>>>>> what can be done in this regards to have this dynamism. > > >>>>>>> > > >>>>>>> With regards, > > >>>>>>> Cos > > >>>>>>> > > >>>>>>> On Tue, Dec 02, 2014 at 08:41AM, Erin Boyd wrote: > > >>>>>>>> Hi Cos, > > >>>>>>>> Currently, adding services dynamically is not supported in > > >>> Ambari. > > >>>>> It is > > >>>>>>>> generally done, as you mentioned through the creation of a stack > > >>>>> with the > > >>>>>>>> services definition defined within in. I also don't believe we > > >>>>> currently > > >>>>>>>> support multiple masters on the same cluster. How would one > > >>>>>>> deferientiate > > >>>>>>>> which master it should be using in such an environment? > > >>>>>>>> > > >>>>>>>> Of course services can be installed independent of Ambari on the > > >>>>> hosts > > >>>>>>>> inside the cluster...but I don't think that is what you are > > >>> getting > > >>>>> at. > > >>>>>>> Are > > >>>>>>>> you expecting the services to have a presence on the UI though > > >>>>> installed > > >>>>>>>> outside of Ambari? > > >>>>>>>> > > >>>>>>>> Erin > > >>>>>>>> > > >>>>>>>> ----- Original Message ----- > > >>>>>>>> From: "Konstantin Boudnik" <c...@apache.org> > > >>>>>>>> To: dev@ambari.apache.org > > >>>>>>>> Sent: Monday, December 1, 2014 6:42:30 PM > > >>>>>>>> Subject: About extensibility of Ambari stacks (not inheritance) > > >>>>>>>> > > >>>>>>>> Guys, > > >>>>>>>> > > >>>>>>>> I am looking into possible stack extensibility properties of > > >>> Ambari > > >>>>> (not > > >>>>>>> to be > > >>>>>>>> confused with inheritance), but haven't been able to derive any > > >>> final > > >>>>>>>> conclusions just yet. Hence, I'd appreciate the input from the > > >>> people > > >>>>>>> behind > > >>>>>>>> the system. > > >>>>>>>> > > >>>>>>>> I have a few questions about current state of the Ambari > (version > > >>>>> 1.6.1 > > >>>>>>> and > > >>>>>>>> coming 1.7, and possibly later?) with regards to ability to > > >>> expand an > > >>>>>>> existing > > >>>>>>>> stack definitions with a 3rd party services and do it in the > > >>> runtime, > > >>>>>>> rather > > >>>>>>>> than only during the installation. We need to be able to run a > > >>>>> multiple > > >>>>>>>> instances of our master service in the cluster, which isn't > > >>> typical > > >>>>> for > > >>>>>>>> "normal" Hadoop concept where only one master can exist for any > > >>>>> giving > > >>>>>>>> service. > > >>>>>>>> > > >>>>>>>> Our use case is to be able to amend an exiting cluster setup > > >>> (HDP, > > >>>>>>> Bigtop, > > >>>>>>>> etc.) with a new service running on top of HDFS; but not to > > >>>>> reinstall the > > >>>>>>>> whole stack. The reason we have the use case is that oftentimes > > >>> our > > >>>>>>> software > > >>>>>>>> is being added to an existing 3rd party environment, as an added > > >>>>> bonus, > > >>>>>>> not > > >>>>>>>> available during the initial planning and setup of the cluster. > > >>>>>>>> > > >>>>>>>> The way I understand Ambari's stack inheritance is that a scion > > >>> stack > > >>>>>>> will > > >>>>>>>> be a brand new entity, e.g. I won't be able to cherry-pick > > >>> services > > >>>>> from > > >>>>>>> it > > >>>>>>>> and add them to the already installed parent stack's cluster, > > >>> right? > > >>>>>>>> > > >>>>>>>> So far from I see it doesn't seem possible without introducing a > > >>>>>>> brand-new > > >>>>>>>> version of a stack e.g. 'stack inheritance'. Which, > > >>> unfortunately, > > >>>>> won't > > >>>>>>> work > > >>>>>>>> as per my explanation of the use above case. I guess another way > > >>> to > > >>>>> look > > >>>>>>> at it > > >>>>>>>> is this: would it be possible to add a component (or a service) > > >>> that > > >>>>> will > > >>>>>>>> override an existing component or a service, but without a need > > >>> to > > >>>>>>> reinstall > > >>>>>>>> the rest of the stack? > > >>>>>>>> > > >>>>>>>> Thanks in advance for any info/ideas. > > >>>>>>>> Cos > > >>>>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> CONFIDENTIALITY NOTICE > > >>>>>> NOTICE: This message is intended for the use of the individual or > > >>> entity > > >>>>> to > > >>>>>> which it is addressed and may contain information that is > > >>> confidential, > > >>>>>> privileged and exempt from disclosure under applicable law. If the > > >>> reader > > >>>>>> of this message is not the intended recipient, you are hereby > > >>> notified > > >>>>> that > > >>>>>> any printing, copying, dissemination, distribution, disclosure or > > >>>>>> forwarding of this communication is strictly prohibited. If you > have > > >>>>>> received this communication in error, please contact the sender > > >>>>> immediately > > >>>>>> and delete it from your system. Thank You. > > >>>>> > > >>>>> > > >>>> > > >>>> -- > > >>>> CONFIDENTIALITY NOTICE > > >>>> NOTICE: This message is intended for the use of the individual or > entity > > >>> to > > >>>> which it is addressed and may contain information that is > confidential, > > >>>> privileged and exempt from disclosure under applicable law. If the > reader > > >>>> of this message is not the intended recipient, you are hereby > notified > > >>> that > > >>>> any printing, copying, dissemination, distribution, disclosure or > > >>>> forwarding of this communication is strictly prohibited. If you have > > >>>> received this communication in error, please contact the sender > > >>> immediately > > >>>> and delete it from your system. Thank You. > > >>> > > >> > > >> -- > > >> CONFIDENTIALITY NOTICE > > >> NOTICE: This message is intended for the use of the individual or > entity to > > >> which it is addressed and may contain information that is > confidential, > > >> privileged and exempt from disclosure under applicable law. If the > reader > > >> of this message is not the intended recipient, you are hereby > notified that > > >> any printing, copying, dissemination, distribution, disclosure or > > >> forwarding of this communication is strictly prohibited. If you have > > >> received this communication in error, please contact the sender > immediately > > >> and delete it from your system. Thank You. > > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
