release Thanks for the info John. I think I am getting there, but not completely....
So, here's what I've done: - added service, bounced Ambari server - curl to http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/stacks/HDP/ says that I have new stack { "href" : "http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/stacks/HDP/versions/2.1.WANdisco";, "Versions" : { "stack_name" : "HDP", "stack_version" : "2.1.WANdisco" } } - ran update curl -u admin:admin -i -X PUT -H 'X-Requested-By: ambari' -d '{"Clusters":{"version":"HDP-2.1.WANdisco"}}' http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/DC-2 The result was: HTTP/1.1 200 OK Set-Cookie: AMBARISESSIONID=vueucwdh8p6mkmcygj803vi9;Path=/ Expires: Thu, 01 Jan 1970 00:00:00 GMT Content-Type: text/plain Content-Length: 0 Server: Jetty(7.6.7.v20120910) However, running curl -u admin:admin http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/ Still gives me this.... { "href" : "http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/";, "items" : [ { "href" : "http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/DC-2";, "Clusters" : { "cluster_name" : "DC-2", "version" : "HDP-2.1" } } ] Any ideas/hints? Thank you very much for your help! Cos On Thu, Dec 04, 2014 at 08:08PM, John Speidel wrote: > Cos, > > Yes, if you are able to push a new stack to the ambari server hosts > filesystem, you can use a REST api call to update the cluster to be > associated with the new stack. > > So, the steps would would look like this: > - install cluster with existing stack > - create new stack with additional services/components > - write the new stack to the filesystem of the ambari server host > - bounce ambari server > -- after the server restart, the hash of the stacks directory is > recalculated which will result in the new updated stack to be pushed to the > agents (not immediate, the check of the stack dir hash occurs every n(not > sure of exact value) heartbeats from the agents) > - do an update (PUT) on the cluster resource changing the value of the > Clusters/version property to point to the new stack > > When these steps are completed the new service/components should be > recognized by ambari. > > -John > > > On Thu, Dec 4, 2014 at 6:07 PM, Konstantin Boudnik <c...@apache.org> wrote: > > > Hi John. > > > > Thanks for more info. I'd like to pry more for a little bit of information > > (in-lined): > > > > On Thu, Dec 04, 2014 at 04:19PM, John Speidel wrote: > > > Hi Cos, > > > As mentioned above, it isn't currently possible to augment an existing > > > stack at runtime with new services/components. You could possibly change > > > the stack associated with a running cluster to a new stack which contains > > > the required services/components, but this new stack definition would > > need > > > to be available to ambari so it really isn't very dynamic. We understand > > > > Let me describe this scenario as I understood this. Let's say I have two > > stacks: > > - StackA (with usual set of components), currently installed on a cluster > > - StackB == StackA + ComponentX > > So, if I add StackB definition to the Ambari server and restart it, will I > > be > > able to switch from StackA to StackB _without_ resintalling the cluster, > > and > > then add/configure ComponentX's service? > > > > If my depiction is correct then it might be an acceptable workaround in the > > absence of dynamic extensibility. > > > > > that going forward that the stack framework will need to be more > > extensible > > > and dynamic as additional stack definitions are created. Some of the > > > underlying work to make stack processing more flexible has already been > > > done in anticipation of new requirements in this area. > > > > Shivaji Dutta, in an offline email exchange, pointed me to AMBARI-7175 and > > AMBARI-7201, which is seemingly going into the right direction, but still > > would require a stack re-definition, if I read it right. > > > > > This is actually the second time that this scenario has came up for me > > > today so it would be nice to get a Jira filed for an enhancement where > > the > > > community can start to discuss the associated use cases. > > > > I will start a JIRA and put together the set of requirements. > > > > Regards, > > Cos > > > > > -John > > > > > > On Tue, Dec 2, 2014 at 2:00 PM, Konstantin Boudnik <c...@apache.org> > > wrote: > > > > > > > Thanks for chiming in, Erin! Do you know if there's any plan to have > > > > dynamic > > > > extensibility for services? For instance, Cloudera Manager has this > > > > concept of > > > > Custom Services. And while it was done as an added late fix for > > parcels, it > > > > provides a way of adding new software components into an existing > > Hadoop > > > > cluster. Which seems to be a pretty handy concept, considering the > > liquid > > > > state of the whole Hadoop ecosystem. > > > > > > > > It seems that this functionality, if desired, is going to be brand new > > and > > > > as > > > > such require not just development but also re-consideration at the > > design > > > > level. Am I right? Appreciate the thoughts! > > > > > > > > The best part of active-active masters paradigm is that a client can > > work > > > > with > > > > either of them without a concern of which one is ahead or behind of the > > > > rest > > > > as they are equal. Essentially this solves some major issues that > > pester > > > > active-standby solutions. > > > > > > > > And yes, we do install the services to the cluster's nodes as you > > > > suggested in > > > > the later part of your email - after all they are just Linux daemons > > > > (although > > > > Ambari tries to hide the fact for whatever reason that is). However > > this > > > > presents the core of the very issue: Ambari isn't aware about services > > > > installed outside of its realm, and it doesn't provide any monitoring > > or > > > > life-cycle management for those. And because of that I am trying to > > figure > > > > out > > > > what can be done in this regards to have this dynamism. > > > > > > > > With regards, > > > > Cos > > > > > > > > On Tue, Dec 02, 2014 at 08:41AM, Erin Boyd wrote: > > > > > Hi Cos, > > > > > Currently, adding services dynamically is not supported in Ambari. > > It is > > > > > generally done, as you mentioned through the creation of a stack > > with the > > > > > services definition defined within in. I also don't believe we > > currently > > > > > support multiple masters on the same cluster. How would one > > > > deferientiate > > > > > which master it should be using in such an environment? > > > > > > > > > > Of course services can be installed independent of Ambari on the > > hosts > > > > > inside the cluster...but I don't think that is what you are getting > > at. > > > > Are > > > > > you expecting the services to have a presence on the UI though > > installed > > > > > outside of Ambari? > > > > > > > > > > Erin > > > > > > > > > > ----- Original Message ----- > > > > > From: "Konstantin Boudnik" <c...@apache.org> > > > > > To: dev@ambari.apache.org > > > > > Sent: Monday, December 1, 2014 6:42:30 PM > > > > > Subject: About extensibility of Ambari stacks (not inheritance) > > > > > > > > > > Guys, > > > > > > > > > > I am looking into possible stack extensibility properties of Ambari > > (not > > > > to be > > > > > confused with inheritance), but haven't been able to derive any final > > > > > conclusions just yet. Hence, I'd appreciate the input from the people > > > > behind > > > > > the system. > > > > > > > > > > I have a few questions about current state of the Ambari (version > > 1.6.1 > > > > and > > > > > coming 1.7, and possibly later?) with regards to ability to expand an > > > > existing > > > > > stack definitions with a 3rd party services and do it in the runtime, > > > > rather > > > > > than only during the installation. We need to be able to run a > > multiple > > > > > instances of our master service in the cluster, which isn't typical > > for > > > > > "normal" Hadoop concept where only one master can exist for any > > giving > > > > > service. > > > > > > > > > > Our use case is to be able to amend an exiting cluster setup (HDP, > > > > Bigtop, > > > > > etc.) with a new service running on top of HDFS; but not to > > reinstall the > > > > > whole stack. The reason we have the use case is that oftentimes our > > > > software > > > > > is being added to an existing 3rd party environment, as an added > > bonus, > > > > not > > > > > available during the initial planning and setup of the cluster. > > > > > > > > > > The way I understand Ambari's stack inheritance is that a scion stack > > > > will > > > > > be a brand new entity, e.g. I won't be able to cherry-pick services > > from > > > > it > > > > > and add them to the already installed parent stack's cluster, right? > > > > > > > > > > So far from I see it doesn't seem possible without introducing a > > > > brand-new > > > > > version of a stack e.g. 'stack inheritance'. Which, unfortunately, > > won't > > > > work > > > > > as per my explanation of the use above case. I guess another way to > > look > > > > at it > > > > > is this: would it be possible to add a component (or a service) that > > will > > > > > override an existing component or a service, but without a need to > > > > reinstall > > > > > the rest of the stack? > > > > > > > > > > Thanks in advance for any info/ideas. > > > > > Cos > > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or entity > > to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
