On Mon, Dec 08, 2014 at 04:14PM, John Speidel wrote: > Cos, > > I don't know why this is occurring. I have not actually ever changed the > stack of a running cluster, my response was a result of my understanding > and talking to others that have done this. > > Hopefully somebody that has actually done this can comment. > In the meantime, I will see if I can find somebody that may may know why > this isn't working for you.
Thank you very much - really appreciate your help! In the interest of full disclosure: I am using 1.6.1 server, not sure if this is of any significance. Cos > On Fri, Dec 5, 2014 at 8:38 PM, Konstantin Boudnik <c...@apache.org> wrote: > > > release Thanks for the info John. I think I am getting there, but not > > completely.... > > > > So, here's what I've done: > > - added service, bounced Ambari server > > > > - curl to http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/stacks/HDP/ > > says that I have new stack > > { > > "href" : " > > http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/stacks/HDP/versions/2.1.WANdisco > > ", > > "Versions" : { > > "stack_name" : "HDP", > > "stack_version" : "2.1.WANdisco" > > } > > } > > > > - ran update > > curl -u admin:admin -i -X PUT -H 'X-Requested-By: ambari' -d > > '{"Clusters":{"version":"HDP-2.1.WANdisco"}}' > > http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/DC-2 > > > > The result was: > > HTTP/1.1 200 OK > > Set-Cookie: AMBARISESSIONID=vueucwdh8p6mkmcygj803vi9;Path=/ > > Expires: Thu, 01 Jan 1970 00:00:00 GMT > > Content-Type: text/plain > > Content-Length: 0 > > Server: Jetty(7.6.7.v20120910) > > > > However, running > > curl -u admin:admin > > http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/ > > > > Still gives me this.... > > { > > "href" : "http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/ > > ", > > "items" : [ > > { > > "href" : " > > http://vmhost05-hbase3.bdva.wandisco.com:8080/api/v1/clusters/DC-2";, > > "Clusters" : { > > "cluster_name" : "DC-2", > > "version" : "HDP-2.1" > > } > > } > > ] > > > > Any ideas/hints? Thank you very much for your help! > > Cos > > > > On Thu, Dec 04, 2014 at 08:08PM, John Speidel wrote: > > > Cos, > > > > > > Yes, if you are able to push a new stack to the ambari server hosts > > > filesystem, you can use a REST api call to update the cluster to be > > > associated with the new stack. > > > > > > So, the steps would would look like this: > > > - install cluster with existing stack > > > - create new stack with additional services/components > > > - write the new stack to the filesystem of the ambari server host > > > - bounce ambari server > > > -- after the server restart, the hash of the stacks directory is > > > recalculated which will result in the new updated stack to be pushed to > > the > > > agents (not immediate, the check of the stack dir hash occurs every n(not > > > sure of exact value) heartbeats from the agents) > > > - do an update (PUT) on the cluster resource changing the value of the > > > Clusters/version property to point to the new stack > > > > > > When these steps are completed the new service/components should be > > > recognized by ambari. > > > > > > -John > > > > > > > > > On Thu, Dec 4, 2014 at 6:07 PM, Konstantin Boudnik <c...@apache.org> > > wrote: > > > > > > > Hi John. > > > > > > > > Thanks for more info. I'd like to pry more for a little bit of > > information > > > > (in-lined): > > > > > > > > On Thu, Dec 04, 2014 at 04:19PM, John Speidel wrote: > > > > > Hi Cos, > > > > > As mentioned above, it isn't currently possible to augment an > > existing > > > > > stack at runtime with new services/components. You could possibly > > change > > > > > the stack associated with a running cluster to a new stack which > > contains > > > > > the required services/components, but this new stack definition would > > > > need > > > > > to be available to ambari so it really isn't very dynamic. We > > understand > > > > > > > > Let me describe this scenario as I understood this. Let's say I have > > two > > > > stacks: > > > > - StackA (with usual set of components), currently installed on a > > cluster > > > > - StackB == StackA + ComponentX > > > > So, if I add StackB definition to the Ambari server and restart it, > > will I > > > > be > > > > able to switch from StackA to StackB _without_ resintalling the > > cluster, > > > > and > > > > then add/configure ComponentX's service? > > > > > > > > If my depiction is correct then it might be an acceptable workaround > > in the > > > > absence of dynamic extensibility. > > > > > > > > > that going forward that the stack framework will need to be more > > > > extensible > > > > > and dynamic as additional stack definitions are created. Some of the > > > > > underlying work to make stack processing more flexible has already > > been > > > > > done in anticipation of new requirements in this area. > > > > > > > > Shivaji Dutta, in an offline email exchange, pointed me to AMBARI-7175 > > and > > > > AMBARI-7201, which is seemingly going into the right direction, but > > still > > > > would require a stack re-definition, if I read it right. > > > > > > > > > This is actually the second time that this scenario has came up for > > me > > > > > today so it would be nice to get a Jira filed for an enhancement > > where > > > > the > > > > > community can start to discuss the associated use cases. > > > > > > > > I will start a JIRA and put together the set of requirements. > > > > > > > > Regards, > > > > Cos > > > > > > > > > -John > > > > > > > > > > On Tue, Dec 2, 2014 at 2:00 PM, Konstantin Boudnik <c...@apache.org> > > > > wrote: > > > > > > > > > > > Thanks for chiming in, Erin! Do you know if there's any plan to > > have > > > > > > dynamic > > > > > > extensibility for services? For instance, Cloudera Manager has this > > > > > > concept of > > > > > > Custom Services. And while it was done as an added late fix for > > > > parcels, it > > > > > > provides a way of adding new software components into an existing > > > > Hadoop > > > > > > cluster. Which seems to be a pretty handy concept, considering the > > > > liquid > > > > > > state of the whole Hadoop ecosystem. > > > > > > > > > > > > It seems that this functionality, if desired, is going to be brand > > new > > > > and > > > > > > as > > > > > > such require not just development but also re-consideration at the > > > > design > > > > > > level. Am I right? Appreciate the thoughts! > > > > > > > > > > > > The best part of active-active masters paradigm is that a client > > can > > > > work > > > > > > with > > > > > > either of them without a concern of which one is ahead or behind > > of the > > > > > > rest > > > > > > as they are equal. Essentially this solves some major issues that > > > > pester > > > > > > active-standby solutions. > > > > > > > > > > > > And yes, we do install the services to the cluster's nodes as you > > > > > > suggested in > > > > > > the later part of your email - after all they are just Linux > > daemons > > > > > > (although > > > > > > Ambari tries to hide the fact for whatever reason that is). However > > > > this > > > > > > presents the core of the very issue: Ambari isn't aware about > > services > > > > > > installed outside of its realm, and it doesn't provide any > > monitoring > > > > or > > > > > > life-cycle management for those. And because of that I am trying to > > > > figure > > > > > > out > > > > > > what can be done in this regards to have this dynamism. > > > > > > > > > > > > With regards, > > > > > > Cos > > > > > > > > > > > > On Tue, Dec 02, 2014 at 08:41AM, Erin Boyd wrote: > > > > > > > Hi Cos, > > > > > > > Currently, adding services dynamically is not supported in > > Ambari. > > > > It is > > > > > > > generally done, as you mentioned through the creation of a stack > > > > with the > > > > > > > services definition defined within in. I also don't believe we > > > > currently > > > > > > > support multiple masters on the same cluster. How would one > > > > > > deferientiate > > > > > > > which master it should be using in such an environment? > > > > > > > > > > > > > > Of course services can be installed independent of Ambari on the > > > > hosts > > > > > > > inside the cluster...but I don't think that is what you are > > getting > > > > at. > > > > > > Are > > > > > > > you expecting the services to have a presence on the UI though > > > > installed > > > > > > > outside of Ambari? > > > > > > > > > > > > > > Erin > > > > > > > > > > > > > > ----- Original Message ----- > > > > > > > From: "Konstantin Boudnik" <c...@apache.org> > > > > > > > To: dev@ambari.apache.org > > > > > > > Sent: Monday, December 1, 2014 6:42:30 PM > > > > > > > Subject: About extensibility of Ambari stacks (not inheritance) > > > > > > > > > > > > > > Guys, > > > > > > > > > > > > > > I am looking into possible stack extensibility properties of > > Ambari > > > > (not > > > > > > to be > > > > > > > confused with inheritance), but haven't been able to derive any > > final > > > > > > > conclusions just yet. Hence, I'd appreciate the input from the > > people > > > > > > behind > > > > > > > the system. > > > > > > > > > > > > > > I have a few questions about current state of the Ambari (version > > > > 1.6.1 > > > > > > and > > > > > > > coming 1.7, and possibly later?) with regards to ability to > > expand an > > > > > > existing > > > > > > > stack definitions with a 3rd party services and do it in the > > runtime, > > > > > > rather > > > > > > > than only during the installation. We need to be able to run a > > > > multiple > > > > > > > instances of our master service in the cluster, which isn't > > typical > > > > for > > > > > > > "normal" Hadoop concept where only one master can exist for any > > > > giving > > > > > > > service. > > > > > > > > > > > > > > Our use case is to be able to amend an exiting cluster setup > > (HDP, > > > > > > Bigtop, > > > > > > > etc.) with a new service running on top of HDFS; but not to > > > > reinstall the > > > > > > > whole stack. The reason we have the use case is that oftentimes > > our > > > > > > software > > > > > > > is being added to an existing 3rd party environment, as an added > > > > bonus, > > > > > > not > > > > > > > available during the initial planning and setup of the cluster. > > > > > > > > > > > > > > The way I understand Ambari's stack inheritance is that a scion > > stack > > > > > > will > > > > > > > be a brand new entity, e.g. I won't be able to cherry-pick > > services > > > > from > > > > > > it > > > > > > > and add them to the already installed parent stack's cluster, > > right? > > > > > > > > > > > > > > So far from I see it doesn't seem possible without introducing a > > > > > > brand-new > > > > > > > version of a stack e.g. 'stack inheritance'. Which, > > unfortunately, > > > > won't > > > > > > work > > > > > > > as per my explanation of the use above case. I guess another way > > to > > > > look > > > > > > at it > > > > > > > is this: would it be possible to add a component (or a service) > > that > > > > will > > > > > > > override an existing component or a service, but without a need > > to > > > > > > reinstall > > > > > > > the rest of the stack? > > > > > > > > > > > > > > Thanks in advance for any info/ideas. > > > > > > > Cos > > > > > > > > > > > > > > > > -- > > > > > CONFIDENTIALITY NOTICE > > > > > NOTICE: This message is intended for the use of the individual or > > entity > > > > to > > > > > which it is addressed and may contain information that is > > confidential, > > > > > privileged and exempt from disclosure under applicable law. If the > > reader > > > > > of this message is not the intended recipient, you are hereby > > notified > > > > that > > > > > any printing, copying, dissemination, distribution, disclosure or > > > > > forwarding of this communication is strictly prohibited. If you have > > > > > received this communication in error, please contact the sender > > > > immediately > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or entity > > to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
