Hi John. Thanks for more info. I'd like to pry more for a little bit of information (in-lined):
On Thu, Dec 04, 2014 at 04:19PM, John Speidel wrote: > Hi Cos, > As mentioned above, it isn't currently possible to augment an existing > stack at runtime with new services/components. You could possibly change > the stack associated with a running cluster to a new stack which contains > the required services/components, but this new stack definition would need > to be available to ambari so it really isn't very dynamic. We understand Let me describe this scenario as I understood this. Let's say I have two stacks: - StackA (with usual set of components), currently installed on a cluster - StackB == StackA + ComponentX So, if I add StackB definition to the Ambari server and restart it, will I be able to switch from StackA to StackB _without_ resintalling the cluster, and then add/configure ComponentX's service? If my depiction is correct then it might be an acceptable workaround in the absence of dynamic extensibility. > that going forward that the stack framework will need to be more extensible > and dynamic as additional stack definitions are created. Some of the > underlying work to make stack processing more flexible has already been > done in anticipation of new requirements in this area. Shivaji Dutta, in an offline email exchange, pointed me to AMBARI-7175 and AMBARI-7201, which is seemingly going into the right direction, but still would require a stack re-definition, if I read it right. > This is actually the second time that this scenario has came up for me > today so it would be nice to get a Jira filed for an enhancement where the > community can start to discuss the associated use cases. I will start a JIRA and put together the set of requirements. Regards, Cos > -John > > On Tue, Dec 2, 2014 at 2:00 PM, Konstantin Boudnik <[email protected]> wrote: > > > Thanks for chiming in, Erin! Do you know if there's any plan to have > > dynamic > > extensibility for services? For instance, Cloudera Manager has this > > concept of > > Custom Services. And while it was done as an added late fix for parcels, it > > provides a way of adding new software components into an existing Hadoop > > cluster. Which seems to be a pretty handy concept, considering the liquid > > state of the whole Hadoop ecosystem. > > > > It seems that this functionality, if desired, is going to be brand new and > > as > > such require not just development but also re-consideration at the design > > level. Am I right? Appreciate the thoughts! > > > > The best part of active-active masters paradigm is that a client can work > > with > > either of them without a concern of which one is ahead or behind of the > > rest > > as they are equal. Essentially this solves some major issues that pester > > active-standby solutions. > > > > And yes, we do install the services to the cluster's nodes as you > > suggested in > > the later part of your email - after all they are just Linux daemons > > (although > > Ambari tries to hide the fact for whatever reason that is). However this > > presents the core of the very issue: Ambari isn't aware about services > > installed outside of its realm, and it doesn't provide any monitoring or > > life-cycle management for those. And because of that I am trying to figure > > out > > what can be done in this regards to have this dynamism. > > > > With regards, > > Cos > > > > On Tue, Dec 02, 2014 at 08:41AM, Erin Boyd wrote: > > > Hi Cos, > > > Currently, adding services dynamically is not supported in Ambari. It is > > > generally done, as you mentioned through the creation of a stack with the > > > services definition defined within in. I also don't believe we currently > > > support multiple masters on the same cluster. How would one > > deferientiate > > > which master it should be using in such an environment? > > > > > > Of course services can be installed independent of Ambari on the hosts > > > inside the cluster...but I don't think that is what you are getting at. > > Are > > > you expecting the services to have a presence on the UI though installed > > > outside of Ambari? > > > > > > Erin > > > > > > ----- Original Message ----- > > > From: "Konstantin Boudnik" <[email protected]> > > > To: [email protected] > > > Sent: Monday, December 1, 2014 6:42:30 PM > > > Subject: About extensibility of Ambari stacks (not inheritance) > > > > > > Guys, > > > > > > I am looking into possible stack extensibility properties of Ambari (not > > to be > > > confused with inheritance), but haven't been able to derive any final > > > conclusions just yet. Hence, I'd appreciate the input from the people > > behind > > > the system. > > > > > > I have a few questions about current state of the Ambari (version 1.6.1 > > and > > > coming 1.7, and possibly later?) with regards to ability to expand an > > existing > > > stack definitions with a 3rd party services and do it in the runtime, > > rather > > > than only during the installation. We need to be able to run a multiple > > > instances of our master service in the cluster, which isn't typical for > > > "normal" Hadoop concept where only one master can exist for any giving > > > service. > > > > > > Our use case is to be able to amend an exiting cluster setup (HDP, > > Bigtop, > > > etc.) with a new service running on top of HDFS; but not to reinstall the > > > whole stack. The reason we have the use case is that oftentimes our > > software > > > is being added to an existing 3rd party environment, as an added bonus, > > not > > > available during the initial planning and setup of the cluster. > > > > > > The way I understand Ambari's stack inheritance is that a scion stack > > will > > > be a brand new entity, e.g. I won't be able to cherry-pick services from > > it > > > and add them to the already installed parent stack's cluster, right? > > > > > > So far from I see it doesn't seem possible without introducing a > > brand-new > > > version of a stack e.g. 'stack inheritance'. Which, unfortunately, won't > > work > > > as per my explanation of the use above case. I guess another way to look > > at it > > > is this: would it be possible to add a component (or a service) that will > > > override an existing component or a service, but without a need to > > reinstall > > > the rest of the stack? > > > > > > Thanks in advance for any info/ideas. > > > Cos > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
