I agree that named provider config should suffice. HA and AclsAuth reference services within the topology and therfore will present a challenge for reuse across topologies.
We'll need to figure that out. On Aug 31, 2017 9:12 AM, "Philip Zampino" <pzamp...@gmail.com> wrote: > Concerning the externalized provider configuration, the idea is to have > sets of "named" (unique ID'd) config, which can be referenced by any Knox > topology. > So, you could have a single provider configuration that is applied (by > reference) to all your cluster topologies, if that made sense for your > deployment. > This will allow a change to a single config (externalized provider config) > to affect all the clusters topologies that reference it; rather than what > is required today, which is to make the same change in every affected > cluster topology config. > Does that answer your question? > > I'll defer the HA question because I'm not yet familiar enough with it. > > Thanks for the feedback, > Phil > > On Thu, Aug 31, 2017 at 3:42 AM, Krishna Pandey <krish.pande...@gmail.com> > wrote: > > > > > > > As a result, I'm wondering if Knox should support provider > configuration > > > references in topology.xml, rather than having to duplicate it across > > > descriptors. > > > > > > +1 to Phil's remark above. Adding to that as Knox provides multi-cluster > > support, can we have external provider config cluster-wise? Or we can > > define multiple external provider config in single "global" provider > config > > file and refer them with some key? > > > > Also, when services are deployed in HA, we specify HaProvider and > multiple > > Service URLs in the topology. Will it make sense to use Load balancer IP > > address / External URL for those services in topology instead of multiple > > service URLs and using HaProvider? > > > > Thanks, > > Krishna Pandey > > > > On Wed, Aug 30, 2017 at 10:05 PM, larry mccay <lmc...@apache.org> wrote: > > > > > Terrific, Phil! > > > > > > > > > On Wed, Aug 30, 2017 at 11:03 AM, Philip Zampino <pzamp...@gmail.com> > > > wrote: > > > > > > > After giving this KIP some thought, I would like to work on it. I've > > > added > > > > more detail to the wiki, and I'll create a JIRA for it. > > > > > > > > On Fri, Aug 25, 2017 at 10:03 AM, larry mccay <lmc...@apache.org> > > wrote: > > > > > > > > > Wonderful details, Phil! > > > > > > > > > > > > > > > On Fri, Aug 25, 2017 at 9:40 AM, Philip Zampino < > pzamp...@gmail.com> > > > > > wrote: > > > > > > > > > > > Thanks! I've added the Ambari API details to the wiki. > > > > > > > > > > > > On Fri, Aug 25, 2017 at 8:27 AM, larry mccay <lmc...@apache.org> > > > > wrote: > > > > > > > > > > > > > HI Phil - > > > > > > > > > > > > > > Thank you for digging into this topic! > > > > > > > > > > > > > > I've added you as a contributor to the wiki and you should be > > able > > > to > > > > > > edit > > > > > > > the KIP now. > > > > > > > > > > > > > > thanks, > > > > > > > > > > > > > > --larry > > > > > > > > > > > > > > On Thu, Aug 24, 2017 at 3:45 PM, Philip Zampino < > > > pzamp...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > I've put together a quick python POC targeting Ambari as the > > > > > discovery > > > > > > > > source, just to prove that we can indeed get the necessary > > > details > > > > > via > > > > > > > > Ambari's REST API. > > > > > > > > It generates a proper topology.xml descriptor based on a > simple > > > > > > > descriptor > > > > > > > > (I chose YAML for this POC), which has a reference to what > I've > > > > > called > > > > > > a > > > > > > > > policy descriptor (the <gateway/> portion of the topology > > > > > descriptor). > > > > > > > > > > > > > > > > I am currently unable to update the KIP, but I can share the > > REST > > > > > APIs > > > > > > > I've > > > > > > > > employed if there is interest. > > > > > > > > > > > > > > > > As a result, I'm wondering if Knox should support provider > > > > > > configuration > > > > > > > > references in topology.xml, rather than having to duplicate > it > > > > across > > > > > > > > descriptors. > > > > > > > > So, instead of <gateway><provider>...</gateway> in each > > > > > topology.xml, > > > > > > > have > > > > > > > > a single element that points to an external provider config > > > (e.g., > > > > > > > > <policy-ref>$GATEWAY_HOME/conf/policy/my-named-provider- > > > > > > > > config.xml</policy-ref>). > > > > > > > > I've already externalized it for input to the POC, but I'm > > still > > > > > > copying > > > > > > > > the contents into the resulting topology descriptor; I think > it > > > > would > > > > > > be > > > > > > > > better to copy only the reference. > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > - Phil > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Aug 18, 2017 at 1:55 PM, larry mccay < > > lmc...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > > > > > Good to hear, Phil. > > > > > > > > > > > > > > > > > > Yes, I was looking to go back and add some of the API > > specifics > > > > and > > > > > > > > > investigation details for at least Ambari and ZK. > > > > > > > > > If others make sense to add such as Consul, etcd, etc that > > > would > > > > be > > > > > > > good > > > > > > > > as > > > > > > > > > well and it would help to tease out some of what may be > > needed > > > > for > > > > > > the > > > > > > > > > abstraction API and config. > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Aug 18, 2017 at 1:52 PM, Philip Zampino < > > > > > pzamp...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > This sounds like a good improvement from a usability > > > > perspective. > > > > > > > > > > > > > > > > > > > > I agree that Knox must continue its support for direct > > > topology > > > > > > (XML) > > > > > > > > > > definitions, but for those deployments where there is a > > > service > > > > > > > > registry > > > > > > > > > > (e.g., Ambari, Zookeeper, Consul, etc...) with knowledge > of > > > the > > > > > > > > topology, > > > > > > > > > > the simplified config coupled with the service discovery > > > could > > > > > > reduce > > > > > > > > > > topology (especially ServiceConnectivity) errors. > > > > > > > > > > > > > > > > > > > > I also agree that consolidating provider configurations > > into > > > > > named > > > > > > > > sets, > > > > > > > > > > which can be referenced from topology configurations, > will > > > be a > > > > > > good > > > > > > > > > > change, especially if there are commonly grouped > providers > > > > > employed > > > > > > > by > > > > > > > > > > multiple topologies. > > > > > > > > > > > > > > > > > > > > Concerning the TODO sections, is your intention is that > > they > > > > > > contain > > > > > > > > the > > > > > > > > > > respective API facilities that will provide the > information > > > > > needed > > > > > > to > > > > > > > > > > populate a topology definition? > > > > > > > > > > > > > > > > > > > > - Phil > > > > > > > > > > > > > > > > > > > > On Fri, Aug 18, 2017 at 12:39 PM, larry mccay < > > > > lmc...@apache.org > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > All - > > > > > > > > > > > > > > > > > > > > > > I have published an initial stab at a proposal for > > > > simplifying > > > > > > > > topology > > > > > > > > > > > management [1] - as I briefly mentioned on the 0.14.0 > > > > planning > > > > > > > > thread. > > > > > > > > > > > > > > > > > > > > > > There are a couple TODO sections that need some > > > investigation > > > > > for > > > > > > > > > Ambari > > > > > > > > > > > API and Zookeeper Registry API as discovery plugins. > > > > > > > > > > > > > > > > > > > > > > Please give the KIP a read and let me know if it makes > > > sense > > > > or > > > > > > if > > > > > > > > you > > > > > > > > > > > would like to take on specific pieces of this work. If > > you > > > > > would > > > > > > > like > > > > > > > > > to > > > > > > > > > > > add other options or ideas, etc. > > > > > > > > > > > > > > > > > > > > > > I think that this will have a huge impact as usability > > > > > > improvements > > > > > > > > for > > > > > > > > > > > management of the gateway! > > > > > > > > > > > > > > > > > > > > > > thanks, > > > > > > > > > > > > > > > > > > > > > > --larry > > > > > > > > > > > > > > > > > > > > > > 1. > > > > > > > > > > > https://cwiki.apache.org/ > confluence/display/KNOX/KIP-8+ > > > > > > > > > > > Service+Discovery+and+Topology+Generation > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
