Keith, I was with you all the way till your last sentence ... > -----Original Message----- > From: Keith Bray [mailto:keith.b...@rackspace.com] > Sent: Friday, April 22, 2016 11:53 AM > To: OpenStack Development Mailing List (not for usage questions) > <openstack-dev@lists.openstack.org> > Subject: Re: [openstack-dev] [magnum][app-catalog][all] Build unified > abstraction for all COEs > > Thanks Amirth… I’m glad to see it hasn’t changed much since I was involved > with Trove in its early days. What you are describing makes sense, and I > view it as an LCD for managing common things across the database types, > not an LCD for the database interaction performed by the user/client > interacting with the application database. This parallels where I think > Magnum should sit, which is general management of the COEs (reinitialize > bay, backup bay, maybe even common configuration of bays, etc. etc.), and > not an LCD for user/application interaction with the COEs. It’s a grey > area for sure, as should “list containers” on a bay be a common > abstraction? I think it’s too early to tell… and, to be clear for all > folks, I’m not opposed to the LCD existing. I just don’t want it to be > required for the operator to run it at this time as part of Magnum given > how quickly the COE technology landscape is evolving. So, optional > support, or separate API/Project make the most sense to me, and can always > be merged in as part of the Magnum project at a future date once the > technology landscape settles. RDBMS has been fairly standard for awhile. >
[amrith] Yes, but these pesky things called NoSQL have not, and Trove tries to handle both. Fun fact, there are more NoSQL databases than there are Ben & Jerry's flavors. There is some of the same kind of challenges that you describe and while we currently have an API that tries to give you the LCD, I can certainly see the value in the approach you propose for Magnum. If there are going to be conversations about this in the magnum sessions, I would certainly like to attend as I think they will help to instruct us on how you are handling a similar class of challenges to ones that we face in Trove. > Thanks for all the input. The context helps. > > -Keith > > > > On 4/22/16, 6:40 AM, "Amrith Kumar" <amr...@tesora.com> wrote: > > >For those interested in one aspect of this discussion (a common compute > >API for bare-metal, VM's and containers), there's a review of a spec in > >Trove [1], and a session at the summit [2]. > > > >Please join [2] if you are able > > > > Trove Container Support > > Thursday, April 28, 9:50am-10:30am > > Hilton Austin - MR 406 > > > >Keith, more detailed answer to one of your questions is below. > > > >Thanks, > > > >-amrith > > > > > >[1] https://review.openstack.org/#/c/307883/4 > >[2] > >https://www.openstack.org/summit/austin-2016/summit-schedule/events/915 > >0 > > > >> -----Original Message----- > >> From: Keith Bray [mailto:keith.b...@rackspace.com] > >> Sent: Thursday, April 21, 2016 5:11 PM > >> To: OpenStack Development Mailing List (not for usage questions) > >> <openstack-dev@lists.openstack.org> > >> Subject: Re: [openstack-dev] [magnum][app-catalog][all] Build unified > >> abstraction for all COEs > >> > >> 100% agreed on all your points… with the addition that the level of > >>functionality you are asking for doesn’t need to be baked into an API > >>service such as Magnum. I.e., Magnum doesn’t have to be the thing > >>providing the easy-button app deployment — Magnum isn’t and shouldn’t > >>be a Docker Hub alternative, a Tutum alternative, etc. A Horizon UI, > >>App Catalog UI, or OpenStack CLI on top of Heat, Murano, Solum, > >>Magnum, etc. > >> etc. can all provide this by pulling together the underlying API > >> services/technologies to give users the easy app deployment buttons. > I > >> don’t think Magnum should do everything (or next thing we know we’ll > >>be trying to make Magnum a PaaS, or make it a CircleCI, or … Ok, I’ve > >>gotten carried away). Hopefully my position is understood, and no > >>problem if folks disagree with me. I’d just rather compartmentalize > >>domain concerns and scope Magnum to something focused, achievable, > >>agnostic, and easy for operators to adopt first. User traction will > >>not be helped by increasing service/operator complexity. I’ll have > >>to go look at the latest Trove and Sahara APIs to see how LCD is > >>incorporated, and would love feedback from > > > >[amrith] Trove provides a common, database agnostic set of API's for a > >number of common database workflows including provisioning and > >lifecycle management. It also provides abstractions for common database > >topologies like replication and clustering, and management actions that > >will manipulate those topologies (grow, shrink, failover, ...). It > >provides abstractions for some common database administration > >activities like user management, database management, and ACL's. It > >allows you to take backups of databases and to launch new instances > >from backups. It provides a simple way in which a user can manage the > >configuration of databases (a subset of the configuration parameters > >that the database supports, the choice the subset being up to the > >operator) in a consistent way. Further it allows users to make > >configuration changes across a group of databases through the process > >of associating a 'configuration group' to database instances. > > > >The important thing about this is that there is a desire to provide all > >of the above capabilities through the Trove API and make these > >capabilities database agnostic. The actual database specific > >implementations are within Trove and largely contained in a database > >specific guest agent that performs the database specific actions to > >achieve the end result that the user requested via the Trove API. > > > >The user interacts directly with the database as well; the application > >speaks native database API's to the database and unlike (for example, > >DynamoDB) Trove does not get into the data path between the application > >and the database itself. Users and administrators are able to interact > >with the database through its native management interfaces as well > >(some restrictions may apply, depending on the level of access that the > >operator allows). > > > >In short, the value provided is that databases are long lived things > >and provisioning and initial configuration are very important, but > >ongoing maintenance and management are as well. The mantra for dba's is > >always to automate and standardize all the repeated workflows. Trove > >does that for you through a single set of API's because todays > >datacenters have a wide diversity of databases. Hope that helps. > > > >> Trove and Sahara operators on the value vs. customer confusion or > >>operator overhead they get from those LCDs if they are required parts > >>of the services. > >> > >> Thanks, > >> -Keith > >> > >> On 4/21/16, 3:31 PM, "Fox, Kevin M" <kevin....@pnnl.gov> wrote: > >> > >> >There are a few reasons, but the primary one that affects me is Its > >> >from the app-catalog use case. > >> > > >> >To gain user support for a product like OpenStack, you need users. > >> >The easier you make it to use, the more users you can potentially get. > >> >Traditional Operating Systems learned this a while back. Rather then > >> >make each OS user have to be a developer and custom deploy every app > >> >they want to run, they split the effort in such a way that > >> >Developers can provide software through channels that Users that are > >> >not skilled Developers can consume and deploy. The "App" culture in > >> >the mobile space it the epitome of that at the moment. My > >> >grandmother fires up the app store on her phone, clicks install on > >> >something interesting, and > >> starts using it. > >> > > >> >Right now, Thats incredibly difficult in OpenStack. You have to find > >> >the software your interested in, figure out which components your > >> >going to consume (nova, magnum, which COE, etc) then use those api's > >> >to launch some resource. Then after that resource is up, then you > >> >have to switch tools and then use those tools to further launch > >> >things, ansible or kubectl or whatever, then further deploy things. > >> > > >> >What I'm looking for, is a unified enough api, that a user can go > >> >into horizon, go to the app catalog, find an interesting app, click > >> >install/run, and then get a link to a service they can click on and > >> >start consuming the app they want in the first place. The number of > >> >users that could use such an interface, and consume OpenStack > >> >resources are several orders of magnitude greater then the numbers > >> >that can manually deploy something ala the procedure in the previous > paragraph. > >> >More of that is good for Users, Developers, and Operators. > >> > > >> >Does that help? > >> > > >> >Thanks, > >> >Kevin > >> > > >> > > >> >________________________________________ > >> >From: Keith Bray [keith.b...@rackspace.com] > >> >Sent: Thursday, April 21, 2016 1:10 PM > >> >To: OpenStack Development Mailing List (not for usage questions) > >> >Subject: Re: [openstack-dev] [magnum][app-catalog][all] Build > >> >unified abstraction for all COEs > >> > > >> >If you don¹t want a user to have to choose a COE, can¹t we just > >> >offer an option for the operator to mark a particular COE as the > >> >³Default COE² that could be defaulted to if one isn¹t specified in > >> >the Bay create call? If the operator didn¹t specify a default one, > >> >then the CLI/UI must submit one in the bay create call otherwise it > would fail. > >> > > >> >Kevin, can you clarify Why you have to write scripts to deploy a > >> container > >> >to the COE? It can be made easy for the user to extract all the > >> >runtime/env vars needed for a user to just do ³docker run Š² and > >> >poof, container running on Swarm on a Magnum bay. Can you help me > understand > >> >the script part of it? I don¹t believe container users want an > >> >abstraction between them and their COE CLIŠ but, what I believe > >> >isn¹t important. What I do think is important is that we not > >> >require OpenStack operators to run that abstraction layer to be > >> >running a ³magnum compliant² service. It should either be an > >> >³optional² API add-on or a separate API or separate project. If > >> >some folks want an abstraction layer, then great, feel free to build > >> >it and even propose > >>it > >> under the OpenStack ecosystem.. > >> >But, that abstraction would be a ³proxy API" over the COEs, and > >> >doesn¹t need to be part of Magnum¹s offering, as it would be > >> >targeted at the COE interactions and not the bay interactions (which > >> >is where Magnum scope is best focused). I don¹t think Magnum should > >> >play in both these distinct domains (Bay interaction vs. COE > >> >interaction). The former (bay > >> >interaction) is an infrastructure cloud thing (fits well with > >> >OpenStack), the latter (COE interaction) is an obfuscation of > >> >emerging technologies, which gets in to the Trap that Adrian > >> >mentioned. The abstraction layer API will forever and always be > >> >drastically behind in trying to keep up with the COE innovation. > >> > > >> >In summary, an abstraction over the COEs would be best served as a > >> >different effort. Magnum would be best focused on bay interactions > >> >and should not try to pick a COE winner or require an operator to > >> >run a lowest-common-demonitor API abstraction. > >> > > >> >Thanks for listening to my soap-box. > >> >-Keith > >> > > >> > > >> > > >> >On 4/21/16, 2:36 PM, "Fox, Kevin M" <kevin....@pnnl.gov> wrote: > >> > > >> >>I agree with that, and thats why providing some bare minimum > >> >>abstraction will help the users not have to choose a COE themselves. > >> >>If we can't decide, why can they? If all they want to do is launch > >> >>a container, they should be able to script up "magnum > >> >>launch-container foo/bar:latest" and get one. That script can then be > relied upon. > >> >> > >> >>Today, they have to write scripts to deploy to the specific COE > >> >>they have chosen. If they chose Docker, and something better comes > >> >>out, they have to go rewrite a bunch of stuff to target the new, > >> >>better thing. This puts a lot of work on others. > >> >> > >> >>Do I think we can provide an abstraction that prevents them from > >> >>ever having to rewrite scripts? No. There are a lot of features in > >> >>the COE world in flight right now and we dont want to solidify an > >> >>api around them yet. We shouldn't even try that. But can we cover a > >> >>few common things now? Yeah. > >> >> > >> >>Thanks, > >> >>Kevin > >> >>________________________________________ > >> >>From: Adrian Otto [adrian.o...@rackspace.com] > >> >>Sent: Thursday, April 21, 2016 7:32 AM > >> >>To: OpenStack Development Mailing List (not for usage questions) > >> >>Subject: Re: [openstack-dev] [magnum][app-catalog][all] Build > >> >>unified abstraction for all COEs > >> >> > >> >>> On Apr 20, 2016, at 2:49 PM, Joshua Harlow > >> >>><harlo...@fastmail.com> > >> >>>wrote: > >> >>> > >> >>> Thierry Carrez wrote: > >> >>>> Adrian Otto wrote: > >> >>>>> This pursuit is a trap. Magnum should focus on making native > >> >>>>>container APIs available. We should not wrap APIs with leaky > >> >>>>>abstractions. The lowest common denominator of all COEs is an > >> >>>>>remarkably low value API that adds considerable complexity to > >> >>>>>Magnum that will not strategically advance OpenStack. If we > >> >>>>>instead focus our effort on making the COEs work better on > >> >>>>>OpenStack, that would be a winning strategy. Support and > >> >>>>>compliment our various COE ecosystems. > >> >>> > >> >>> So I'm all for avoiding 'wrap APIs with leaky abstractions' and > >> >>>'making COEs work better on OpenStack' but I do dislike the part > >> >>>about COEs > >> >>>(plural) because it is once again the old non-opinionated problem > >> >>>that we (as a community) suffer from. > >> >>> > >> >>> Just my 2 cents, but I'd almost rather we pick one COE and > >> >>>integrate that deeply/tightly with openstack, and yes if this > >> >>>causes some part of the openstack community to be annoyed, meh, to > >> >>>bad. Sadly I have a feeling we are hurting ourselves by continuing > >> >>>to try to be everything and not picking anything (it's a general > >> >>>thing we, as a group, seem to be good at, lol). I mean I get the > >> >>>reason to just support all the things, but it feels like we as a > >> >>>community could just pick something, work together on figuring out > >> >>>how to pick one, using all these bright leaders we have to help > >> >>>make that possible (and yes this might piss some people off, to > >> >>>bad). Then work toward making that something great and move onŠ > >> >> > >> >>The key issue preventing the selection of only one COE is that this > >> >>area is moving very quickly. If we would have decided what to pick > >> >>at the time the Magnum idea was created, we would have selected > Docker. > >> >>If you look at it today, you might pick something else. A few > >> >>months down the road, there may be yet another choice that is more > >> >>compelling. The fact that a cloud operator can integrate services > >> >>with OpenStack, and have the freedom to offer support for a > >> >>selection of COE¹s is a form of insurance against the risk of > >> >>picking the wrong one. Our compute service offers a choice of > >> >>hypervisors, our block storage service offers a choice of storage > >> >>hardware drivers, our networking service allows a choice of network > >> >>drivers. Magnum is following the same pattern of choice that has > >> >>made OpenStack compelling for a very diverse community. That design > >> >>consideration was > >> intentional. > >> >> > >> >>Over time, we can focus the majority of our effort on deep > >> >>integration with COEs that users select the most. I¹m convinced > >> >>it¹s still too early to bet the farm on just one choice. > >> >> > >> >>Adrian > >> >> > >> >>>> I'm with Adrian on that one. I've attended a lot of > >> >>>>container-oriented conferences over the past year and my main > >> >>>>takeaway is that this new crowd of potential users is not > >> >>>>interested (at all) in an OpenStack-specific lowest common > >> >>>>denominator API for COEs. They want to take advantage of the > >> >>>>cool features in Kubernetes API or the versatility of Mesos. > >> >>>>They want to avoid caring about the infrastructure provider bit > >> >>>>(and not deploy Mesos or Kubernetes themselves). > >> >>>> > >> >>>> Let's focus on the infrastructure provider bit -- that is what > >> >>>>we do and what the ecosystem wants us to provide. > >> >>>> > >> >>> > >> >>> > >> >>>__________________________________________________________________ > >> >>>___ > >> >>>___ > >> >>>_ > >> >>>_ > >> >>> OpenStack Development Mailing List (not for usage questions) > >> >>> Unsubscribe: > >> >>>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> >> > >> >>___________________________________________________________________ > >> >>___ > >> >>___ > >> >>_ > >> >>OpenStack Development Mailing List (not for usage questions) > >> >>Unsubscribe: > >> >>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> >> > >> >>___________________________________________________________________ > >> >>___ > >> >>___ > >> >>_ > >> >>OpenStack Development Mailing List (not for usage questions) > >> >>Unsubscribe: > >> >>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > >> > > >> >____________________________________________________________________ > >> >___ ___ OpenStack Development Mailing List (not for usage questions) > >> >Unsubscribe: > >> >openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > >> >____________________________________________________________________ > >> >___ ___ OpenStack Development Mailing List (not for usage questions) > >> >Unsubscribe: > >> >openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > >> > >>______________________________________________________________________ > >>___ > >>_ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: > >>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >_______________________________________________________________________ > >___ OpenStack Development Mailing List (not for usage questions) > >Unsubscribe: > >openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev