I'll answer inline, so that it's easier to understand what part of your
message I'm responding to.
On 07/03/2018 02:37 PM, Fox, Kevin M wrote:
Yes/no on the vendor distro thing. They do provide a lot of options, but they
also provide a fully k8s tested/provided route too. kubeadm. I can take linux
distro of choice, curl down kubeadm and get a working kubernetes in literally a
couple minutes.
How is this different from devstack?
With both approaches:
* Download and run a single script
* Any sort of networking outside of super basic setup requires manual
intervention
* Not recommended for "production"
* Require workarounds when running as not-root
Is it that you prefer the single Go binary approach of kubeadm which
hides much of the details that devstack was designed to output (to help
teach people what's going on under the hood)?
>
No compiling anything or building containers. That is what I mean when
I say they have a product.
What does devstack compile?
By "compile" are you referring to downloading code from git
repositories? Or are you referring to the fact that with kubeadm you are
downloading a Go binary that hides the downloading and installation of
all the other Kubernetes images for you [1]?
[1]
https://github.com/kubernetes/kubernetes/blob/8d73473ce8118422c9e0c2ba8ea669ebbf8cee1c/cmd/kubeadm/app/cmd/init.go#L267
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/images/images.go#L63
> Other vendors provide their own builds, release tooling, or config
management integration. which is why that list is so big. But it is up
to the Operators to decide the route and due to k8s having a very clean,
easy, low bar for entry it sets the bar for the other products to be
even better.
I fail to see how devstack and kubeadm aren't very much in the same vein?
The reason people started adopting clouds was because it was very quick to
request resources. One of clouds features (some say drawbacks) vs VM farms has
been ephemeralness. You build applications on top of VMs to provide a Service
to your Users. Great. Things like Containers though launch much faster and have
generally more functionality for plumbing them together then VMs do though.
Not sure what this has to do with what we've been discussing.
>
So these days containers are out clouding vms at this use case. So,
does Nova continue to be cloudy vm or does it go for the more production
vm use case like oVirt and VMware?
"production VM" use case like oVirt or VMWare? I don't know what that
means. You mean "a GUI-based VM management system"?
Without strong orchestration of some kind on top the cloudy use case is also
really hard with Nova. So we keep getting into this tug of war between people
wanting VM's as a building blocks of cloud scale applications, and those that
want Nova to be an oVirt/VMware replacement. Honestly, its not doing either use
case great because it cant decide what to focus on.
No, that's not at all what I've been saying. I continue to see Nova (and
other services in its layer of OpenStack) as a building block *for
higher-level systems like Kubernetes or Heat*. There is a reason that
Kubernetes has an OpenStack cloud provider plugin, and that plugin calls
imperative Nova, Neutron, Cinder, and Keystone API calls.
oVirt is a better VMware alternative today then Nova is, having used it. It
focuses specifically on the same use cases. Nova is better at being a cloud
then oVirt and VMware. but lags behind Azure/AWS a lot when it comes to having
apps self host on it. (progress is being made again finally. but its slow)
I'm not particularly interested in having Nova be a free VMWare
replacement -- or in trying to be whatever oVirt has become.
Some might see usefulness in these things, and as long as the feature
requests to Nova don't cause Nova to become something other than
low-level compute plumbing, I'm fine with that.
While some people only ever consider running Kubernetes on top of a cloud, some
of us realize maintaining both a cloud an a kubernetes is unnecessary and can
greatly simplify things simply by running k8s on bare metal. This does then
make it a competitor to Nova as a platform for running workload on.
What percentage of Kubernetes users deploy on baremetal (and continue to
deploy on baremetal in production as opposed to just toying around with it)?
> As k8s gains more multitenancy features, this trend will continue to
grow I think. OpenStack needs to be ready for when that becomes a thing.
OpenStack is already multi-tenant, being designed as such from day one.
With the exception of Ironic, which uses Nova to enable multi-tenancy.
What specifically are you referring to "OpenStack needs to be ready"?
Also, what specific parts of OpenStack are you referring to there?
Heat is a good start for an orchestration system, but it is hamstrung by it
being an optional component, by there still not being a way to download secrets
to a vm securely from the secret store, by the secret store also being
completely optional, etc. An app developer can't rely on any of it. :/ Heat is
hamstrung by the lack of blessing so many other OpenStack services are. You
can't fix it until you fix that fundamental brokenness in OpenStack.
I guess I just fundamentally disagree that having a monolithic
all-things-for-all-users application architecture and feature set is
something that OpenStack should be.
There is a *reason* that Kubernetes jettisoned all the cloud provider
code from its core. The reason is because setting up that base stuff is
*hard* and that work isn't germane to what Kubernetes is (a container
orchestration system, not a datacenter resource management system).
Heat is also hamstrung being an orchestrator of existing API's by there being
holes in the API's.
I agree there are some holes in some of the APIs. Happy to work on
plugging those holes as long as the holes are properly identified as
belonging to the correct API and are not simply a feature request what
would expand the scope of lower-level plumbing services like Nova.
Think of OpenStack like a game console. The moment you make a component
optional and make it takes extra effort to obtain, few software developers
target it and rarely does anyone one buy the addons it because there isn't
software for it. Right now, just about everything in OpenStack is an addon.
Thats a problem.
I don't have any game consoles nor do I develop software for them, so I
don't really see the correlation here. That said, I'm 100% against a
monolithic application approach, as I've mentioned before.
Best,
-jay
Thanks,
Kevin
________________________________________
From: Jay Pipes [[email protected]]
Sent: Monday, July 02, 2018 4:13 PM
To: [email protected]
Subject: Re: [openstack-dev] [tc] [all] TC Report 18-26
On 06/27/2018 07:23 PM, Zane Bitter wrote:
On 27/06/18 07:55, Jay Pipes wrote:
Above, I was saying that the scope of the *OpenStack* community is
already too broad (IMHO). An example of projects that have made the
*OpenStack* community too broad are purpose-built telco applications
like Tacker [1] and Service Function Chaining. [2]
I've also argued in the past that all distro- or vendor-specific
deployment tools (Fuel, Triple-O, etc [3]) should live outside of
OpenStack because these projects are more products and the relentless
drive of vendor product management (rightfully) pushes the scope of
these applications to gobble up more and more feature space that may
or may not have anything to do with the core OpenStack mission (and
have more to do with those companies' product roadmap).
I'm still sad that we've never managed to come up with a single way to
install OpenStack. The amount of duplicated effort expended on that
problem is mind-boggling. At least we tried though. Excluding those
projects from the community would have just meant giving up from the
beginning.
You have to have motivation from vendors in order to achieve said single
way of installing OpenStack. I gave up a long time ago on distros and
vendors to get behind such an effort.
Where vendors see $$$, they will attempt to carve out value
differentiation. And value differentiation leads to, well, differences,
naturally.
And, despite what some might misguidedly think, Kubernetes has no single
installation method. Their *official* setup/install page is here:
https://kubernetes.io/docs/setup/pick-right-solution/
It lists no fewer than *37* (!) different ways of installing Kubernetes,
and I'm not even including anything listed in the "Custom Solutions"
section.
I think Thierry's new map, that collects installer services in a
separate bucket (that may eventually come with a separate git namespace)
is a helpful way of communicating to users what's happening without
forcing those projects outside of the community.
Sure, I agree the separate bucket is useful, particularly when paired
with information that allows operators to know how stable and/or
bleeding edge the code is expected to be -- you know, those "tags" that
the TC spent time curating.
So to answer your question:
<jaypipes> zaneb: yeah... nobody I know who argues for a small stable
core (in Nova) has ever said there should be fewer higher layer
services.
<jaypipes> zaneb: I'm not entirely sure where you got that idea from.
Note the emphasis on *Nova* above?
Also note that when I've said that *OpenStack* should have a smaller
mission and scope, that doesn't mean that higher-level services aren't
necessary or wanted.
Thank you for saying this, and could I please ask you to repeat this
disclaimer whenever you talk about a smaller scope for OpenStack.
Yes. I shall shout it from the highest mountains. [1]
Because for those of us working on higher-level services it feels like
there has been a non-stop chorus (both inside and outside the project)
of people wanting to redefine OpenStack as something that doesn't
include us.
I've said in the past (on Twitter, can't find the link right now, but
it's out there somewhere) something to the effect of "at some point,
someone just needs to come out and say that OpenStack is, at its core,
Nova, Neutron, Keystone, Glance and Cinder".
Perhaps this is what you were recollecting. I would use a different
phrase nowadays to describe what I was thinking with the above.
I would say instead "Nova, Neutron, Cinder, Keystone and Glance [2] are
a definitive lower level of an OpenStack deployment. They represent a
set of required integrated services that supply the most basic
infrastructure for datacenter resource management when deploying OpenStack."
Note the difference in wording. Instead of saying "OpenStack is X", I'm
saying "These particular services represent a specific layer of an
OpenStack deployment".
Nowadays, I would further add something to the effect of "Depending on
the particular use cases and workloads the OpenStack deployer wishes to
promote, an additional layer of services provides workload orchestration
and workflow management capabilities. This layer of services include
Heat, Mistral, Tacker, Service Function Chaining, Murano, etc".
Does that provide you with some closure on this feeling of "non-stop
chorus" of exclusion that you mentioned above?
The reason I haven't dropped this discussion is because I really want to
know if _all_ of those people were actually talking about something else
(e.g. a smaller scope for Nova), or if it's just you. Because you and I
are in complete agreement that Nova has grown a lot of obscure
capabilities that make it fiendishly difficult to maintain, and that in
many cases might never have been requested if we'd had higher-level
tools that could meet the same use cases by composing simpler operations.
IMHO some of the contributing factors to that were:
* The aforementioned hostility from some quarters to the existence of
higher-level projects in OpenStack.
* The ongoing hostility of operators to deploying any projects outside
of Keystone/Nova/Glance/Neutron/Cinder (*still* seen playing out in the
Barbican vs. Castellan debate, where we can't even correct one of
OpenStack's original sins and bake in a secret store - something k8s
managed from day one - because people don't want to install another ReST
API even over a backend that they'll already have to install anyway).
* The illegibility of public Nova interfaces to potential higher-level
tools.
I would like to point something else out here. Something that may not be
pleasant to confront.
Heat's competition (for resources and mindshare) is Kubernetes, plain
and simple.
Heat's competition is not other OpenStack projects.
Nova's competition is not Kubernetes (despite various people continuing
to say that it is).
Nova is not an orchestration system. Never was and (as long as I'm
kicking and screaming) never will be.
Nova's primary competition is:
* Stand-alone Ironic
* oVirt and stand-alone virsh callers
* Parts of VMWare vCenter [3]
* MaaS in some respects
* The *compute provisioning* parts of EC2, Azure, and GCP
This is why there is a Kubernetes OpenStack cloud provider plugin [4].
This plugin uses Nova [5] (which can potentially use Ironic), Cinder,
Keystone and Neutron to deploy kubelets to act as nodes in a Kubernetes
cluster and load balancer objects to act as the proxies that k8s itself
uses when deploying Pods and Services.
Heat's architecture, template language and object constructs are in
direct competition with Kubernetes' API and architecture, with the
primary difference being a VM-centric [6] vs. a container-centric object
model.
Heat's template language is similar to Helm's chart template YAML
structure [7], and with Heat's evolution to the "convergence model",
Heat's architecture actually got closer to Kubernetes' architecture:
that of continually attempting to converge an observed state with a
desired state.
So, what is Heat to do?
The hype and marketing machine is never-ending, I'm afraid. [8]
I'm not sure there's actually anything that can be done about this.
Perhaps it is a fait accomplis that Kubernetes/Helm will/has become
synonymous with "orchestration of things". Perhaps not. I'm not an
oracle, unfortunately.
Maybe the only thing that Heat can do to fend off the coming doom is to
make a case that Heat's performance, reliability, feature set or
integration with OpenStack's other services make it a better candidate
for orchestrating virtual machine or baremetal workloads on an OpenStack
deployment than Kubernetes is.
Sorry to be the bearer of bad news,
-jay
[1] I live in Florida, though, which has no mountains. But, when I
visit, say, North Carolina, I shall certainly shout it from their mountains.
[2] some would also say Castellan, Ironic and Designate belong here.
[3] Though VMWare is still trying to be everything that certain IT
administrators ever needed, including orchestration, backup services,
block storage pooling, high availability, quota management, etc etc
[4]
https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/#openstack
[5]
https://github.com/kubernetes/kubernetes/blob/92b81114f43f3ca74988194406957a5d1ffd1c5d/pkg/cloudprovider/providers/openstack/openstack.go#L377
[6] The fact that Heat started as a CloudFormation API clone gave it its
VM-centricity.
[7]
https://github.com/kubernetes/helm/blob/master/docs/chart_template_guide/index.md
[8] The Kubernetes' machine has essentially decimated all the other
"orchestration of things" projects' resources and mindshare, including a
number of them that were very well architected, well coded, and well
documented:
* Mesos with Marathon/Aurora
* Rancher
* OpenShift (you know, the original, original one...)
* Nomad
* Docker Swarm/Compose
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
