On Wed, Sep 28, 2016 at 11:54 AM, Mike <mikesam...@gmail.com> wrote: > Hi Tim, > > Thank you for the answer. The goal is to share the control plane among ,say, > 100 smaller clusters (only worker nodes in each cluster) which will save you > something like 100*3=300 control plane nodes so it seems to be significant > unless I am missing something when you refer to it as not gaining much. > Would you please elaborate on this? Not that I disagree with you but I just > want to understand your point.
Numerically it saves YOU but the amount of engineering to make it work is still disproportionate to the savings. This is something that very few people have asked for over the course of 2 years. If we had hundreds of people pounding the table and demanding it, it might make more sense, but we just don't (yet?). > Stating the worker groups to be in different cloud accounts emphasis that > actually each worker group is, in fact, a different pool of VMs but they all > share a single control plane for cost saving and simplicity. Again this is > how amazon ECS seems to operate. Sure, but ECS is an software project of enormous scale, with thousands of customers being serviced. It is internally sharded and federated, so that when portions of it go down, it doesn't ALL go down. Google Cloud is exactly the same. In truth, these mega-services follow something much closer to a federated model. > Don't you need a control plane per worker group/pool to be able to use > federation? Yes - this avoids coordinated failures, among other reasons. I'm not saying it wouldn't be interesting to have this - I am saying we DON'T have it, and it would be really hard to build, and that there are 80% solutions along the way that seem like likely resting places :) > Thanks, > Mike > > On Wednesday, September 28, 2016 at 9:50:11 AM UTC-7, Tim Hockin wrote: >> >> This is another variant of multi-tenancy, which is not a first-class >> supported thing in Kubernetes yet. You're actually making it vastly >> more complicated by describing it as multiple cloud accounts. That >> implies different pools of VMs, which implies different clusters. I >> think federation is what you want. We have talked about multi-cluster >> masters before but it is a LOT of work for not that much gain, >> frankly. Once we have some notions of multi-tenancy, maybe this is a >> followup to that. >> >> On Wed, Sep 28, 2016 at 2:03 AM, Mike <mikes...@gmail.com> wrote: >> > Essentially the workers in one group should have no way of knowing >> > anything >> > or affecting anything in other groups. Actually, each group can also be >> > in a >> > separate cloud account and so this isolation already exists at the cloud >> > level. It is just sharing the control plane (which also lives in a >> > different >> > cloud account itself) enables any vulnerability from other groups. >> > >> > We also want each worker group to be able to be broken up into >> > namespaces >> > and essentially be treated as if it was a standalone Kubernetes cluster. >> > Really the goal in here is to have, say, hundreds of smaller >> > Kubernetes-like >> > clusters in say different cloud accounts/vpc's but make them all to >> > share >> > the control plane to save the cost of having to have a control plane per >> > group. I think this is how the amazon ECS runs. >> > >> > On Tuesday, September 27, 2016 at 7:10:37 PM UTC-7, Ian Lewis wrote: >> >> >> >> What kind of isolation are you talking about? Network isolation? Do you >> >> need network encapsulation? Resource (noisy neighbor) isolation? Is >> >> nothing >> >> shared between the groups? e.g. volumes etc.? Do pods within groups >> >> need to >> >> be broken up into namespaces? or can namespaces be the way you define >> >> groups? >> >> >> >> Ian >> >> >> >> On Wed, Sep 28, 2016 at 10:59 AM Mike <mikes...@gmail.com> wrote: >> >>> >> >>> I am new to Kubernetes and I have a question regarding the possibility >> >>> of >> >>> sharing the control plane of a single Kubernetes cluster among a whole >> >>> bunch >> >>> of workers that are >> >>> >> >>> grouped into 100% isolated groups >> >>> each workers group does not need to access any pod in any other group >> >>> and >> >>> actually a zero access is a must >> >>> groups are often in isolated cloud accounts so access to the control >> >>> plane must be through publicly IPs which translates to control plane >> >>> machines in the same region of the same provider in a different cloud >> >>> account. >> >>> >> >>> Please note that this is NOT like the new federation features where we >> >>> have multiple separate Kubernetes clusters. This is more like the >> >>> amazon ECS >> >>> where isolated workers of customer talk to a single control plane >> >>> running on >> >>> aws backend services. >> >>> >> >>> Is this possible today? >> >>> If not, what is missing? >> >>> Can worker isolation be guaranteed with today's design? >> >>> >> >>> -- >> >>> You received this message because you are subscribed to the Google >> >>> Groups >> >>> "Kubernetes user discussion and Q&A" group. >> >>> To unsubscribe from this group and stop receiving emails from it, send >> >>> an >> >>> email to kubernetes-use...@googlegroups.com. >> >>> To post to this group, send email to kubernet...@googlegroups.com. >> >>> Visit this group at https://groups.google.com/group/kubernetes-users. >> >>> For more options, visit https://groups.google.com/d/optout. >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Kubernetes user discussion and Q&A" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to kubernetes-use...@googlegroups.com. >> > To post to this group, send email to kubernet...@googlegroups.com. >> > Visit this group at https://groups.google.com/group/kubernetes-users. >> > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Kubernetes user discussion and Q&A" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to kubernetes-users+unsubscr...@googlegroups.com. > To post to this group, send email to kubernetes-users@googlegroups.com. > Visit this group at https://groups.google.com/group/kubernetes-users. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
