Re: [kubernetes-users] Re: Is Kubernetes better as one cluster per subteam, or should the entire org run on a single cluster?

['Suresh Visvanathan' via Kubernetes user discussion and Q]

We at Oath (Yahoo) manage single large k8s cluster per region/colo, Overall
we have 6 clusters across 6 regions/colos around the world for high
availability and uptime.

The shared cluster is running the critical applications such as Yahoo
Sports, Yahoo Finance and supporting a Media organization with 500+ dev
engineers.  Every team within Media Org, say sports team has its own
namespace defined and resource quota has been assigned to the namespace.
The sports team deploys all their applications to their namespace and all
sports applications share the resources (CPU, memory) assigned to the
namespace, they have HPA defined which can adjust their pods based on the
event and still work within the namespace quota. Similarly, Finance team
within Media Org has its own namespace.  With this shared cluster model, we
provide soft multi-tenancy and importantly nature of all Media applications
are same, serving content and ads.



Monitoring solution(Prometheus and Yamas)  provides detail break up of
resource usage per namespace, we have the resource usage per namespace
handy to bill the team if needed.

The Cluster admin has the ability to adjust the namespace quota( if needed)
based on actual usage and has clear visibility into overall cluster usage



https://www.youtube.com/watch?v=GxH1-sFGMJ8=845s has a good amount of
detail about our k8s deployment.


On Mon, Jan 8, 2018 at 7:50 AM,  wrote:

> On Sunday, January 7, 2018 at 3:29:37 AM UTC+3:30, dax@gmail.com
> wrote:
> > My manager is starting to look into moving us off Azure Web App into
> some kind of container management system, either k8s or service fabric
> (we're *mostly* a MS shop but not entirely).  I was talking with him
> yesterday and he mentioned his plan is that each of the teams (~5-10 devs
> each, generally one main web app and a few background jobs) in our billing
> group (~50 devs total) would run their own cluster.
> >
> > My naive understanding is that somewhat defeats the primary purpose of
> k8s.  I was imagining the the entire billing group would have a single
> cluster, and the various teams would then not have to think about how to
> manage it; things would "just work".  My manager's perspective is that with
> a big shared cluster everyone would be stepping on each others toes and it
> would become *more* difficult to manage rather than *less*.  Plus org
> structure is always fluid and teams get reorganized into other departments
> etc every so often, so that could be messy.  But neither of us really know.
> >
> > Anyone have experience or advice on things like this?
>
> I prefer having a big cluster separated and managed with namespaces, RBAC,
> QoS than having multiple clusters. Managing one cluster is faster than
> multi, it reduces complexity and duplication.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[kubernetes-users] Ask for user cases: Local Service

[Frank]

How all,
We’ve seen a number of requests for “per-node services” - when I ask for a 
Service IP, I always get the service on my local node. But what about the 
arbitrary topoloy key, e.g. same rack, same failure zone?
There is a proposal https://github.com/kubernetes/community/pull/1551, 
which is available now. The objective of this proposal is to figure out a 
generic way to implement the "local service", say "topology aware routing 
of service". Locality is defined by user, it can be any topology-related 
thing. "Local" means the "same topology level", e.g. same node, same rack, 
same failure zone, same failure region, same cloud provider etc.
*I am writing this letter for asking user cases to refine service API.* 
Please feel free to fill comment in  
https://github.com/kubernetes/community/pull/1551 
directly.
Thanks!
--BR,Du Jun

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[kubernetes-users] Re: Is Kubernetes better as one cluster per subteam, or should the entire org run on a single cluster?

[k1 . hedayati93]

On Sunday, January 7, 2018 at 3:29:37 AM UTC+3:30, dax@gmail.com wrote:
> My manager is starting to look into moving us off Azure Web App into some 
> kind of container management system, either k8s or service fabric (we're 
> *mostly* a MS shop but not entirely).  I was talking with him yesterday and 
> he mentioned his plan is that each of the teams (~5-10 devs each, generally 
> one main web app and a few background jobs) in our billing group (~50 devs 
> total) would run their own cluster.
> 
> My naive understanding is that somewhat defeats the primary purpose of k8s.  
> I was imagining the the entire billing group would have a single cluster, and 
> the various teams would then not have to think about how to manage it; things 
> would "just work".  My manager's perspective is that with a big shared 
> cluster everyone would be stepping on each others toes and it would become 
> *more* difficult to manage rather than *less*.  Plus org structure is always 
> fluid and teams get reorganized into other departments etc every so often, so 
> that could be messy.  But neither of us really know.
> 
> Anyone have experience or advice on things like this?

I prefer having a big cluster separated and managed with namespaces, RBAC, QoS 
than having multiple clusters. Managing one cluster is faster than multi, it 
reduces complexity and duplication.

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[kubernetes-users] DNS queries doesn't resolve randomly

[k1 . hedayati93]

Hello,
I've set up a cluster in Azure using Kubespray, VMs are Ubuntu 16.04, and 
Kubespray configs are pretty much default, after installation when I try to 
resolve DNS name I get NXDOMAIN randomly.
I've tried instructions in this link:
https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#debugging-dns-resolution
This is the response:

  k1@azurekube1:~$ kubectl exec -it busybox -- nslookup kubernetes.default
  Server:10.233.0.3
  Address 1: 10.233.0.3

  Name:  kubernetes.default
  Address 1: 10.233.0.1 kubernetes.default.svc.azurekube.pinsvc.net

  k1@azurekube1:~$ kubectl exec -it busybox -- time nslookup kubernetes.default
  Server:10.233.0.3
  Address 1: 10.233.0.3 kube-dns.kube-system.svc.azurekube.pinsvc.net

  nslookup: can't resolve 'kubernetes.default'
  command terminated with exit code 1
  real0m 40.03s
  user0m 0.00s
  sys 0m 0.00s

First query gets resolved but second does not. What's interesting is when I 
don't get response it takes like 20, 30 or 40 seconds to respond. And I've 
tried to increase logging in kube-dns, I don't see any warning or errors in 
kubedns container, but dnsmasq has something like this:

  I0108 11:48:16.711820 dnsmasq[14]: query[A] 
kubernetes.default.svc.azurekube.pinsvc.net from 127.0.0.1
  I0108 11:48:16.711913 dnsmasq[14]: cached 
kubernetes.default.svc.azurekube.pinsvc.net is 10.233.0.1
  I0108 11:48:19.686014 dnsmasq[14]: reply 10.233.0.3 is 
kube-dns.kube-system.svc.azurekube.pinsvc.net
  I0108 11:48:21.712354 dnsmasq[14]: query[A] 
kubernetes.default.svc.azurekube.pinsvc.net from 127.0.0.1
  I0108 11:48:21.712437 dnsmasq[14]: cached 
kubernetes.default.svc.azurekube.pinsvc.net is 10.233.0.1
  I0108 11:48:24.691391 dnsmasq[14]: query[] kubernetes.default from 
10.233.124.134
  I0108 11:48:24.691457 dnsmasq[14]: cached kubernetes.default is NXDOMAIN
  I0108 11:48:25.462845 processUpdate {Version: Data:map[]}
  I0108 11:48:25.462895 Config was unchanged (version )
  I0108 11:48:26.712929 dnsmasq[14]: query[A] 
kubernetes.default.svc.azurekube.pinsvc.net from 127.0.0.1
  I0108 11:48:26.712986 dnsmasq[14]: cached 
kubernetes.default.svc.azurekube.pinsvc.net is 10.233.0.1
  I0108 11:48:29.691979 dnsmasq[14]: query[] kubernetes.default from 
10.233.124.134
  I0108 11:48:29.692043 dnsmasq[14]: cached kubernetes.default is NXDOMAIN
  I0108 11:48:29.692204 dnsmasq[14]: query[] 
kubernetes.default.default.svc.azurekube.pinsvc.net from 10.233.124.134
  I0108 11:48:29.692532 dnsmasq[14]: forwarded 
kubernetes.default.default.svc.azurekube.pinsvc.net to 127.0.0.1
  I0108 11:48:29.692832 dnsmasq[14]: reply 
kubernetes.default.default.svc.azurekube.pinsvc.net is NXDOMAIN
  I0108 11:48:29.692999 dnsmasq[14]: query[] 
kubernetes.default.svc.azurekube.pinsvc.net from 10.233.124.134
  I0108 11:48:29.693076 dnsmasq[14]: cached 
kubernetes.default.svc.azurekube.pinsvc.net is NODATA-IPv6
  I0108 11:48:31.713324 dnsmasq[14]: query[A] 
kubernetes.default.svc.azurekube.pinsvc.net from 127.0.0.1
  I0108 11:48:31.713365 dnsmasq[14]: cached 
kubernetes.default.svc.azurekube.pinsvc.net is 10.233.0.1

I've tried setting up cluster multiple times with Kubernetes 1.9 and 1.8.

I'm out leads to debug this situation. Why this happens randomly? 

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[kubernetes-users] Re: Kubernetes, GCP, and IP Aliases

[divij . sehgaal7]

On Saturday, September 16, 2017 at 4:43:53 AM UTC+5:30, Mark Petrovic wrote:
> Hello.
> 
> 
> I would have made this shorter if I could.  Sorry.  My context is
> Kubernetes, but my immediate questions are around clusters I configure on
> Google Compute Engine (GCE).  Someone out there is bound to be in my 
> situation, so I feel
> comfortable coming here, having been here a few times over the years.
> 
> 
> I am in pre-production research mode for running Kubernetes clusters
> on regular GCE  VMs.  I know about Google Container Engine, but I'm not ready 
> to take that step.
> 
> 
> My work history: I have a couple years of working with Kubernetes
> in various ways, but I'm still far from an expert.
> 
> 
> The very recent past:
> 
> 
> I've successfully setup a k8s cluster on GCE where the control plane
> VMs (master, scheduler, controller, kubelet, kube-proxy) resided
> on a GCE-Custom VPC network 10.0.0.0/24  (I'm avoiding the regional
> default networks because I'm in learning mode and I want and learn
> from that control).  In this k8s cluster, I created a second VPC
> "podVPC" 172.16.0.0/16 from which pod IPs are allocated.  On each
> node's kubelet, I configure a /24 from the podVPC for pods.  I know
> the controller-manager *can* be involved in pod CIDR management,
> but I have chosen that it not be.  I tell the kubelet what pod cidr,
> via the kubelet param --pod_cidr, it can use, not the controller.
> I followed what I call the "cbr0" model in crafting the cluster
> config, found here:
> https://kubernetes.io/docs/getting-started-guides/scratch/.  That
> guide is dated, but I pieced it together.
> 
> 
> In this model, to make pod IPs routable within the cluster you have
> to create GCE VPC Routes that route pod IPs through their respective
> nodes.  Did that, and it works fine.   You also need GCE firewall rules so 
> the control plane members on net-10 can 
> talk to each other; did that, works fine.
> 
> 
> This cluster works as intended.
> 
> 
> Now, the problem with this network approach is that if you want to route
> pod IPs across a VPN to your corp network via, say, BGP + Cloud
> Router, this design won't work because GCE just won't do that routing
> yet.
> 
> 
> So, enter GCE IP Aliases: https://cloud.google.com/compute/docs/alias-ip/
> 
> 
> The present:
> 
> 
> I need those pod IPs routed to my corp network, so I need to evolve my design.
> 
> 
> Keep the cluster configuration the same as the cluster above.
> Meaning, no changes to the kubelet or controller manager.
> 
> 
> However, the GCE VM configs *do* change.  Now you create VMs with
> GCE-secondary subnets, aka IP Aliases.  Out of these per-VM secondary
> ranges, you allocate pod IPs.  This means you do not create a second
> podVPC as above and manually route pod CIDRs to their respective
> nodes.  When you define a secondary subnet on a network, GCE will
> setup those routes for you, and announce those routes over VPN to
> your corp network.
> 
> 
> My first problem:  if I bring up a couple of nodes with IP Alias
> ranges defined on them, without any pods running at all, I can
> already ping addresses where the pods will be allocated.  This makes
> me think two things:  1) I've read the IP Alias docs carefully but
> I've already screwed up my VM config, 2) my node VM config is correct
> and nodes are supposed to masquerade as secondary workloads.  And
> if 2 obtains, when a real pod does come up, how do I tell (via some
> k8s control plane flag??) the GCE fabric to stop masquerading as
> the pod?
> 
> 
> Thanks for reading this far.

Are there any problems present if I try running Kubernetes Cluster present in a 
distributed form across a number of VMs that are part of a VPN?

If yes, what problems might be headed my way and how can I avoid them?

Although this is a Kubernetes group, if someone has experience doing something 
similar, running docker in Swarm mode, running containers on VMs present in a 
VPN, instead of a Kubernetes cluster, kindly share your two cents. I am new to 
the Kubernetes/docker-swarm mode but have worked with Docker in the past.

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] CRD for PODs with identity

[Michele Bertasi]

Makes sense. Thanks Tim.

On Fri, Jan 5, 2018 at 8:40 PM, 'Tim Hockin' via Kubernetes user discussion
and Q  wrote:

> I think this is exactly the sort of thing that a custom deployment-like
> operator is good for.  You have particular needs that are not easily
> satisfied with existing constructs.  CRDs and controllers let you build
> this, and figure out how you want it to work.
>
> Later, maybe, you can solicit other users, and see if it satisfies them
> too.  Or not.
>
> On Jan 5, 2018 11:09 AM, "Michele Bertasi"  brightcomputing.com> wrote:
>
>> Hi everyone,
>>
>> I'm trying to implement an operator to manage a Custom Resource
>> Definition for this proof-of-concept application:
>> * a user creates an instance of my CRD
>> * the operator creates a POD for that CRD
>> * after a specified timeout, both the POD and the CRD disappear
>>
>> the use case is a scratch space for users, where a container with sshd is
>> created, they can connect there and play (every POD gets a different secret
>> mounted with different allowed SSH keys). Then the container is removed.
>> The PODs will be exposed through a NodePort service or a L4 ingress (but
>> that's not the point of my question).
>>
>> Of course I can create all the PODs myself through the operator, then
>> manage eventual POD deletions, scaling up and down, etc.
>> What I'm trying though is to not reinvent the wheel and try to reuse as
>> much as possible existing constructs, so I was looking at
>> ReplicationControllers for example. I could let a RC manage the number of
>> replicas, and when I need a new POD, I just scale the number up. When I
>> have to downscale it gets more tricky, because I have to delete a specific
>> one (and not a random one). I also looked at StatefulSets but in that case
>> downscaling deletes the POD with the highest ID. So I'm a bit stuck.
>>
>> What do you guys suggest? Is there anything I can reuse or I really need
>> to manage the resources myself? Maybe there's also a similar CRD that I
>> could reuse; the idea that a POD has a definite lifetime doesn't seem so
>> crazy to me.
>>
>> Thanks
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes user discussion and Q" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Kubernetes user discussion and Q" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/kubernetes-users/A9EfSw8-RlI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Re: [kubernetes-users] Re: Need help in setting up kubernetes cluster manually without tools like kubeadm, kops, minikube etc

['Robert Bailey' via Kubernetes user discussion and Q]

I'm not sure whether anyone has tried running kubelet or kube-proxy on Mac
OS X (outside of a virtual machine or docker container).

On Mon, Jan 8, 2018 at 1:24 AM, Sree Harissh V  wrote:

> Hi Robert,
>
> Thanks for the information regarding heterogeneous architectures. After a
> long struggle we were able to set up a heterogeneous cluster with ubuntu
> and coreos machines too. Is it possible to run kubelet and kube-proxy in
> Mac OS X machines too ? Are we supposed to download the kubernetes source
> and compile them to achieve this ?
>
> Thank you.
>
> On Sat, Jan 6, 2018 at 3:54 AM, 'Robert Bailey' via Kubernetes user
> discussion and Q  wrote:
>
>>
>>
>> On Mon, Dec 25, 2017 at 8:17 PM, Sree Harissh V 
>> wrote:
>>
>>> Thank you Robert Bailey! I will look in to kubeadm phases. Are there any
>>> tools like kops, kubeadm, etc to set up a heterogeneous cluster(i.e
>>> kubelets running on Ubuntu, CoreOS, CentOS etc)?
>>>
>>
>> You can definitely use kubeadm to set up a cluster with heterogenous
>> nodes operating systems (I actually tried that the other day with Ubuntu
>> and CentOS).
>>
>> One thing we don't have great support for yet is heterogenous
>> architectures (amd64 + arm). You can do it manually today if you apply
>> proper labels to your nodes and system pods and manually run a second set
>> of system pods on the secondary architecture. With manifest lists coming
>> soon to Kubernetes the experience should get a lot simpler in the future
>> (see https://github.com/kubernetes/kubernetes/issues/38067 and
>> https://github.com/kubernetes/kubernetes/issues/57869).
>>
>>
>>
>>>
>>> On Sat, Dec 23, 2017 at 12:11 PM, 'Robert Bailey' via Kubernetes user
>>> discussion and Q  wrote:
>>>
 This was an area that the cluster lifecycle SIG recently decided that
 it should own, but unfortunately we haven't yet produced the documentation
 that you are looking for. The best that we can offer is to look at the
 phases in kubeadm or the kubeadm source code to see how it performs the
 clustering steps.

 On Fri, Dec 22, 2017 at 5:57 AM,  wrote:

> Le vendredi 22 décembre 2017 11:40:29 UTC+1, vhari...@gmail.com a
> écrit :
> > Is there any other documentation/link that guides to set up a
> kubernetes cluster manually apart from "custom cluster from scratch" link
> in official kubernetes documentation.
>
> Hello,
>
> have you tried https://github.com/kelseyhight
> ower/kubernetes-the-hard-way ?
>
> cdt,
>
> rdesousa
>
> --
> You received this message because you are subscribed to the Google
> Groups "Kubernetes user discussion and Q" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>

 --
 You received this message because you are subscribed to a topic in the
 Google Groups "Kubernetes user discussion and Q" group.
 To unsubscribe from this topic, visit https://groups.google.com/d/to
 pic/kubernetes-users/w_ODIyMThoY/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 kubernetes-users+unsubscr...@googlegroups.com.
 To post to this group, send email to kubernetes-users@googlegroups.com.
 Visit this group at https://groups.google.com/group/kubernetes-users.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>>
>>> --
>>> Regards,
>>> Sree Harissh Venu
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Kubernetes user discussion and Q" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to kubernetes-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to kubernetes-users@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Kubernetes user discussion and Q" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/kubernetes-users/w_ODIyMThoY/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Regards,
> Sree Harissh 

Re: [kubernetes-users] Re: Need help in setting up kubernetes cluster manually without tools like kubeadm, kops, minikube etc

[Sree Harissh V]

Hi Robert,

Thanks for the information regarding heterogeneous architectures. After a
long struggle we were able to set up a heterogeneous cluster with ubuntu
and coreos machines too. Is it possible to run kubelet and kube-proxy in
Mac OS X machines too ? Are we supposed to download the kubernetes source
and compile them to achieve this ?

Thank you.

On Sat, Jan 6, 2018 at 3:54 AM, 'Robert Bailey' via Kubernetes user
discussion and Q  wrote:

>
>
> On Mon, Dec 25, 2017 at 8:17 PM, Sree Harissh V 
> wrote:
>
>> Thank you Robert Bailey! I will look in to kubeadm phases. Are there any
>> tools like kops, kubeadm, etc to set up a heterogeneous cluster(i.e
>> kubelets running on Ubuntu, CoreOS, CentOS etc)?
>>
>
> You can definitely use kubeadm to set up a cluster with heterogenous nodes
> operating systems (I actually tried that the other day with Ubuntu and
> CentOS).
>
> One thing we don't have great support for yet is heterogenous
> architectures (amd64 + arm). You can do it manually today if you apply
> proper labels to your nodes and system pods and manually run a second set
> of system pods on the secondary architecture. With manifest lists coming
> soon to Kubernetes the experience should get a lot simpler in the future
> (see https://github.com/kubernetes/kubernetes/issues/38067 and
> https://github.com/kubernetes/kubernetes/issues/57869).
>
>
>
>>
>> On Sat, Dec 23, 2017 at 12:11 PM, 'Robert Bailey' via Kubernetes user
>> discussion and Q  wrote:
>>
>>> This was an area that the cluster lifecycle SIG recently decided that it
>>> should own, but unfortunately we haven't yet produced the documentation
>>> that you are looking for. The best that we can offer is to look at the
>>> phases in kubeadm or the kubeadm source code to see how it performs the
>>> clustering steps.
>>>
>>> On Fri, Dec 22, 2017 at 5:57 AM,  wrote:
>>>
 Le vendredi 22 décembre 2017 11:40:29 UTC+1, vhari...@gmail.com a
 écrit :
 > Is there any other documentation/link that guides to set up a
 kubernetes cluster manually apart from "custom cluster from scratch" link
 in official kubernetes documentation.

 Hello,

 have you tried https://github.com/kelseyhight
 ower/kubernetes-the-hard-way ?

 cdt,

 rdesousa

 --
 You received this message because you are subscribed to the Google
 Groups "Kubernetes user discussion and Q" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to kubernetes-users+unsubscr...@googlegroups.com.
 To post to this group, send email to kubernetes-users@googlegroups.com.
 Visit this group at https://groups.google.com/group/kubernetes-users.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Kubernetes user discussion and Q" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>>> pic/kubernetes-users/w_ODIyMThoY/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> kubernetes-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to kubernetes-users@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Regards,
>> Sree Harissh Venu
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes user discussion and Q" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to kubernetes-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kubernetes-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Kubernetes user discussion and Q" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/kubernetes-users/w_ODIyMThoY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Regards,
Sree Harissh Venu

-- 
You received this message because you are subscribed to the Google Groups 
"Kubernetes user discussion and Q" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at