[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Dusty Mabe]

dustymabe added a new comment to an issue you are following:
``
kubernetes rpms are in the install tree now: 
https://pagure.io/fedora-atomic/pull-request/39

the images produced tonight should have the content in there. Let's close this 
issue if testing is successful there. 
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Jason Brooks]

jasonbrooks added a new comment to an issue you are following:
``
Some updates:

* the issue of fedora having an aged kube is resolved, v1.4.5 is in f25 stable
* the issue of not being able to just `rpm-ostree install kubernetes` and 
proceed as if we never removed kube from the image is open. This won't work 
because [rpm-ostree can't deal](: 
https://github.com/projectatomic/rpm-ostree/issues/462) w/ the kube-apiserver 
binary in kubernetes-master
* the issue of running kube in containers on fedora atomic is in progress:
* I've been working w/ these [kube 
containers](https://github.com/jasonbrooks/k8s-images/tree/f25), and these 
[etcd](https://github.com/projectatomic/atomic-system-containers/tree/master/etcd)
 and 
[flannel](https://github.com/projectatomic/atomic-system-containers/tree/master/flannel)
 containers. They work (but need [this 
PR](https://github.com/projectatomic/atomic-system-containers/pulls)), and I'm 
adapting the [upstream ansible 
scripts](https://github.com/kubernetes/contrib/tree/master/ansible) to use them.
* I need to get those kube, etcd and flannel containers into the new fedora 
build system, and could use help with that.
* I need to get the ansible changes finished and upstreamed, and that ought 
to be straightforward, but I could use feedback on my PR once I get it together.
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Jason Brooks]

jasonbrooks added a new comment to an issue you are following:
``
kube 1.4.5 for f25 in bodhi: 
https://bodhi.fedoraproject.org/updates/FEDORA-2016-fffea4b1c3

I've been working on getting it running in rpm-installed node pkgs / master 
pkgs in containers. Notes on that here: 
https://gist.github.com/jasonbrooks/bdbb0b142fc82bbd7b33759b3d934ba0
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Josh Berkus]

jberkus added a new comment to an issue you are following:
``
Well, the whole point of Atomic is to be immutable: "configure once, deploy 
many times".  Having a key piece of infrastructure depend on per-server package 
layering kinda breaks that.  It makes it hard to explain why people should use 
Atomic in the first place.

Offering multiple trees would definitely help, but do we have the ability to do 
that?  I mean, we have issues with the one tree on a fairly regular basis.

Rebuilding kubeadm as system containers would be *awesome* if we could really 
do that.  Weren't there technical issues with running Kubelet in a container 
though?
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Jason Brooks]

jasonbrooks added a new comment to an issue you are following:
``

> 1) it requires package layering, which really pushes up the idea of having a 
> way for users to build their own OStree servers.

What do you mean by that? Package layering doesn't require someone to build 
their own ostree server, it does require installing the layered pkgs separately 
on each host, though, and rebooting in between, although Colin had thoughts on 
cutting out the reboot step, which would make it nicer.

We could also offer multiple trees, some with fewer things rolled in by 
default, some with more.

And we could run these bits as system containers, like the etcd and flannel 
ones. I don't understand exactly what's involved in making those, in particular 
the bits about creating the config.json.template. It seems that this project 
could help w/ that part: https://github.com/jessfraz/riddler but I haven't been 
able to figure out how to build it to use it.
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Josh Berkus]

jberkus added a new comment to an issue you are following:
``
Well, OpenShift is its own thing rather than an alternative to Kubernetes.  
Some people want Kube, some want OpenShift.

So I've been doing a bunch with Kubeadm on AH.  Jason's version does actually 
work, with some caveats:

1) it requires package layering, which really pushes up the idea of having a 
way for users to build their own OStree servers.

2) it doesn't work with setenforce=1.  Even relabelling a bunch of directories, 
I have to setenforce 0 to get kubernetes to work reliably.

If we can get over those humps, though, I 'd like to push forward with 
something based on Kubeadm.  I think it's the way the kubernetes project is 
headed, and it means that we can track them for anyone who wants to use 
kube-latest on Atomic.  Importantly, the kubernetes community will continue to 
add features to kubeadm (like let's encrypt support), which won't happen with 
solutions we devise.

One of the things I was wondering about is maybe installing the kubeadm 
packages as part of Atomic.  My reasoning is this: kubeadm is an installer 
rather than kubernetes itself.  You can, in fact, use it to install an 
older/stable version of Kubernetes (back to 1.4.0, though), so we could track 
upstream Kubeadm without breaking people's stuff.  Also, if not enabled in 
systemd (which it wouldn't be, by default), it doesn't interfere with 
installing something else like OpenShift.



``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Dusty Mabe]

dustymabe added a new comment to an issue you are following:
``
as an alternative for now I have been working on getting the openshift-ansible 
installer to work against F25 atomic host. I have opened up several issuses:

https://github.com/openshift/openshift-ansible/issues/2808
https://github.com/openshift/openshift-ansible/issues/2809
https://github.com/openshift/openshift-ansible/issues/2811
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Giuseppe Scrivano]

gscrivano added a new comment to an issue you are following:
``
There are already two system containers for etcd and flannel, already available 
on Docker hub as gscrivano/etcd and gscrivano/flannel.  I'll move them to 
Fedora as soon as you tell me how to proceed for doing that.   There is no 
system container for Kubernetes yet, so we will need to run it in Docker
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Daniel J Walsh]

dwalsh added a new comment to an issue you are following:
``
I have asked Giuseppe Scrivano  to move forward on system 
containers to implement kubernetes workflow on atomic host.  He currently has 
most of services available as system containers and is moving them into 
github.com/projectatomic/atomic-system-containers

We need to get these containers built for Fedora 25.

He even has an experimental system container that runs docker in it.
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Jason Brooks]

jasonbrooks added a new comment to an issue you are following:
``
> 
> There's a kube 1.4.5 in koji for fc26 
> (http://koji.fedoraproject.org/koji/buildinfo?buildID=816481), and I've asked 
> Jan about making it available for fedora 25.
> 
> Can you keep us in the loop here on the progress of this one?

Will do
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Dusty Mabe]

dustymabe added a new comment to an issue you are following:
``

> There's a kube 1.4.5 in koji for fc26 
> (http://koji.fedoraproject.org/koji/buildinfo?buildID=816481), and I've asked 
> Jan about making it available for fedora 25.

Can you keep us in the loop here on the progress of this one? 
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org

[atomic-wg] Issue #176 `status of kubernetes on fedora atomic 25`

[Jason Brooks]

jasonbrooks reported a new issue against the project: `atomic-wg` that you are 
following:
``
We removed kubernetes, etcd, flannel, gluster and ceph from the atomic host for 
Fedora 25: 
https://pagure.io/fedora-atomic/c/219c9bb26426811a5f32188c59682ad70c3283e5?branch=f25

The idea is that these pkgs can/should be run as docker containers or system 
containers. I wrote a post about running kube in containers on CentOS Atomic, 
but things work the same for Fedora Atomic: 
http://www.projectatomic.io/blog/2016/09/running-kubernetes-in-containers-on-atomic/

However, we don't currently have fedora containers for kubernetes. There's a 
year-old PR for this: 
https://github.com/fedora-cloud/Fedora-Dockerfiles/pull/112. I used this PR as 
the basis for kube containers for CentOS: 
https://github.com/CentOS/CentOS-Dockerfiles/tree/master/kubernetes. 

Another option is using package layering to install the removed rpms if 
desired: 
http://www.projectatomic.io/blog/2016/07/hacking-and-extending-atomic-host/. 
This works, mostly, but the kube-apiserver that comes with kubernetes-master is 
granted CAP_NET_BIND_SERVICE so it can use port 443, and package layering won't 
support this: https://github.com/projectatomic/rpm-ostree/issues/462.

Whether we install kubernetes via package layering or containers, another issue 
is that the kube we have for fedora 25 is very old, 1.2, and current is 1.4.5. 
There's a kube 1.4.5 in koji for fc26 
(http://koji.fedoraproject.org/koji/buildinfo?buildID=816481), and I've asked 
Jan about making it available for fedora 25. I built it for f25 and el7 in this 
copr: https://copr.fedorainfracloud.org/coprs/jasonbrooks/kubernetes/.

kubeadm (http://kubernetes.io/docs/getting-started-guides/kubeadm/) is a slick 
way to run an up-to-date kube cluster, and the kubernetes project offers rpms 
for kubeadm, but the kubernetes-cni package they provide doesn't work with 
atomic. I made a patched package in a copr for f25 and el7 
(https://copr.fedorainfracloud.org/coprs/jasonbrooks/kube-release/) and wrote 
about it here: 
https://jebpages.com/2016/11/01/installing-kubernetes-on-centos-atomic-host-with-kubeadm/.
 The rpm grabs the binaries from the upstream project, if fedora were to 
provide this package, we'd probably want to build the binaries ourselves. 
kubeadm runs kubernetes and etcd in containers, and these containers aren't 
fedora-based, and they come from upstream. They appear to be based on busybox. 
Also, kubeadm is considered alpha at this point.

We should:

* try to move to a more recent kubernetes version
* get kubernetes containers in place for fedora
* get fedora versions of the flannel and etcd system containers in place
* document how to install kubernetes and how to use package layering on fedora
* look further into kubeadm 

``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/176
___
cloud mailing list -- cloud@lists.fedoraproject.org
To unsubscribe send an email to cloud-le...@lists.fedoraproject.org