On Mon, Jul 6, 2020 at 4:33 AM Brendan Doyle <brendan.do...@oracle.com> wrote:
> Hi, > > So I'm really confused by what you have pointed me to here. As stated I do > NOT > want to use OVN as a CNI. I have a k8s cluster that use flannel as the > CNI. I simply > want to create an OVN "central" cluster as a Stateful set in my *existing* > K8 > config. > > This repo: > > https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d > > Seems to be for setting up a K8s cluster to use OVN as the CNI?? > Have you tried this? > What IP do the ovn-controllers use to reach the OVN "central cluster? > It seems to use an OVN docker image from docker.io, I want to use my own > OVN src > Do I use/modify the dist/images/Dockerfile in this repo? that has loads of > references to CNI > like I said I don't want to use OVN as the CNI?? > > A pre-req for running ovn central as a k8s app is containerize ovn central components. Hence, you need to start your own containers using docker. Either you follow the approach from ovn-k8s repo as to how to build ovn images or refer to the docker instructions in ovn repo. Since this app (ovn central) will run behind a k8s service, ovn-controller should point to the service ip of ovn central k8s app. k8s folks can comment on how to build image that is in k8s pod specs e.g http://docker.io/ovnkube/ovn-daemonset:latest > > The instructions here > https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst > seem more promising, if not a little confusing: > > IN the section "Starting OVN Central services in containers" > > Export following variables in .env and place it under project root: > > $ OVN_BRANCH=<BRANCH> > $ OVN_VERSION=<VERSION> > $ DISTRO=<LINUX_DISTRO> > $ KERNEL_VERSION=<LINUX_KERNEL_VERSION> > $ GITHUB_SRC=<GITHUB_URL> > $ DOCKER_REPO=<REPO_TO_PUSH_IMAGE> > > > Does it mean create a file called ".env" and place it in the toplevel dir > of the cloned ovn repo? > Or does it mean just add these to you shell environment (i.e put them in > .bashrc)? > > You can just export OVN_BRANCH=xx in your shell for all variables and build your containers with desired distro/version using make build > > Then we have: > > 1) > > Start OVN containers using below command: > > $ docker run -itd --net=host --name=ovn-nb \ > <docker_repo>:<tag> ovn-nb-tcp > > $ docker run -itd --net=host --name=ovn-sb \ > <docker_repo>:<tag> ovn-sb-tcp > > $ docker run -itd --net=host --name=ovn-northd \ > <docker_repo>:<tag> ovn-northd-tcp > > followed by > > 2) > > $ docker run -e "host_ip=<host_ip>" -e "nb_db_port=<port>" -itd \ > --name=ovn-nb-raft --net=host --privileged <docker_repo>:<tag> \ > ovn-nb-cluster-create > > $ docker run -e "host_ip=<host_ip>" -e "sb_db_port=<port>" -itd \ > --name=ovn-sb-raft --net=host --privileged <docker_repo>:<tag> \ > ovn-sb-cluster-create > > $ docker run -e "OVN_NB_DB=tcp:<node1>:6641,tcp:<node2>:6641,\ > tcp:<node3>:6641" -e "OVN_SB_DB=tcp:<node1>:6642,tcp:<node2>:6642,\ > tcp:<node3>:6642" -itd --name=ovn-northd-raft <docker_repo>:<tag> \ > ovn-northd-cluster > > Does it mean do 1), then 2) or does it mean do 1) for non HA OVN central > *OR* 2) > for HA/clustered OVN Central? > > Doc says Start OVN containers in cluster mode using below command on node2 and node3 to make them join the peer using below command:. Hence, you can even play with just docker on 3 nodes where you run step1 on node1 that creates cluster and do the join-cluster on rest two nodes to give you a clear idea before moving to pod in k8s. Not sure if you need more details to update doc. We can always improvise. Upstream ovn-k8s does the same for pods where e.g. ovn-kube0 pod creates a cluster and rest two pods joins > It's not clear > > Thanks > > > > > > > On 25/06/2020 17:36, aginwala wrote: > > Hi: > > There are a couple of options as I have been exploring this too: > > 1. Upstream ovn-k8s patches ( > https://github.com/ovn-org/ovn-kubernetes/commit/a07b1a01af7e37b15c2e5f179ffad2b9f25a083d) > uses statefulset and headless service for starting ovn central raft cluster > with 3 replicas. Cluster startup code and pod specs are pretty neat that > addresses most of the doubts. > > OVN components have been containerized too to start them in pods. You can > also refer to > https://github.com/ovn-org/ovn/blob/d6b56b1629d5984ef91864510f918e232efb89de/Documentation/intro/install/general.rst > for the same and use them to make it work in pod specs too. > > > 2. Write a new ovn operator similar to etcd operator > https://github.com/coreos/etcd-operator which just takes the count of > raft replicas and does the job in the background. > > I also added ovn-k8s group so they can comment on any other ideas too. > Hope it helps. > > > > On Thu, Jun 25, 2020 at 7:15 AM Brendan Doyle <brendan.do...@oracle.com> > wrote: > >> Hi, >> >> So I'm trying to find information on setting up an OVN "Central/Master" >> high availability (HA) >> Not as Active-Backup with Pacemaker, but as a cluster. But I want to >> deploy and manage that >> cluster as a Kubernetes service . >> >> I can find lots of stuff on "ovn-kube" but this seems to be using OVN as >> a kubernetes CNI instead of >> Flannel etc. But this is not what I want to do, I have a kubernetes >> cluster using Flannel as the CNI, >> now I want to deploy a HA OVN "Central" as a kubernetes service. Kind >> of like how you can deploy >> a MySQL cluster in kubernetes using a SatefulSet deployment. >> >> I have found this: >> https://github.com/ovn-org/ovn-kubernetes#readme >> >> But it is not clear to me if this is how to setup OVN as a kubernetes >> CNI or it's how to setup a HA >> OVN central as kubernetes service. >> >> Can anybody comment, has anyone done this? >> >> >> I guess I could run an OVN central as standalone and use a kubernetes >> deployment with 3 >> replica sets and "export" as a NodePort service. And have a >> floating/VIP on my kubernetes >> nodes. And direct ovn-controllers to the VIP. So only the pod that holds >> the VIP would service >> requests. This would work and give HA, but you don't get the performance >> of an OVN >> clustered Database Model, where each OVN central could service requests. >> >> >> >> >> Thanks >> >> >> Rdgs >> Brendan >> >> _______________________________________________ >> discuss mailing list >> disc...@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >> > >
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss