Hi Wei, Thanks for the sharing, i also tried to install the systemvm 4.19 I have control node and worker node under template systemvm 4.19 ( http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2 ) I tried with community cks iso 1.25.0 and 1.28.9 On systemvm 4.19 docker was not present by default, just containerd in 1.6.x version I installed docker-ce and docker-ce-cli :
apt install -y apt-transport-https ca-certificates curl software-properties-common curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null remove line about docker in double in /etc/apt/sources.list apt update apt install -y docker-ce apt install -y docker-cli i mounted the 1.28.4 cks community iso in control vm and /opt/bin/setup-kube-system i have these errors : Is it the good way to manually install kube system root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core# /opt/bin/setup-kube-system Installing binaries from /mnt/k8sdisk/ 5b1fa8e3e100: Loading layer [==================================================>] 803.8kB/803.8kB 39c831b1aa26: Loading layer [==================================================>] 26.25MB/26.25MB Loaded image: apache/cloudstack-kubernetes-autoscaler:latest 417cb9b79ade: Loading layer [==================================================>] 657.7kB/657.7kB 8d323b160d65: Loading layer [==================================================>] 24.95MB/24.95MB Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0 6a4a177e62f3: Loading layer [==================================================>] 115.2kB/115.2kB 398c9baff0ce: Loading layer [==================================================>] 16.07MB/16.07MB Loaded image: registry.k8s.io/coredns/coredns:v1.10.1 bd8a70623766: Loading layer [==================================================>] 75.78MB/75.78MB c88361932af5: Loading layer [==================================================>] 508B/508B Loaded image: kubernetesui/dashboard:v2.7.0 e023e0e48e6e: Loading layer [==================================================>] 103.7kB/103.7kB 6fbdf253bbc2: Loading layer [==================================================>] 21.2kB/21.2kB 7bea6b893187: Loading layer [==================================================>] 716.5kB/716.5kB ff5700ec5418: Loading layer [==================================================>] 317B/317B d52f02c6501c: Loading layer [==================================================>] 198B/198B e624a5370eca: Loading layer [==================================================>] 113B/113B 1a73b54f556b: Loading layer [==================================================>] 385B/385B d2d7ec0f6756: Loading layer [==================================================>] 355B/355B 4cb10dd2545b: Loading layer [==================================================>] 130.6kB/130.6kB ba9afb2b3e0c: Loading layer [==================================================>] 1.076MB/1.076MB 22bba3da6b0d: Loading layer [==================================================>] 91.29MB/91.29MB a4563151d59b: Loading layer [==================================================>] 9.544MB/9.544MB Loaded image: registry.k8s.io/etcd:3.5.9-0 54ad2ec71039: Loading layer [==================================================>] 103.7kB/103.7kB 6fbdf253bbc2: Loading layer [==================================================>] 21.2kB/21.2kB 7bea6b893187: Loading layer [==================================================>] 716.5kB/716.5kB ff5700ec5418: Loading layer [==================================================>] 317B/317B d52f02c6501c: Loading layer [==================================================>] 198B/198B e624a5370eca: Loading layer [==================================================>] 113B/113B 1a73b54f556b: Loading layer [==================================================>] 385B/385B d2d7ec0f6756: Loading layer [==================================================>] 355B/355B 4cb10dd2545b: Loading layer [==================================================>] 130.6kB/130.6kB 3f3c83f9a2fa: Loading layer [==================================================>] 724.8kB/724.8kB 6a2c4bedb8de: Loading layer [==================================================>] 32.98MB/32.98MB Loaded image: registry.k8s.io/kube-apiserver:v1.28.4 e0b89915180d: Loading layer [==================================================>] 31.72MB/31.72MB Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4 d2a4b045f3d0: Loading layer [==================================================>] 7.73MB/7.73MB 3905b20a2df2: Loading layer [==================================================>] 16.85MB/16.85MB Loaded image: registry.k8s.io/kube-proxy:v1.28.4 f35f4c1ae44f: Loading layer [==================================================>] 17.13MB/17.13MB Loaded image: registry.k8s.io/kube-scheduler:v1.28.4 d01384fea991: Loading layer [==================================================>] 19.74MB/19.74MB bcec7eb9e567: Loading layer [==================================================>] 530B/530B Loaded image: kubernetesui/metrics-scraper:v1.0.8 e3e5579ddd43: Loading layer [==================================================>] 317.6kB/317.6kB Loaded image: registry.k8s.io/pause:3.9 1b3ee35aacca: Loading layer [==================================================>] 2.796MB/2.796MB 910ce076f504: Loading layer [==================================================>] 7.425MB/7.425MB a8e8b7b8e08a: Loading layer [==================================================>] 8.101MB/8.101MB 084e56a9c24b: Loading layer [==================================================>] 4.376MB/4.376MB 334d70dc85ec: Loading layer [==================================================>] 159B/159B a13197d8bda5: Loading layer [==================================================>] 8.216MB/8.216MB Loaded image: weaveworks/weave-kube:2.8.1 998efb010df6: Loading layer [==================================================>] 728.2kB/728.2kB ed37391def99: Loading layer [==================================================>] 9.28MB/9.28MB 175a472c5f77: Loading layer [==================================================>] 299B/299B a8764e32e9fe: Loading layer [==================================================>] 613B/613B Loaded image: weaveworks/weave-npc:2.8.1 net.bridge.bridge-nf-call-iptables = 1 [init] Using Kubernetes version: v1.28.4 [preflight] Running pre-flight checks error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR CRI]: container runtime is not running: output: time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"unix:///var/run/containerd/containerd.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService" , error: exit status 1 [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher [init] Using Kubernetes version: v1.28.4 [preflight] Running pre-flight checks error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR CRI]: container runtime is not running: output: time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"unix:///var/run/containerd/containerd.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService" , error: exit status 1 [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher [init] Using Kubernetes version: v1.28.4 [preflight] Running pre-flight checks error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR CRI]: container runtime is not running: output: time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"unix:///var/run/containerd/containerd.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService" , error: exit status 1 [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher Error: kubeadm init failed! Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <ustcweiz...@gmail.com> a écrit : > Hi, > > Please refer to > https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241 > > The containerd in 4.16/4.17 sysyemvm template is too old. > > -Wei > > > On Thursday, April 11, 2024, benoit lair <kurushi4...@gmail.com> wrote: > > > Hi Rohit, > > > > I already tested with different iso from cloudstack from CKS repos, > > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from 1.24.0 > > and following others versions > > > > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails, > > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3 > > and 1.28.4 > > > > Last try, i tested k8s community iso 1.25.0 > > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in progress" > > and it fails > > If i try to ssh on control node with VR_Pub_IP port 2222 > > i have nothing in execution > > > > If i try manual install : > > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo /opt/bin/setup-kube-system > > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk. > > Installing binaries from /mnt/k8sdisk/ > > 5b1fa8e3e100: Loading layer > > [==================================================>] 803.8kB/803.8kB > > 39c831b1aa26: Loading layer > > [==================================================>] 26.25MB/26.25MB > > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest > > 417cb9b79ade: Loading layer > > [==================================================>] 657.7kB/657.7kB > > 8d323b160d65: Loading layer > > [==================================================>] 24.95MB/24.95MB > > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0 > > 256bc5c338a6: Loading layer > > [==================================================>] 122.2kB/122.2kB > > df1818f16337: Loading layer > > [==================================================>] 14.71MB/14.71MB > > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3 > > bd8a70623766: Loading layer > > [==================================================>] 75.78MB/75.78MB > > c88361932af5: Loading layer > > [==================================================>] 508B/508B > > Loaded image: kubernetesui/dashboard:v2.7.0 > > 0b031aac6569: Loading layer > > [==================================================>] 804kB/804kB > > cc0597072b83: Loading layer > > [==================================================>] 1.076MB/1.076MB > > 5a8045f731fc: Loading layer > > [==================================================>] 90.58MB/90.58MB > > f5a5ef1a005f: Loading layer > > [==================================================>] 1.188MB/1.188MB > > 5cf3bfe5417d: Loading layer > > [==================================================>] 8.506MB/8.506MB > > Loaded image: registry.k8s.io/etcd:3.5.4-0 > > 8d7366c22fd8: Loading layer > > [==================================================>] 804.1kB/804.1kB > > 7d5f5bba7c45: Loading layer > > [==================================================>] 694.9kB/694.9kB > > 8154c9483a6c: Loading layer > > [==================================================>] 32.72MB/32.72MB > > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0 > > c5f1a47b47bc: Loading layer > > [==================================================>] 29.76MB/29.76MB > > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0 > > e4d19dd12df2: Loading layer > > [==================================================>] 7.631MB/7.631MB > > ba3637ccd78a: Loading layer > > [==================================================>] 12.63MB/12.63MB > > Loaded image: registry.k8s.io/kube-proxy:v1.25.0 > > 026969d859f7: Loading layer > > [==================================================>] 14.29MB/14.29MB > > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0 > > d01384fea991: Loading layer > > [==================================================>] 19.74MB/19.74MB > > bcec7eb9e567: Loading layer > > [==================================================>] 530B/530B > > Loaded image: kubernetesui/metrics-scraper:v1.0.8 > > 961e93cda9dd: Loading layer > > [==================================================>] 307kB/307kB > > Loaded image: registry.k8s.io/pause:3.8 > > 1b3ee35aacca: Loading layer > > [==================================================>] 2.796MB/2.796MB > > 17d89e2a5199: Loading layer > > [==================================================>] 7.425MB/7.425MB > > 148b94859892: Loading layer > > [==================================================>] 8.1MB/8.1MB > > dbea472c1ced: Loading layer > > [==================================================>] 4.376MB/4.376MB > > e81f1a367c17: Loading layer > > [==================================================>] 131B/131B > > 351848f6b295: Loading layer > > [==================================================>] 8.216MB/8.216MB > > Loaded image: weaveworks/weave-kube:latest > > b20f7ff12b24: Loading layer > > [==================================================>] 727.9kB/727.9kB > > 1d49e386162e: Loading layer > > [==================================================>] 9.281MB/9.281MB > > 20145eeac146: Loading layer > > [==================================================>] 297B/297B > > d4cc72748587: Loading layer > > [==================================================>] 600B/600B > > Loaded image: weaveworks/weave-npc:latest > > net.bridge.bridge-nf-call-iptables = 1 > > [init] Using Kubernetes version: v1.25.0 > > [preflight] Running pre-flight checks > > [WARNING SystemVerification]: missing optional cgroups: blkio > > error execution phase preflight: [preflight] Some fatal errors occurred: > > [ERROR CRI]: container runtime is not running: output: E0411 > > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime > service > > failed" err="rpc error: code = Unimplemented desc = unknown service > > runtime.v1alpha2.RuntimeService" > > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime: > rpc > > error: code = Unimplemented desc = unknown service > > runtime.v1alpha2.RuntimeService" > > , error: exit status 1 > > [preflight] If you know what you are doing, you can make a check > non-fatal > > with `--ignore-preflight-errors=...` > > To see the stack trace of this error execute with --v=5 or higher > > [init] Using Kubernetes version: v1.25.0 > > [preflight] Running pre-flight checks > > [WARNING SystemVerification]: missing optional cgroups: blkio > > error execution phase preflight: [preflight] Some fatal errors occurred: > > [ERROR CRI]: container runtime is not running: output: E0411 > > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime > service > > failed" err="rpc error: code = Unimplemented desc = unknown service > > runtime.v1alpha2.RuntimeService" > > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: > rpc > > error: code = Unimplemented desc = unknown service > > runtime.v1alpha2.RuntimeService" > > , error: exit status 1 > > [preflight] If you know what you are doing, you can make a check > non-fatal > > with `--ignore-preflight-errors=...` > > To see the stack trace of this error execute with --v=5 or higher > > [init] Using Kubernetes version: v1.25.0 > > [preflight] Running pre-flight checks > > [WARNING SystemVerification]: missing optional cgroups: blkio > > error execution phase preflight: [preflight] Some fatal errors occurred: > > [ERROR CRI]: container runtime is not running: output: E0411 > > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime > service > > failed" err="rpc error: code = Unimplemented desc = unknown service > > runtime.v1alpha2.RuntimeService" > > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: > rpc > > error: code = Unimplemented desc = unknown service > > runtime.v1alpha2.RuntimeService" > > , error: exit status 1 > > [preflight] If you know what you are doing, you can make a check > non-fatal > > with `--ignore-preflight-errors=...` > > To see the stack trace of this error execute with --v=5 or higher > > Error: kubeadm init failed! > > > > > > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <rohit.ya...@shapeblue.com> a > > écrit : > > > > > Hi Benoit, > > > > > > The CKS feature has been improving over versions, I don't know if what > > > you're trying to achieve is possible with it. Maybe try a different > > version > > > of the data iso: > > > http://download.cloudstack.org/cks/ > > > > > > Alternatively, you can also have a look at the CAPC project: > > > https://cluster-api-cloudstack.sigs.k8s.io > > > > > > > > > Regards. > > > > > > > > > > > > > > > ________________________________ > > > From: benoit lair <kurushi4...@gmail.com> > > > Sent: Thursday, April 11, 2024 14:16 > > > To: us...@cloudstack.apache.org <us...@cloudstack.apache.org>; dev < > > > dev@cloudstack.apache.org> > > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS > > > > > > Any advices ? > > > > > > Best regards > > > > > > Le lun. 8 avr. 2024 à 16:53, benoit lair <kurushi4...@gmail.com> a > > écrit : > > > > > > > I am opened to every alternatives of changing system vm templates > > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :) > > > > > > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <kurushi4...@gmail.com> a > > > écrit : > > > > > > > >> Hello Folks, > > > >> > > > >> I am trying to install K8s cluster with community iso 1.28.4 > > > >> I am with a ACS 4.16.0 environment > > > >> It seems K8S is not working out of the box with v > 1.23.3 due to > > > >> containerd.io version > > > >> > > > >> I would like to release a template who will work with K8s > 1.23.3 > on > > > acs > > > >> 4.16 > > > >> How can i tell CS to take another system vm template, avoiding to > mess > > > >> normal features with VR and VPC VR keeping the systemvm template > > issued > > > >> with CS 4.16 ? > > > >> > > > >> I am with XCP-NG 8.2.1 in production > > > >> > > > >> Thanks for your help or advises > > > >> Best regards, > > > >> Benoit > > > >> > > > > > > > > > >
