That seems to be the issue then – one of the mariadb-galera pods is down, which is preventing the mariadb-galera service from coming online.
In my test environment, I can usually clear that by bringing down all the ONAP pods, removing everything under /dockerdata-nfs and restarting. As long as you don’t need to preserve any data, you can try that. Usually when mariadb-galera gets in that state – at least in my experience – it’s because somehow the pods can no longer replicate to each other. In a non-kubernetes environment, I’d normally be able to work around that by bringing down all but the lead node, then bouncing the lead and restarting the others so that I force the ordering. In kubernetes, it’s not quite so simple, since kubernetes will restart pods for you as you kill them – so it’s harder to force the ordering. Or perhaps there is a way to force it, but I’m afraid that’s beyond my knowledge. That’s why I go with the more brute force method of stopping everything and removing the backing store (/dockerdata-nfs) to force mariadb-galera to come up clean. Dan From: onap-discuss <onap-discuss@lists.onap.org> on behalf of Gülsüm Atıcı <gulsum.at...@ulakhaberlesme.com.tr> Reply-To: onap-discuss <onap-discuss@lists.onap.org>, "gulsum.at...@ulakhaberlesme.com.tr" <gulsum.at...@ulakhaberlesme.com.tr> Date: Wednesday, June 24, 2020 at 9:44 AM To: "timo...@m0049297.ppops.net-00191d01" <timo...@m0049297.ppops.net-00191d01>, "TIMONEY, DAN" <dt5...@att.com>, onap-discuss <onap-discuss@lists.onap.org> Subject: Re: [onap-discuss] Frankfurt SDNC modul deploy fails Dear Dan, Here are the MariaDB pods status. ubuntu@onap-nfs:~/scripts$ kubectl -n onap get pods | grep -i maria dev-mariadb-594dbb55b5-wbqs4 1/1 Running 0 163m dev-mariadb-galera-0 1/1 Running 0 129m dev-mariadb-galera-1 0/1 CrashLoopBackOff 24 128m dev-modeling-mariadb-0 1/1 Running 0 129m dev-modeling-mariadb-1 1/1 Running 6 128m dev-modeling-mariadb-2 0/1 Running 30 108m dev-policy-mariadb-0 1/1 Running 0 128m dev-so-mariadb-config-job-42g9r 0/1 Init:Error 0 23m dev-so-mariadb-config-job-cl8tp 0/1 Init:Error 0 63m dev-so-mariadb-config-job-d4rx7 0/1 Init:0/1 0 7m2s dev-so-mariadb-config-job-frvws 0/1 Init:Error 0 73m dev-so-mariadb-config-job-m2mqt 0/1 Init:Error 0 38m dev-so-mariadb-config-job-m5mvf 0/1 Init:Error 0 84m dev-so-mariadb-config-job-ncgbj 0/1 Init:Error 0 51m dev-so-mariadb-config-job-qdw99 0/1 Init:Error 0 94m dev-vfc-mariadb-0 1/1 Running 0 61m dev-vfc-mariadb-1 1/1 Running 4 50m dev-vfc-mariadb-2 1/1 Running 0 42m ubuntu@onap-nfs:~/scripts$ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21487): https://lists.onap.org/g/onap-discuss/message/21487 Mute This Topic: https://lists.onap.org/mt/75080232/21656 Group Owner: onap-discuss+ow...@lists.onap.org Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
