dev-sdnc-controller-blueprints-54dc68ddcd-5mn44 0/1
Init:0/1 10 1h
so that container is waiting for someone else to finish
probably:
dev-sdnc-controller-blueprints-db-0 0/1 Running
0 1h
running but not finished 1/1
We need an sdnc person but I would check for errors in dev-sdnc-sdnc-0 and
dev-sdnc-sdnc-db-0
Something isnt finishing cleanly
Brian
From: Vivekanandan Muthukrishnan <[email protected]>
Sent: Tuesday, March 05, 2019 10:36 AM
To: FREEMAN, BRIAN D <[email protected]>
Cc: [email protected]
Subject: Re: [onap-discuss] Casablanca SDNC health check failed for
controller-blueprints-db-0
Hi Brain,
I could not get the logs from POD
dev-sdnc-controller-blueprints-54dc68ddcd-5mn44 as it is in the Init state.
We deployed OOM from Casablanca branch ( $ git clone -b casablanca
http://gerrit.onap.org/r/oom<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.google.com_url-3Fq-3Dhttp-3A__gerrit.onap.org_r_oom-26sa-3DD-26source-3Dhangouts-26ust-3D1551886440210000-26usg-3DAFQjCNFEVj0MJakVZNA4o1MwBv0lKIKKfw&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=ypy47zF8YroYpEAQzr1sIY73OPZSJqX9jqjjtAVyL9g&s=4SuHlKIrJouJ84p3hO3IG8beMXm85UxAKW0bsNE34FA&e=>
)
# POD status
$ kubectl get pods -n onap | grep sdnc
dev-sdnc-controller-blueprints-54dc68ddcd-5mn44 0/1
Init:0/1 10 1h
dev-sdnc-controller-blueprints-db-0 0/1 Running
0 1h
dev-sdnc-nengdb-0 1/1 Running
4 2d
dev-sdnc-network-name-gen-64fbd6cdfb-gxc7w 1/1 Running
0 2d
dev-sdnc-sdnc-0 2/2 Running
0 2d
dev-sdnc-sdnc-ansible-server-7fc98c97f4-64wfv 1/1 Running
0 2d
dev-sdnc-sdnc-db-0 2/2 Running
0 2d
dev-sdnc-sdnc-dgbuilder-754b86c668-jnj9v 1/1 Running
0 2d
dev-sdnc-sdnc-dmaap-listener-75f6b78686-ckkgx 1/1 Running
0 2d
dev-sdnc-sdnc-portal-794556b78d-72wmf 1/1 Running
0 2d
dev-sdnc-sdnc-ueb-listener-857dffb89b-khqzl 1/1 Running
2 2d
dev-so-so-sdnc-adapter-54947779dc-rv5cc 1/1 Running
0 1h
dev-vfc-vfc-zte-sdnc-driver-dbcf67f5b-z658s 1/1 Running
0 2d
$ kubectl logs -fn onap dev-sdnc-controller-blueprints-54dc68ddcd-5mn44
Error from server (BadRequest): container "controller-blueprints" in pod
"dev-sdnc-controller-blueprints-54dc68ddcd-5mn44" is waiting to start:
PodInitializing
# Here is the image summary
POD Name : dev-sdnc-controller-blueprints-54dc68ddcd-5mn44
Status : Init:0/1
Image :
nexus3.onap.org:10001/onap/ccsdk-controllerblueprints:0.3.3<https://urldefense.proofpoint.com/v2/url?u=http-3A__nexus3.onap.org-3A10001_onap_ccsdk-2Dcontrollerblueprints-3A0.3.3&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=ypy47zF8YroYpEAQzr1sIY73OPZSJqX9jqjjtAVyL9g&s=ZBGoKQwaFRPGfBhpEpWpyd2IO1ndyJCKDwoO4CcMrk0&e=>
POD Name : dev-sdnc-controller-blueprints-db-0
Status : Running 0/1
Image :
nexus3.onap.org:10001/adfinissygroup/k8s-mariadb-galera-centos:v002<https://urldefense.proofpoint.com/v2/url?u=http-3A__nexus3.onap.org-3A10001_adfinissygroup_k8s-2Dmariadb-2Dgalera-2Dcentos-3Av002&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=ypy47zF8YroYpEAQzr1sIY73OPZSJqX9jqjjtAVyL9g&s=V37b9AIOGxw0Y5X5P11JtSNxfomxlr2H2uf8HJNpbjs&e=>
Is there something wrong with the image version.
Regards
Vivek
On Tue, Mar 5, 2019 at 7:56 PM FREEMAN, BRIAN D
<[email protected]<mailto:[email protected]>> wrote:
I dont see any jira’s on this for casablanca. There was a jira on dublin but
that is was on 0.4.0-STAGING-latest
kubectl logs -fn onap onap-sdnc-controller-blueprints-6547954976-q8ctc
Is there a java exception in the either blueprint container log ?
Which version of the oom helm charts did you use for the helm deploy ?
Brian
From: [email protected]<mailto:[email protected]>
<[email protected]<mailto:[email protected]>> On Behalf Of
Vivekanandan Muthukrishnan
Sent: Tuesday, March 05, 2019 8:58 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: [onap-discuss] Casablanca SDNC health check failed for
controller-blueprints-db-0
Hi All
We tried to bounce the failed SDNC PODs the same behaviour exists. I am
attaching herewith is the logs for dev-sdnc-controller-blueprints &
sdnc-controller-blueprints-db-0 for your reference.
Could someone throw some pointers to debug this one would be of great
Thanks & Regards
Vivek
POD sdnc-controller-blueprints-db-0 Log snippet from
2019-03-05 13:41:07 140693469132544 [Note] InnoDB: Dumping buffer pool(s) not
yet started
2019-03-05 13:41:07 140694477551872 [Note] Plugin 'FEEDBACK' is disabled.
2019-03-05 13:41:07 140694477551872 [Note] Server socket created on IP:
'0.0.0.0'.
2019-03-05 13:41:08 140694477551872 [Note] mysqld: ready for connections.
Version: '10.1.24-MariaDB' socket: '/var/lib/mysql/mysql.sock' port: 3306
MariaDB Server
2019-03-05 13:41:08 140694476184320 [Note] WSREP: wsrep_notify_cmd is not
defined, skipping notification.
2019-03-05 13:41:08 140694476184320 [Note] WSREP: REPL Protocols: 7 (3, 2)
2019-03-05 13:41:08 140694476184320 [Note] WSREP: Assign initial position for
certification: 0, protocol version: 3
2019-03-05 13:41:08 140694211520256 [Note] WSREP: Service thread queue flushed.
2019-03-05 13:41:08 140694476184320 [Note] WSREP: Synchronized with group,
ready for connections
2019-03-05 13:41:08 140694476184320 [Note] WSREP: wsrep_notify_cmd is not
defined, skipping notification.
On Sun, Mar 3, 2019 at 4:10 PM Vivekanandan Muthukrishnan via
Lists.Onap.Org<https://urldefense.proofpoint.com/v2/url?u=http-3A__Lists.Onap.Org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=AgADsmgQteeWJe8fxATLZFMspKnDersoXIGBEasdZ7M&s=p7UjVlWt5ADuXfRfjrq-U4sRvOM18bSyYHOK6JSMoCY&e=>
<[email protected]<mailto:[email protected]>>
wrote:
Hi All,
We deployed ONAP Casablanca and we see that SDNC health check is failing for
dev-sdnc-controller-blueprints-db-0.
Kindly note that all SNDC PODs came up except sdnc-controller-blueprints and
sdnc-controller-blueprints-db-0.
Did any one face this issue? Please let me know, If you have any workaround
for this one.
Regards
Vivek
$ kubectl describe pod -n onap dev-sdnc-controller-blueprints-db-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 1m (x680 over 1h) kubelet, casablanca02 Readiness probe
failed: ERROR 1045 (28000): Access denied for user 'readinessProbe'@'localhost'
(using password: YES)
$ kubectl get pods -n onap | grep sdnc
dev-sdnc-controller-blueprints-54dc68ddcd-76vjz 0/1
Init:0/1 7 1h
dev-sdnc-controller-blueprints-db-0 0/1 Running
1 1h
dev-sdnc-nengdb-0 1/1 Running
4 1h
dev-sdnc-network-name-gen-64fbd6cdfb-gxc7w 1/1 Running
0 1h
dev-sdnc-sdnc-0 2/2 Running
0 1h
dev-sdnc-sdnc-ansible-server-7fc98c97f4-64wfv 1/1 Running
0 1h
dev-sdnc-sdnc-db-0 2/2 Running
0 1h
dev-sdnc-sdnc-dgbuilder-754b86c668-jnj9v 1/1 Running
0 1h
dev-sdnc-sdnc-dmaap-listener-75f6b78686-ckkgx 1/1 Running
0 1h
dev-sdnc-sdnc-portal-794556b78d-72wmf 1/1 Running
0 1h
dev-sdnc-sdnc-ueb-listener-857dffb89b-khqzl 1/1 Running
2 1h
dev-so-so-sdnc-adapter-568b59d568-nrx94 1/1 Running
0 1h
dev-vfc-vfc-zte-sdnc-driver-dbcf67f5b-z658s 1/1 Running
0 1h
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#15882): https://lists.onap.org/g/onap-discuss/message/15882
Mute This Topic: https://lists.onap.org/mt/30194482/21656
Group Owner: [email protected]
Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
