Hi, In Casablanca, Those ODL features were installed when you are installing APPC container. We moved ODL feature installation when APPC container gets built in R4, so it won’t take that long in R4.
But 1802 seconds installing is just too long. Did you check your k8s env or VM capacity? Taka From: [email protected] <[email protected]> On Behalf Of Vivekanandan Muthukrishnan Sent: Sunday, March 3, 2019 6:23 AM To: [email protected] Subject: [onap-discuss] Casablanca APPC Readiness probe failed: APPC is not healthy for more than 2 hours Hi All, It seems like APPC POD dev-appc-appc-0 (Container aapc) is taking more time to install KARAF bundles. And dev-appc-appc-ansible-server is keep getting restarted. It this expected ? Are there any workaround to offload KARAF packages from local maven repository? # All APPC PODs $ kubectl get pods -n onap | grep appc dev-appc-appc-0 1/2 Running 0 2h dev-appc-appc-ansible-server-6877b497df-j544r 0/1 Init:0/1 3 37m dev-appc-appc-cdt-77bccf4847-fmtpw 1/1 Running 0 2h dev-appc-appc-db-0 1/1 Running 1 2h dev-appc-appc-db-1 1/1 Running 0 2h dev-appc-appc-db-2 1/1 Running 1 2h dev-appc-appc-dgbuilder-f7565468-fnrz6 1/1 Running 0 2h # It seems like ODL KARAF features are still getting installed and it is taking more time $ kubectl logs -n onap dev-appc-appc-0 -c appc Adding feature url mvn:org.onap.appc/onap-appc-design-services/1.4.4/xml/features Archive: /opt/onap/appc/features/appc-interfaces-service/appc-interfaces-service-1.4.4.zip creating: /opt/opendaylight/system/com/google/code/gson/gson/2.8.0/ creating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/ creating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/1.4.4/ creating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/ creating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/1.4.4/ creating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/ creating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/1.4.4/ inflating: /opt/opendaylight/system/com/google/code/gson/gson/2.8.0/_remote.repositories inflating: /opt/opendaylight/system/com/google/code/gson/gson/2.8.0/gson-2.8.0.jar inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/1.4.4/appc-interfaces-service-model-1.4.4.jar inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/1.4.4/_remote.repositories inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-model/maven-metadata-local.xml inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/1.4.4/appc-interfaces-service-bundle-1.4.4.jar inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/1.4.4/_remote.repositories inflating: /opt/opendaylight/system/org/onap/appc/appc-interfaces-service-bundle/maven-metadata-local.xml inflating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/1.4.4/_remote.repositories inflating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/1.4.4/onap-appc-interfaces-service-1.4.4-features.xml inflating: /opt/opendaylight/system/org/onap/appc/onap-appc-interfaces-service/maven-metadata-local.xml Adding feature url mvn:org.onap.appc/onap-appc-interfaces-service/1.4.4/xml/features Installing onap-appc-core Install of onap-appc-core took 1802 seconds Sleep Finished Installing onap-appc-metric $ kubectl describe pod -n onap dev-appc-appc-0 Name: dev-appc-appc-0 Namespace: onap Node: casablanca03/192.168.122.233<https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.122.233&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=5QdUf3h3TDBxaeo_dw16EKiqx8IfBbGdt_NveymVWFc&e=> Start Time: Sun, 03 Mar 2019 08:39:03 +0000 Labels: app=appc controller-revision-hash=dev-appc-appc-69d746947b release=dev-appc statefulset.kubernetes.io/pod-name=dev-appc-appc-0<https://urldefense.proofpoint.com/v2/url?u=http-3A__statefulset.kubernetes.io_pod-2Dname-3Ddev-2Dappc-2Dappc-2D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=RxKrnUof5JnuS1iyR6UR_vS682wQdsNjgRy0LMrRu_s&e=> Annotations: <none> Status: Running IP: 10.42.135.188 Controlled By: StatefulSet/dev-appc-appc Init Containers: appc-readiness: Container ID: docker://219c491421ce1c5f83539e84508a642c0726930fe4078a72fbf0de65d17aa236 Image: oomk8s/readiness-check:2.0.0 Image ID: docker://sha256:867cb038e1d2445a6e5aedc3b5f970dacc8249ab119d6c2e088e10df886ff51f Port: <none> Host Port: <none> Command: /root/ready.py Args: --container-name appc-db State: Terminated Reason: Completed Exit Code: 0 Started: Sun, 03 Mar 2019 08:40:03 +0000 Finished: Sun, 03 Mar 2019 08:45:25 +0000 Ready: True Restart Count: 0 Environment: NAMESPACE: onap (v1:metadata.namespace) Mounts: /var/run/secrets/kubernetes.io/serviceaccount<https://urldefense.proofpoint.com/v2/url?u=http-3A__kubernetes.io_serviceaccount&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=K_7RtZDFgrdJQ-k66gJ91ENalU4kW5c5vPLpdlFU4tY&e=> from default-token-lw9wt (ro) Containers: appc: Container ID: docker://f1c058a8fb540c20beaffcad0349191877527824a0b6e11d55150694c65d6427 Image: nexus3.onap.org:10001/onap/appc-image:1.4.4<https://urldefense.proofpoint.com/v2/url?u=http-3A__nexus3.onap.org-3A10001_onap_appc-2Dimage-3A1.4.4&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=L6GfGwiv2Emr4_HdXjZBEWlS1WNnZZ1VF2wD6zQe5Wc&e=> Image ID: docker-pullable://nexus3.onap.org:10001/onap/appc-image@sha256:a60c532af06550e0fe9847c5f9d7e30d1902980439497dc98b2b32068839d2b6<https://urldefense.proofpoint.com/v2/url?u=http-3A__nexus3.onap.org-3A10001_onap_appc-2Dimage-40sha256-3Aa60c532af06550e0fe9847c5f9d7e30d1902980439497dc98b2b32068839d2b6&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=BvjPIa7liaAeP3XAggqQY4NvJZWOxNG26BYanJjbcwk&e=> Ports: 8181/TCP, 1830/TCP Host Ports: 0/TCP, 0/TCP Command: /opt/appc/bin/startODL.sh State: Running Started: Sun, 03 Mar 2019 08:48:29 +0000 Ready: False Restart Count: 0 Readiness: exec [/opt/appc/bin/health_check.sh] delay=10s timeout=1s period=10s #success=1 #failure=3 Environment: MYSQL_ROOT_PASSWORD: <set to the key 'db-root-password' in secret 'dev-appc-appc'> Optional: false SDNC_CONFIG_DIR: /opt/onap/appc/data/properties APPC_CONFIG_DIR: /opt/onap/appc/data/properties DMAAP_TOPIC_ENV: SUCCESS ENABLE_AAF: true ENABLE_ODL_CLUSTER: false APPC_REPLICAS: 1 Mounts: /etc/localtime from localtime (ro) /opt/onap/appc/bin/health_check.sh from onap-appc-bin (rw) /opt/onap/appc/bin/installAppcDb.sh from onap-appc-bin (rw) /opt/onap/appc/bin/startODL.sh from onap-appc-bin (rw) /opt/onap/appc/data/properties/aaa-app-config.xml from onap-appc-data-properties (rw) /opt/onap/appc/data/properties/aaiclient.properties from onap-appc-data-properties (rw) /opt/onap/appc/data/properties/appc.properties from onap-appc-data-properties (rw) /opt/onap/appc/data/properties/cadi.properties from onap-appc-data-properties (rw) /opt/onap/appc/data/properties/dblib.properties from onap-appc-data-properties (rw) /opt/onap/appc/data/properties/svclogic.properties from onap-appc-data-properties (rw) /opt/onap/appc/svclogic/bin/showActiveGraphs.sh from onap-appc-svclogic-bin (rw) /opt/onap/appc/svclogic/config/svclogic.properties from onap-appc-svclogic-config (rw) /opt/onap/ccsdk/bin/installSdncDb.sh from onap-sdnc-bin (rw) /opt/onap/ccsdk/bin/startODL.sh from onap-sdnc-bin (rw) /opt/onap/ccsdk/data/properties/aaiclient.properties from onap-sdnc-data-properties (rw) /opt/onap/ccsdk/data/properties/dblib.properties from onap-sdnc-data-properties (rw) /opt/onap/ccsdk/data/properties/svclogic.properties from onap-sdnc-data-properties (rw) /opt/onap/ccsdk/svclogic/bin/showActiveGraphs.sh from onap-sdnc-svclogic-bin (rw) /opt/onap/ccsdk/svclogic/config/svclogic.properties from onap-sdnc-svclogic-config (rw) /opt/opendaylight/current/daexim from dev-appc-appc-data (rw) /opt/opendaylight/current/etc/org.ops4j.pax.logging.cfg from log-config (rw) /var/log/onap from logs (rw) /var/run/secrets/kubernetes.io/serviceaccount<https://urldefense.proofpoint.com/v2/url?u=http-3A__kubernetes.io_serviceaccount&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=K_7RtZDFgrdJQ-k66gJ91ENalU4kW5c5vPLpdlFU4tY&e=> from default-token-lw9wt (ro) filebeat-onap: Container ID: docker://daf7ddc4a3e4945a1a4cab940906022248696e72c90bb15fa01144cacd3a1833 Image: docker.elastic.co/beats/filebeat:5.5.0<https://urldefense.proofpoint.com/v2/url?u=http-3A__docker.elastic.co_beats_filebeat-3A5.5.0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=8NTH89zzSFu9SMF9-3wjBtn0DoxJDqZsB9iv-3UTZdc&e=> Image ID: docker://sha256:b61327632415b6d374b9f34cea71cb14f9c352e5259140ce6e3c8eaf8becaa1b Port: <none> Host Port: <none> State: Running Started: Sun, 03 Mar 2019 08:48:30 +0000 Ready: True Restart Count: 0 Environment: <none> Mounts: /usr/share/filebeat/data from data-filebeat (rw) /usr/share/filebeat/filebeat.yml from filebeat-conf (rw) /var/log/onap from logs (rw) /var/run/secrets/kubernetes.io/serviceaccount<https://urldefense.proofpoint.com/v2/url?u=http-3A__kubernetes.io_serviceaccount&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=K_7RtZDFgrdJQ-k66gJ91ENalU4kW5c5vPLpdlFU4tY&e=> from default-token-lw9wt (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: dev-appc-appc-data: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: dev-appc-appc-data-dev-appc-appc-0 ReadOnly: false localtime: Type: HostPath (bare host directory volume) Path: /etc/localtime HostPathType: filebeat-conf: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-filebeat Optional: false log-config: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-logging-cfg Optional: false logs: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: data-filebeat: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: onap-appc-data-properties: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-onap-appc-data-properties Optional: false onap-appc-svclogic-config: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-onap-appc-svclogic-config Optional: false onap-appc-svclogic-bin: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-onap-appc-svclogic-bin Optional: false onap-appc-bin: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-onap-appc-bin Optional: false onap-sdnc-data-properties: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-onap-sdnc-data-properties Optional: false onap-sdnc-svclogic-config: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-onap-sdnc-svclogic-config Optional: false onap-sdnc-svclogic-bin: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-onap-sdnc-svclogic-bin Optional: false onap-sdnc-bin: Type: ConfigMap (a volume populated by a ConfigMap) Name: dev-appc-appc-onap-sdnc-bin Optional: false default-token-lw9wt: Type: Secret (a volume populated by a Secret) SecretName: default-token-lw9wt Optional: false QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute<https://urldefense.proofpoint.com/v2/url?u=http-3A__node.kubernetes.io_not-2Dready-3ANoExecute&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=r99_KtHvCDVNp-zIMMlkm8mrd6LpRRJbJEB9yR9yXLY&e=> for 300s node.kubernetes.io/unreachable:NoExecute<https://urldefense.proofpoint.com/v2/url?u=http-3A__node.kubernetes.io_unreachable-3ANoExecute&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=i5VHNTZ3SDPgIii87sudZA&m=PeuJ6XKmM0ul8_BO9rGOx8ZhqqQzlnaReJYk6TIJ6gE&s=ByCvoW97wNZ5FWPnIYUkhGPdq3rSabaE9J4sGAQr1nw&e=> for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Unhealthy 26m (x9 over 1h) kubelet, casablanca03 Readiness probe failed: APPC is not healthy. ++ ps -e ++ grep startODL ++ wc -l + startODL_status=1 ++ grep Waiting ++ wc -l ++ /opt/opendaylight/current/bin/client bundle:list + waiting_bundles=0 ++ /opt/opendaylight/current/bin/client system:start-level + run_level='Level 100' + '[' 'Level 100' == 'Level 100' ']' + '[' 1 -lt 1 ']' + echo APPC is not healthy. + exit 1 Warning Unhealthy 20m (x212 over 1h) kubelet, casablanca03 (combined from similar events): Readiness probe failed: APPC is not healthy. ++ ps -e ++ grep startODL ++ wc -l + startODL_status=1 ++ grep Waiting ++ /opt/opendaylight/current/bin/client bundle:list ++ wc -l + waiting_bundles=0 ++ /opt/opendaylight/current/bin/client system:start-level + run_level='Level 100' + '[' 'Level 100' == 'Level 100' ']' + '[' 1 -lt 1 ']' + echo APPC is not healthy. + exit 1 Warning Unhealthy 16m kubelet, casablanca03 Readiness probe failed: APPC is not healthy. ++ wc -l ++ ps -e ++ grep startODL + startODL_status=1 ++ /opt/opendaylight/current/bin/client bundle:list ++ grep Waiting ++ wc -l + waiting_bundles=0 ++ /opt/opendaylight/current/bin/client system:start-level + run_level='Level 100' + '[' 'Level 100' == 'Level 100' ']' + '[' 1 -lt 1 ']' + echo APPC is not healthy. + exit 1 Warning Unhealthy 11m (x203 over 1h) kubelet, casablanca03 Readiness probe failed: APPC is not healthy. ++ ps -e ++ grep startODL ++ wc -l + startODL_status=1 ++ /opt/opendaylight/current/bin/client bundle:list ++ grep Waiting ++ wc -l + waiting_bundles=0 ++ /opt/opendaylight/current/bin/client system:start-level + run_level='Level 100' + '[' 'Level 100' == 'Level 100' ']' + '[' 1 -lt 1 ']' + echo APPC is not healthy. + exit 1 Warning Unhealthy 5m (x12 over 1h) kubelet, casablanca03 Readiness probe failed: APPC is not healthy. ++ ps -e ++ wc -l ++ grep startODL + startODL_status=1 ++ /opt/opendaylight/current/bin/client bundle:list ++ wc -l ++ grep Waiting + waiting_bundles=0 ++ /opt/opendaylight/current/bin/client system:start-level + run_level='Level 100' + '[' 'Level 100' == 'Level 100' ']' + '[' 1 -lt 1 ']' + echo APPC is not healthy. + exit 1 Warning Unhealthy 1m (x31 over 1h) kubelet, casablanca03 Readiness probe failed: APPC is not healthy. ++ ps -e ++ wc -l ++ grep startODL + startODL_status=1 ++ /opt/opendaylight/current/bin/client bundle:list ++ grep Waiting ++ wc -l + waiting_bundles=0 ++ /opt/opendaylight/current/bin/client system:start-level + run_level='Level 100' + '[' 'Level 100' == 'Level 100' ']' + '[' 1 -lt 1 ']' + echo APPC is not healthy. + exit 1 === We can access the karaf $ kubectl exec -it -n onap dev-appc-appc-0 -c appc -- /bin/bash root@dev-appc-appc-0:/# ls bin dev home lib64 mnt proc run srv tmp var boot etc lib media opt root sbin sys usr root@dev-appc-appc-0:/# ps -ef | grep java root 217 156 15 08:48 ? 00:21:44 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Djava.security.properties=/opt/opendaylight/etc/odl.java.security -Xms128M -Xmx2048m -XX:+UnlockDiagnosticVMOptions -XX:+HeapDumpOnOutOfMemoryError -Dcom.sun.management.jmxremote -Djava.security.egd=file:/dev/./urandom -Djava.endorsed.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/endorsed:/usr/lib/jvm/java-8-openjdk-amd64/lib/endorsed:/opt/opendaylight/lib/endorsed -Djava.ext.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/usr/lib/jvm/java-8-openjdk-amd64/lib/ext:/opt/opendaylight/lib/ext -Dkaraf.instances=/opt/opendaylight/instances -Dkaraf.home=/opt/opendaylight -Dkaraf.base=/opt/opendaylight -Dkaraf.data=/opt/opendaylight/data -Dkaraf.etc=/opt/opendaylight/etc -Dkaraf.restart.jvm.supported=true -Djava.io.tmpdir=/opt/opendaylight/data/tmp -Djava.util.logging.config.file=/opt/opendaylight/etc/java.util.logging.properties -Dkaraf.startLocalConsole=false -Dkaraf.startRemoteShell=true -classpath /opt/opendaylight/lib/boot/org.apache.karaf.diagnostic.boot-4.1.5.jar:/opt/opendaylight/lib/boot/org.apache.karaf.jaas.boot-4.1.5.jar:/opt/opendaylight/lib/boot/org.apache.karaf.main-4.1.5.jar:/opt/opendaylight/lib/boot/org.osgi.core-6.0.0.jar org.apache.karaf.main.Main root 10908 1660 0 10:53 ? 00:00:04 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dkaraf.instances=/opt/opendaylight/instances -Dkaraf.home=/opt/opendaylight -Dkaraf.base=/opt/opendaylight -Dkaraf.etc=/opt/opendaylight/etc -Djava.io.tmpdir=/opt/opendaylight/data/tmp -Djava.util.logging.config.file=/opt/opendaylight/etc/java.util.logging.properties -classpath /opt/opendaylight/system/org/apache/karaf/org.apache.karaf.client/4.1.5/org.apache.karaf.client-4.1.5.jar:/opt/opendaylight/system/org/apache/sshd/sshd-core/1.6.0/sshd-core-1.6.0.jar:/opt/opendaylight/system/org/fusesource/jansi/jansi/1.17/jansi-1.17.jar:/opt/opendaylight/system/org/jline/jline/3.6.0/jline-3.6.0.jar:/opt/opendaylight/system/org/slf4j/slf4j-api/1.7.12/slf4j-api-1.7.12.jar org.apache.karaf.client.Main feature:install -r onap-appc-metric root 25586 25509 0 11:10 ? 00:00:00 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Dkaraf.instances=/opt/opendaylight/current/instances -Dkaraf.home=/opt/opendaylight/current -Dkaraf.base=/opt/opendaylight/current -Dkaraf.etc=/opt/opendaylight/current/etc -Djava.io.tmpdir=/opt/opendaylight/current/data/tmp -Djava.util.logging.config.file=/opt/opendaylight/current/etc/java.util.logging.properties -classpath /opt/opendaylight/current/system/org/apache/karaf/org.apache.karaf.client/4.1.5/org.apache.karaf.client-4.1.5.jar:/opt/opendaylight/current/system/org/apache/sshd/sshd-core/1.6.0/sshd-core-1.6.0.jar:/opt/opendaylight/current/system/org/fusesource/jansi/jansi/1.17/jansi-1.17.jar:/opt/opendaylight/current/system/org/jline/jline/3.6.0/jline-3.6.0.jar:/opt/opendaylight/current/system/org/slf4j/slf4j-api/1.7.12/slf4j-api-1.7.12.jar org.apache.karaf.client.Main system:start-level root 25637 25428 0 11:10 ? 00:00:00 grep --color=auto java -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15849): https://lists.onap.org/g/onap-discuss/message/15849 Mute This Topic: https://lists.onap.org/mt/30194640/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
