Kibana uses a lot of resources when it is initializing and creating indices. When idle, the utilization does drop, but our experience has shown that using Kibana when there’s actual data in ElasticSearch can be resource intensive.
From: sylvain.desbure...@orange.com <sylvain.desbure...@orange.com> Sent: Friday, November 23, 2018 10:53 AM To: James MacNider <james.macni...@amdocs.com>; onap-discuss@lists.onap.org Subject: Re: [onap-discuss] [OOM][Casablanca][POMBA] pomba kibana is OOMKilled all the time James, I moved to limits to 1.6G and it works now: onap onap-pomba-pomba-kibana-dbf869f9b-9cgs2 1 (12%) 2 (25%) 600Mi (3%) 1600Mi (10%) the weird stuff is that we don’t use that much as you can see… --- Sylvain Desbureaux De : James MacNider <james.macni...@amdocs.com<mailto:james.macni...@amdocs.com>> Date : vendredi 23 novembre 2018 à 16:23 À : "onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>" <onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>>, DESBUREAUX Sylvain TGI/OLN <sylvain.desbure...@orange.com<mailto:sylvain.desbure...@orange.com>> Objet : RE: [onap-discuss] [OOM][Casablanca][POMBA] pomba kibana is OOMKilled all the time Hi Sylvain, If it’s not too much trouble could you try with 1.6Gi? Should that fail I’d step up to 1.8Gi. Thanks, James From: onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> <onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>> On Behalf Of Sylvain Desbureaux Sent: Friday, November 23, 2018 8:45 AM To: James MacNider <james.macni...@amdocs.com<mailto:james.macni...@amdocs.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Subject: Re: [onap-discuss] [OOM][Casablanca][POMBA] pomba kibana is OOMKilled all the time Helllo James, That’s why I said the other kibana were workins as one as higher resources limits (logging) and the other is not using resources limits (see https://jira.onap.org/browse/CLAMP-250). I can test with higher values if you want. Regards, --- Sylvain Desbureaux De : James MacNider <james.macni...@amdocs.com<mailto:james.macni...@amdocs.com>> Date : vendredi 23 novembre 2018 à 14:18 À : "onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>" <onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>>, DESBUREAUX Sylvain TGI/OLN <sylvain.desbure...@orange.com<mailto:sylvain.desbure...@orange.com>> Objet : RE: [OOM][Casablanca][POMBA] pomba kibana is OOMKilled all the time Hi Sylvain, There have been occasional reports of this problem with this pod but it’s not been readily repeatable. Believe it or not, it is probably due to the resource limits being too conservative. The other ONAP kibana instances you mention both have higher resource limits (2Gi for clamp, 4Gi for logging). I’ve raised https://jira.onap.org/browse/LOG-855 to track this issue. Thanks, James From: onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> <onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>> On Behalf Of Sylvain Desbureaux Sent: Friday, November 23, 2018 5:27 AM To: onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Subject: [onap-discuss] [OOM][Casablanca][POMBA] pomba kibana is OOMKilled all the time Hello, I’m doing a daily deployment of a full ONAP (using https://gitlab.com/Orange-OpenSource/lfn/onap/onap_oom_automatic_installation). Most of the deployment is working but I keep getting some errors. In particular, I’ve got pomba-kibana pods which gets “OOMKilled” at every deployment on the last week. I’m deploying a “small” flavor with request at 1 Cpu (which is HUGE by the way) and 600M RAM and limits at 2 Cpu, 1.2G RAM. The 2 other kibana (from log and clamp) are working fine. Am I the only one having this issue? -- [cid:image001.png@01D33D23.D1353070]<http://www.orange.com/> Sylvain Desbureaux Senior Automation Architect ORANGE/IMT/OLN/CNC/NCA/SINA Fixe : +33 2 96 07 13 80 <https://monsi.sso.francetelecom.fr/index.asp?target=http%3A%2F%2Fclicvoice.sso.francetelecom.fr%2FClicvoiceV2%2FToolBar.do%3Faction%3Ddefault%26rootservice%3DSIGNATURE%26to%3D+33%202%2096%2007%2013%2080> Mobile : +33 6 71 17 25 57 <https://monsi.sso.francetelecom.fr/index.asp?target=http%3A%2F%2Fclicvoice.sso.francetelecom.fr%2FClicvoiceV2%2FToolBar.do%3Faction%3Ddefault%26rootservice%3DSIGNATURE%26to%3D+33%206%2071%2017%2025%2057> sylvain.desbure...@orange.com<mailto:sylvain.desbure...@orange.com> _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service <https://www.amdocs.com/about/email-terms-of-service> -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14016): https://lists.onap.org/g/onap-discuss/message/14016 Mute This Topic: https://lists.onap.org/mt/28292082/21656 Group Owner: onap-discuss+ow...@lists.onap.org Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
