Thank you Michael, thats helpful. Best Regards Sirisha Gopigiri
On Wed, Oct 17, 2018 at 8:57 PM Michael O'Brien <frank.obr...@amdocs.com> wrote: > Sirisha, > > Hi, that is a common OS based issue for some new VM’s depending on your > undercloud – this is the vm.max_map_count of 64k instead of 256Kb > limitation - not 256Gb on the OS. > > This occurs for all elasticsearch containers whether AAI, CLAMP, LOG, > SDC…. > > Run the following before doing a helm install – and all the ES > containers including the one in AAI will come up fine. > > > > sudo sysctl -w vm.max_map_count=*262144* > > > > There is documentation on this. > > First noticed this in Rancher 1.6.11 – fixed via > https://jira.onap.org/browse/LOG-334 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__jira.onap.org_browse_LOG-2D334&d=DwMGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=5PWCtn_zD8jT6xuuwP7DpGwXiA7-u1RBXx8E9Vrv2NI&m=afk3qptXbsXCXwnbkUPflsF2JY0_3IwzwHnt5X0mMQA&s=E_mtPnHjkrSJuYnaMPeIMyR30ruPDcLOtbnAVHat0w4&e=> > (OOM-431). > > > https://wiki.onap.org/display/DW/Cloud+Native+Deployment#CloudNativeDeployment-vm.max_map_count64to256kblimit > <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.onap.org_display_DW_Cloud-2BNative-2BDeployment-23CloudNativeDeployment-2Dvm.max-5Fmap-5Fcount64to256kblimit&d=DwMGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=5PWCtn_zD8jT6xuuwP7DpGwXiA7-u1RBXx8E9Vrv2NI&m=afk3qptXbsXCXwnbkUPflsF2JY0_3IwzwHnt5X0mMQA&s=DcjGuARrs2fovHN-EXiwmqjAhWcakr9RRDHhvQ3BwME&e=> > > > > And I run this command by default in the installation script to > bootstrap Helm/Kubernetes/Docker via the Rancher install in (there are 4 > other installers that do the same in ONAP) > > Not in > > > https://git.onap.org/logging-analytics/tree/deploy/rancher/oom_rancher_setup.sh > <https://urldefense.proofpoint.com/v2/url?u=https-3A__git.onap.org_logging-2Danalytics_tree_deploy_rancher_oom-5Francher-5Fsetup.sh&d=DwMGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=5PWCtn_zD8jT6xuuwP7DpGwXiA7-u1RBXx8E9Vrv2NI&m=afk3qptXbsXCXwnbkUPflsF2JY0_3IwzwHnt5X0mMQA&s=dHVEv1UKsqLXUkBxveomzkPCz_gWl0AlD_rVBZWfdYw&e=> > > but in the OOM CD deployment script just before doing a helm install > > https://git.onap.org/logging-analytics/tree/deploy/cd.sh#n49 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__git.onap.org_logging-2Danalytics_tree_deploy_cd.sh-23n49&d=DwMGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=5PWCtn_zD8jT6xuuwP7DpGwXiA7-u1RBXx8E9Vrv2NI&m=afk3qptXbsXCXwnbkUPflsF2JY0_3IwzwHnt5X0mMQA&s=ZKOR5gENseMIlGPYii-D11WSHuypug5jOqpfzPv3dSI&e=> > > # fix virtual memory for onap-log:elasticsearch under Rancher 1.6.11 - > OOM-431 > > sudo sysctl -w vm.max_map_count=*262144* > > > > thank you > > /michael > > > > > > *From:* onap-discuss@lists.onap.org <onap-discuss@lists.onap.org> *On > Behalf Of *Gopigiri, Sirisha via Lists.Onap.Org > *Sent:* Wednesday, October 17, 2018 9:56 AM > *To:* onap-discuss@lists.onap.org; Viswanath V Kumar Skand Priya < > viswanath.kumarskandpr...@verizon.com> > *Subject:* [onap-discuss] [AAI] [OOM] AAI Elastic search pod not coming > up in master branch > > > > Hi All, > > > > I am installing ONAP using OOM master branch with AAI enabled, in a two > node kubernetes cluster with each node having 64GB RAM. But the AAI elastic > search pod is expecting 256GB RAM and is stuck in crashloopback state. > Please find the logs below. > > > > So, to install OOM should the kubernetes node have 256 GB, or am I missing > something. Kindly let me know. > > > > [2018-10-17T13:20:19,547][INFO ][o.e.n.Node ] [ES_ONAP] > initializing ... > > [2018-10-17T13:20:19,739][INFO ][o.e.e.NodeEnvironment ] [ES_ONAP] > using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/vda1)]], > net usable_space [81.1gb], net total_space [96.8gb], types [ext4] > > [2018-10-17T13:20:19,740][INFO ][o.e.e.NodeEnvironment ] [ES_ONAP] heap > size [989.8mb], compressed ordinary object pointers [true] > > [2018-10-17T13:20:19,743][INFO ][o.e.n.Node ] [ES_ONAP] node > name [ES_ONAP], node ID [BL4G5DFCS9G-wcrQS64tUQ] > > [2018-10-17T13:20:19,744][INFO ][o.e.n.Node ] [ES_ONAP] > version[6.1.2], pid[1], build[5b1fea5/2018-01-10T02:35:59.208Z], > OS[Linux/4.4.0-135-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit > Server VM/1.8.0_151/25.151-b12] > > [2018-10-17T13:20:19,744][INFO ][o.e.n.Node ] [ES_ONAP] JVM > arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, > -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, > -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, > -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, > -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, > -Dio.netty.recycler.maxCapacityPerThread=0, > -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, > -XX:+HeapDumpOnOutOfMemoryError, -Des.cgroups.hierarchy.override=/, > -Des.path.home=/usr/share/elasticsearch, > -Des.path.conf=/usr/share/elasticsearch/config] > > [2018-10-17T13:20:23,681][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [aggs-matrix-stats] > > [2018-10-17T13:20:23,682][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [analysis-common] > > [2018-10-17T13:20:23,682][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [ingest-common] > > [2018-10-17T13:20:23,683][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [lang-expression] > > [2018-10-17T13:20:23,683][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [lang-mustache] > > [2018-10-17T13:20:23,683][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [lang-painless] > > [2018-10-17T13:20:23,683][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [mapper-extras] > > [2018-10-17T13:20:23,684][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [parent-join] > > [2018-10-17T13:20:23,684][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [percolator] > > [2018-10-17T13:20:23,685][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [reindex] > > [2018-10-17T13:20:23,685][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [repository-url] > > [2018-10-17T13:20:23,685][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [transport-netty4] > > [2018-10-17T13:20:23,685][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded module [tribe] > > [2018-10-17T13:20:23,687][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded plugin [ingest-geoip] > > [2018-10-17T13:20:23,687][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded plugin [ingest-user-agent] > > [2018-10-17T13:20:23,687][INFO ][o.e.p.PluginsService ] [ES_ONAP] > loaded plugin [x-pack] > > [2018-10-17T13:20:28,981][INFO ][o.e.d.DiscoveryModule ] [ES_ONAP] > using discovery type [zen] > > [2018-10-17T13:20:30,228][INFO ][o.e.n.Node ] [ES_ONAP] > initialized > > [2018-10-17T13:20:30,229][INFO ][o.e.n.Node ] [ES_ONAP] > starting ... > > [2018-10-17T13:20:30,520][INFO ][o.e.t.TransportService ] [ES_ONAP] > publish_address {127.0.0.1:8443 > <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8443&d=DwMGaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=5PWCtn_zD8jT6xuuwP7DpGwXiA7-u1RBXx8E9Vrv2NI&m=afk3qptXbsXCXwnbkUPflsF2JY0_3IwzwHnt5X0mMQA&s=TjiPBnS3R_FNcrEf-gCBo3wxP9XwYY8ZF8lvu7LyDtI&e=>}, > bound_addresses {[::]:8443} > > [2018-10-17T13:20:30,557][INFO ][o.e.b.BootstrapChecks ] [ES_ONAP] > bound or publishing to a non-loopback address, enforcing bootstrap checks > > ERROR: [1] bootstrap checks failed > > [1]: max virtual memory areas vm.max_map_count [65530] is too low, > increase to at least [262144] > > [2018-10-17T13:20:30,574][INFO ][o.e.n.Node ] [ES_ONAP] > stopping ... > > [2018-10-17T13:20:30,729][INFO ][o.e.n.Node ] [ES_ONAP] > stopped > > [2018-10-17T13:20:30,729][INFO ][o.e.n.Node ] [ES_ONAP] > closing ... > > [2018-10-17T13:20:30,755][INFO ][o.e.n.Node ] [ES_ONAP] > closed > > > > Thank you in advance! > > > > Best Regards > > Sirisha Gopigiri > > > > *“Amdocs’ email platform is based on a third-party, worldwide, cloud-based > system. Any emails sent to Amdocs will be processed and stored using such > system and are accessible by third party providers of such system on a > limited basis. Your sending of emails to Amdocs evidences your consent to > the use of such system and such processing, storing and access”.* > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13162): https://lists.onap.org/g/onap-discuss/message/13162 Mute This Topic: https://lists.onap.org/mt/27465698/21656 Group Owner: onap-discuss+ow...@lists.onap.org Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
