Hello,
We have a SolrCloud(8.6) with 3 servers with the same characteristics and
configuration. We assigned 32GB for heap memory each, and after some short
period of time sending 40 concurrent requests to the SolrCloud using a load
balancer, we are getting the following error that shutdown each Solr Server and
Zookeeper:
OpenJDK 64-Bit Server VM warning: Failed to reserve large pages memory
req_addr: 0x0000000000000000 bytes: 536870912 (errno = 12).
OpenJDK 64-Bit Server VM warning: Attempt to deallocate stack guard pages
failed.
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007edd4d9da000,
12288, 0) failed; error='Cannot allocate memory' (errno=12)
20201201 10:43:29.495 [ERROR] {qtp2051853139-23369} [c:express s:shard1
r:core_node6 x:express_shard1_replica_n4]
[org.apache.solr.handler.RequestHandlerBase, 148] |
org.apache.solr.common.SolrException: Cannot talk to ZooKeeper - Updates are
disabled.
at
org.apache.solr.update.processor.DistributedZkUpdateProcessor.zkCheck(DistributedZkUpdateProcessor.java:1245)
at
org.apache.solr.update.processor.DistributedZkUpdateProcessor.setupRequest(DistributedZkUpdateProcessor.java:582)
at
org.apache.solr.update.processor.DistributedZkUpdateProcessor.processAdd(DistributedZkUpdateProcessor.java:239)
[cid:image004.jpg@01D6CF3F.27574B90]
We have a one collection with one shard, almost 400 million documents (~334GB).
$ sysctl vm.nr_hugepages
vm.nr_hugepages = 32768
$ sysctl vm.max_map_count
vm.max_map_count = 131072
/etc/security/limits.conf
* - core unlimited
* - data unlimited
* - priority unlimited
* - fsize unlimited
* - sigpending 513928
* - memlock unlimited
* - nofile 131072
* - msgqueue 819200
* - rtprio 0
* - stack 8192
* - cpu unlimited
* - rss unlimited #virtual memory unlimited
* - locks unlimited
* soft nproc 65536
* hard nproc 65536
* - nofile 131072
/etc/sysctl.conf
vm.nr_hugepages = 32768
vm.max_map_count = 131072
Could you please provide me some advice to fix this error?
Thanks,
Emmanuel Altamirano
