Re: Response time under 1 second?
On 2/22/2018 10:45 AM, LOPEZ-CORTES Mariano-ext wrote: For the moment, I have the following information: 12GB is max java heap. Total memory i don't know. No direct access to host. 2 replicas = Size 1 = 11.51 GB Size 2 = 11.82 GB (Sizes showed in the Core-Overview admin gui) OK, so you have about 23GB of total index data on the machine. With a 12GB heap, and assuming there's no other software running on the machine, then for good performance I would want to have at least 32GB total memory, which leaves around 20GB for the OS to cache the 23GB index. More memory would be better, but probably isn't a requirement. If there is other software running on the machine, then that will increase the total memory requirement. It is always possible that your Solr install is in a situation where 12GB of heap is actually not quite big enough. If that happens, performance will usually be a lot worse than in situations where the left-over memory is not enough for the OS to cache the index properly. You might be able to get decent performance if the total memory is about 24GB, but that much might NOT be enough. There are a lot of factors affecting actual memory requirements. The Solr admin UI will tell you what the total physical memory in the system is, on the dashboard. It will be the upper right graph. Note that this graph is likely to show 100% or nearly 100% full. Don't let this alarm you -- it's normal. How did you arrive at the 12GB size for your heap? Have you tried reducing this number so that there is more memory left for the OS to handle disk caching? I have no idea whether your Solr install will still work properly with a smaller heap, so be aware that reducing the heap might cause more problems. https://wiki.apache.org/solr/SolrPerformanceProblems#RAM Thanks, Shawn
RE: Response time under 1 second?
For the moment, I have the following information: 12GB is max java heap. Total memory i don't know. No direct access to host. 2 replicas = Size 1 = 11.51 GB Size 2 = 11.82 GB (Sizes showed in the Core-Overview admin gui) Thanks very much! -Message d'origine- De : Shawn Heisey [mailto:elyog...@elyograg.org] Envoyé : jeudi 22 février 2018 17:06 À : solr-user@lucene.apache.org Objet : Re: Response time under 1 second? On 2/22/2018 8:53 AM, LOPEZ-CORTES Mariano-ext wrote: > With a 3 nodes cluster each 12GB and a corpus of 5GB (CSV format). > > Is it better to disable completely Solr cache ? There is enough RAM for the > entire index. The size of the input data will have an effect on how big the index is, but it is not a direct indication of the index size. The size of the index is more important than the size of the data that you send to Solr to create the index. You say 12GB ... but is this total system memory, or the max Java heap size for Solr? What are these two numbers for your servers? If you go to the admin UI for one of these servers and look at the Overview page for all of the index cores it contains, you will be able to see how many documents and what size each index is on disk. What are these numbers? If the numbers are similar for all the servers, then I will only need to see it for one of them. If the machine is running an OS like Linux that has the gnu top program, then I can see a lot of useful information from that program. Run "top" (not htop or other variants), press shift-M to sort the list by memory, and grab a screenshot. This will probably be an image file, so you'll need to find a file sharing site and give us a URL to access the file. Attachments rarely make it to the mailing list. Thanks, Shawn
Re: Response time under 1 second?
On 2/22/2018 8:53 AM, LOPEZ-CORTES Mariano-ext wrote: With a 3 nodes cluster each 12GB and a corpus of 5GB (CSV format). Is it better to disable completely Solr cache ? There is enough RAM for the entire index. The size of the input data will have an effect on how big the index is, but it is not a direct indication of the index size. The size of the index is more important than the size of the data that you send to Solr to create the index. You say 12GB ... but is this total system memory, or the max Java heap size for Solr? What are these two numbers for your servers? If you go to the admin UI for one of these servers and look at the Overview page for all of the index cores it contains, you will be able to see how many documents and what size each index is on disk. What are these numbers? If the numbers are similar for all the servers, then I will only need to see it for one of them. If the machine is running an OS like Linux that has the gnu top program, then I can see a lot of useful information from that program. Run "top" (not htop or other variants), press shift-M to sort the list by memory, and grab a screenshot. This will probably be an image file, so you'll need to find a file sharing site and give us a URL to access the file. Attachments rarely make it to the mailing list. Thanks, Shawn
