These are really good metrics for me: You say that RAM size should be at least index size, and it is better to have a RAM size twice the index size (because of worst case scenario).
On the other hand let's assume that I have a RAM size that is bigger than twice of indexes at machine. Can Solr use that extra RAM or is it a approximately maximum limit (to have twice size of indexes at machine)? 2013/4/10 Shawn Heisey <s...@elyograg.org> > On 4/9/2013 4:06 PM, Furkan KAMACI wrote: > >> Are there anybody who can help me about how to guess the approximately >> needed RAM for 5000 query/second at a Solr machine? >> > > You've already gotten some good replies, and I'm aware that they haven't > really answered your question. This is the kind of question that cannot be > answered. > > The amount of RAM that you'll need for extreme performance actually isn't > hard to figure out - you need enough free RAM for the OS to cache the > maximum amount of disk space all your indexes will ever use. Normally this > will be twice the size of all the indexes on the machine, because that's > how much disk space will likely be used in a worst-case merge scenario > (optimize). That's very expensive, so it is cheaper to budget for only the > size of the index. > > A load of 5000 queries per second is pretty high, and probably something > you will not achieve with a single-server (not counting backup) approach. > All of the tricks that high-volume website developers use are also > applicable to Solr. > > Once you have enough RAM, you need to worry more about the number of > servers, the number of CPU cores in each server, and the speed of those CPU > cores. Testing with actual production queries is the only way to find out > what you really need. > > Beyond hardware design, making the requests as simple as possible and > taking advantage of caches is important. Solr has caches for queries, > filters, and documents. You can also put a caching proxy (something like > Varnish) in front of Solr, but that would make NRT updates pretty much > impossible, and that kind of caching can be difficult to get working right. > > Thanks, > Shawn > >
