On 4/9/2013 4:06 PM, Furkan KAMACI wrote:
Are there anybody who can help me about how to guess the approximately needed RAM for 5000 query/second at a Solr machine?
You've already gotten some good replies, and I'm aware that they haven't really answered your question. This is the kind of question that cannot be answered.
The amount of RAM that you'll need for extreme performance actually isn't hard to figure out - you need enough free RAM for the OS to cache the maximum amount of disk space all your indexes will ever use. Normally this will be twice the size of all the indexes on the machine, because that's how much disk space will likely be used in a worst-case merge scenario (optimize). That's very expensive, so it is cheaper to budget for only the size of the index.
A load of 5000 queries per second is pretty high, and probably something you will not achieve with a single-server (not counting backup) approach. All of the tricks that high-volume website developers use are also applicable to Solr.
Once you have enough RAM, you need to worry more about the number of servers, the number of CPU cores in each server, and the speed of those CPU cores. Testing with actual production queries is the only way to find out what you really need.
Beyond hardware design, making the requests as simple as possible and taking advantage of caches is important. Solr has caches for queries, filters, and documents. You can also put a caching proxy (something like Varnish) in front of Solr, but that would make NRT updates pretty much impossible, and that kind of caching can be difficult to get working right.
Thanks, Shawn
