I think it is not fair blaiming Solr not also having a load balancer.
It is up to you and your needs to set up the required infrastucture
including load balancing. The are many products available on the market.
If your current system can't handle all requests then install more replicas.
Regards
Bernd
Am 27.05.19 um 10:33 schrieb Joe Doupnik:
While on the topic of resource consumption and locks etc, there is one other aspect to which Solr has been vulnerable. It is failing to
fend off too many requests at one time. The standard approach is, of course, named back pressure, such as not replying to a query until
resources permit and thus keeping competion outside of the application. That limits resource consumption, including locks, memory and sundry,
while permiting normal work within to progress smoothly. Let the crowds coming to a hit show queue in the rain outside the theatre until empty
seats become available.
On 27/05/2019 08:52, Joe Doupnik wrote:
Generalizations tend to fail when confronted with conflicting evidence. The simple evidence is asking how much real memory the Solr owned
process has been allocated (top, or ps aux or similar) and that yields two very different values (the ~1.6GB of Solr v8.0 and 4.5+GB of Solr
v8.1). I have no knowledge of how Java chooses to name its usage (heap or otherwise). Prior to v8.1 Solr memory consumption varied with
activity, thus memory management was occuring, memory was borrowed from and returned to the system. What might be happening in Solr v8.1 is
the new memory management code is failing to do a proper job, for reasons which are not visible to us in the field, and that failure is
important to us.
In regard to the referenced lock discussion, it would be a good idea to not let the tail wag the dog, tend the common cases and live with
a few corner case difficulties because perfection is not possible.
Thanks,
Joe D.
On 26/05/2019 20:30, Shawn Heisey wrote:
On 5/26/2019 12:52 PM, Joe Doupnik wrote:
I do queries while indexing, have done so for a long time, without difficulty nor memory usage spikes from dual use. The system has
been designed to support that.
Again, one may look at the numbers using "top" or similar. Try Solr v8.0 and 8.1 to see the difference which I experience here. For
reference, the only memory adjustables set in my configuration is in the Solr startup script solr.in.sh saying add "-Xss1024k" in the
SOLR_OPTS list and setting SOLR_HEAP="4024m".
There is one significant difference between 8.0 and 8.1 in the realm of memory management -- we have switched from the CMS garbage collector
to the G1 collector. So the way that Java manages the heap has changed. This was done because the CMS collector is slated for removal from
Java.
https://issues.apache.org/jira/browse/SOLR-13394
Java is unlike other programs in one respect -- once it allocates heap from the OS, it never gives it back. This behavior has given Java an
undeserved reputation as a memory hog ... but in fact Java's overall memory usage can be very easily limited ... an option that many other
programs do NOT have.
In your configuration, you set the max heap to a little less than 4GB. You have to expect that it *WILL* use that memory. By using the
SOLR_HEAP variable, you have instructed Solr's startup script to use the same setting for the minimum heap as well as the maximum heap. This
is the design intent.
If you want to know how much heap is being used, you can't ask the operating system, which means tools like top. You have to ask Java. And
you will have to look at a long-term graph, finding the low points. An instananeous look at Java's heap usage could show you that the whole
heap is allocated ... but a significant part of that allocation could be garbage, which becomes available once the garbage is collected.
Thanks,
Shawn
