sometimes just throwing money/ram/ssd at the problem is just the best answer.
On Mon, Jun 29, 2020 at 11:38 AM Ryan W <rya...@gmail.com> wrote: > Thanks everyone. Just to give an update on this issue, I bumped the RAM > available to Solr up to 16GB a couple weeks ago, and haven’t had any > problem since. > > > On Tue, Jun 16, 2020 at 1:00 PM David Hastings < > hastings.recurs...@gmail.com> > wrote: > > > me personally, around 290gb. as much as we could shove into them > > > > On Tue, Jun 16, 2020 at 12:44 PM Erick Erickson <erickerick...@gmail.com > > > > wrote: > > > > > How much physical RAM? A rule of thumb is that you should allocate no > > more > > > than 25-50 percent of the total physical RAM to Solr. That's > cumulative, > > > i.e. the sum of the heap allocations across all your JVMs should be > below > > > that percentage. See Uwe Schindler's mmapdirectiry blog... > > > > > > Shot in the dark... > > > > > > On Tue, Jun 16, 2020, 11:51 David Hastings < > hastings.recurs...@gmail.com > > > > > > wrote: > > > > > > > To add to this, i generally have solr start with this: > > > > -Xms31000m-Xmx31000m > > > > > > > > and the only other thing that runs on them are maria db gallera > cluster > > > > nodes that are not in use (aside from replication) > > > > > > > > the 31gb is not an accident either, you dont want 32gb. > > > > > > > > > > > > On Tue, Jun 16, 2020 at 11:26 AM Shawn Heisey <apa...@elyograg.org> > > > wrote: > > > > > > > > > On 6/11/2020 11:52 AM, Ryan W wrote: > > > > > >> I will check "dmesg" first, to find out any hardware error > > message. > > > > > > > > > > <snip> > > > > > > > > > > > [1521232.781801] Out of memory: Kill process 117529 (httpd) > score 9 > > > or > > > > > > sacrifice child > > > > > > [1521232.782908] Killed process 117529 (httpd), UID 48, > > > > > total-vm:675824kB, > > > > > > anon-rss:181844kB, file-rss:0kB, shmem-rss:0kB > > > > > > > > > > > > Is this a relevant "Out of memory" message? Does this suggest an > > OOM > > > > > > situation is the culprit? > > > > > > > > > > Because this was in the "dmesg" output, it indicates that it is the > > > > > operating system killing programs because the *system* doesn't have > > any > > > > > memory left. It wasn't Java that did this, and it wasn't Solr that > > was > > > > > killed. It very well could have been Solr that was killed at > another > > > > > time, though. > > > > > > > > > > The process that it killed this time is named httpd ... which is > most > > > > > likely the Apache webserver. Because the UID is 48, this is > probably > > > an > > > > > OS derived from Redhat, where the "apache" user has UID and GID 48 > by > > > > > default. Apache with its default config can be VERY memory hungry > > when > > > > > it gets busy. > > > > > > > > > > > -XX:InitialHeapSize=536870912 -XX:MaxHeapSize=536870912 > > > > > > > > > > This says that you started Solr with the default 512MB heap. Which > > is > > > > > VERY VERY small. The default is small so that Solr will start on > > > > > virtually any hardware. Almost every user must increase the heap > > size. > > > > > And because the OS is killing processes, it is likely that the > system > > > > > does not have enough memory installed for what you have running on > > it. > > > > > > > > > > It is generally not a good idea to share the server hardware > between > > > > > Solr and other software, unless the system has a lot of spare > > > resources, > > > > > memory in particular. > > > > > > > > > > Thanks, > > > > > Shawn > > > > > > > > > > > > > > >
