Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Shawn Heisey
On 5/26/2017 2:21 PM, Rick Leir wrote: > (off topic) Strange. Your java has a resident size much lower than its > virtual size, so I assumed it was mostly in swap. I suppose that the > virtual size could be mostly lots of readonly or unmodified pages, but > that seems unlikely. This suggests that y

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Rick Leir
(off topic) Strange. Your java has a resident size much lower than its virtual size, so I assumed it was mostly in swap. I suppose that the virtual size could be mostly lots of readonly or unmodified pages, but that seems unlikely. This suggests that you are using some swap for java pages. But I

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Robert Brown
Thanks Rick, Swap is actually turned off, but reducing the number of Perl processes is a quick win. On 26/05/17 17:06, Rick Leir wrote: Robert, Cool, perl is taking most of your memory. 12 fcgi processes, at about 8% memory each. Try changing the web server config so it just forks 2 or 4 o

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Rick Leir
Robert, Cool, perl is taking most of your memory. 12 fcgi processes, at about 8% memory each. Try changing the web server config so it just forks 2 or 4 of them. And check whether your swap device is working. With a working swap disk, maybe your system would just slow down instead of crashing. N

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Robert Brown
Thanks Shawn, It's more inquisitiveness now more than anything. http://web.lavoco.com/top.png (forgot to mention mariadb on there too :) On 26/05/17 16:20, Shawn Heisey wrote: On 5/26/2017 11:01 AM, Robert Brown wrote: Let's assume I can't get more RAM - why would an index of no more than

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Shawn Heisey
On 5/26/2017 11:01 AM, Robert Brown wrote: > Let's assume I can't get more RAM - why would an index of no more than > 1MB (on disk) need so much? > > (without getting into why I'm using Solr on such a small index in the > first place :) > > My docs consist of 3 text fields for searching, all other

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Robert Brown
Let's assume I can't get more RAM - why would an index of no more than 1MB (on disk) need so much? (without getting into why I'm using Solr on such a small index in the first place :) My docs consist of 3 text fields for searching, all others are strings/ints for facets and filtering, about

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Erick Erickson
Or get more physical memory? Solr _likes_ memory, you won't be able to do much with only 2G physical memory.. On Fri, May 26, 2017 at 2:00 AM, Robert Brown wrote: > Thanks Rick, > > Turns out it was the kernel killing it, dmesg showed: > > Out of memory: Kill process 2647 (java) score 118 or

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Robert Brown
Thanks Rick, Turns out it was the kernel killing it, dmesg showed: Out of memory: Kill process 2647 (java) score 118 or sacrifice child Killed process 2647, UID 1006, (java) total-vm:2857484kB, anon-rss:227440kB, file-rss:12kB Now I just need to tell the kernel not to do that. The other thin

Re: (Tiny Index) Solr dies but not OOM

2017-05-26 Thread Rick Leir
Robert, What is at the end of solr.log when it has died? Is there anything in syslog or messages? What is the other app? Run the top command, memory screen, on Ubuntu: $ top -o RES I have never used strace(1) on Solr, but that is an option. Run Solr in strace with the appropriate options to

(Tiny Index) Solr dies but not OOM

2017-05-25 Thread Robert Brown
Hi, I'm currently running 6.5.1 with a tiny index, less than 1MB. When I restart another app on the same server as Solr, Solr occasionally dies, but no solr_oom_killer.log file. Heap size is 256MB (~30MB used), Physical RAM 2GB, typically using 1.5GB. How else can I debug what's causing it?