Guys, (@Erick & @Shawn), Thanks for the great suggestions!
I have increased Tomcat MaxThreads from 200 to 10000 on our staging environment. So far so good. I will perform some more indexing test and see how it goes. Many thanks, Tim -----Original Message----- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Monday, 8 August 2016 11:44 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory On 8/7/2016 6:53 PM, Tim Chen wrote: > Exception in thread "http-bio-8983-exec-6571" java.lang.OutOfMemoryError: > unable to create new native thread > at java.lang.Thread.start0(Native Method) > at java.lang.Thread.start(Thread.java:714) > at > java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949) > at > java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1017) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1163) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:745) I find myself chasing Erick once again. :) Supplementing what he told you: There are two things that might be happening here. 1) The Tomcat setting "maxThreads" may limiting the number of threads. This defaults to 200, and should be increased to 10000. The specific error doesn't sound like an application limit, though -- it acts more like Java itself can't create the thread. If you have already adjusted maxThreads, then it's more likely to be the second option: 2) The operating system may be imposing a limit on the number of processes/threads a user is allowed to start. On Linux systems, this is typically 1024. For other operating systems, I am not sure what the default limit is. Thanks, Shawn [Premiere League Starts Saturday 13 August 9.30pm on SBS]<http://theworldgame.sbs.com.au/>