Hello Deepak, Not in-between.
Thanks, Markus -----Original message----- > From:Deepak Goel <deic...@gmail.com> > Sent: Friday 8th June 2018 17:14 > To: solr-user@lucene.apache.org > Subject: Re: 7.3.1 creates thousands of threads after start up > > Do these machines have a firewall in-between? > > On Fri, 8 Jun 2018, 20:29 Markus Jelsma, <markus.jel...@openindex.io> wrote: > > > Hello Shawn, > > > > The logs appear useless, they are littered with these: > > > > 2018-06-08 14:02:47.382 ERROR (qtp1458849419-1263) [ ] > > o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: Error > > trying to proxy request for url: http://idx2:8983/solr/ > > search/admin/ping <http://idx2:8983/solr/search/admin/ping> > > at > > org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:647) > > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:501) > > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384) > > .. > > at > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) > > at > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: org.eclipse.jetty.io.EofException > > at > > org.eclipse.jetty.server.HttpConnection$SendCallback.reset(HttpConnection.java:704) > > .. > > at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:509) > > > > Regarding the versions, it is a bit hard to recall but i do no thing i > > have seen this on 7.2, most certainly not on 7.1. > > > > We operate three distinct type of Solr collections, they only share the > > same Zookeeper quorum. The other two collections do not seem to have this > > problem, but i don't restart those as often as i restart this collection, > > as i am STILL trying to REPRODUCE the dreaded memory leak i reported having > > on 7.3 about two weeks ago. Sorry, but i drives me nuts! > > > > Thanks, > > Markus > > > > -----Original message----- > > > From:Shawn Heisey <apa...@elyograg.org> > > > Sent: Friday 8th June 2018 16:47 > > > To: solr-user@lucene.apache.org > > > Subject: Re: 7.3.1 creates thousands of threads after start up > > > > > > On 6/8/2018 8:17 AM, Markus Jelsma wrote: > > > > Our local test environment mini cluster goes nuts right after start > > up. It is a two node/shard/replica collection starts up normally if only > > one node start up. But as soon as the second node attempts to join the > > cluster, both nodes go crazy, creating thousands of threads with identical > > stack traces. > > > > > > > > "qtp1458849419-4738" - Thread t@4738 > > > > java.lang.Thread.State: TIMED_WAITING > > > > at sun.misc.Unsafe.park(Native Method) > > > > - parking to wait for <6ee32168> (a > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > > > > at > > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) > > > > at > > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) > > > > at > > org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) > > > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) > > > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) > > > > at > > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) > > > > at java.lang.Thread.run(Thread.java:748) > > > > > > > > Locked ownable synchronizers: > > > > - None > > > > > > > > If does not happen always, but most of the time i am unable to boot > > the cluster normally. Sometimes, apparently right now for the first time, > > the GUI is still accessible. > > > > > > > > Is this a known issue? > > > > > > It's not a problem that I've heard of. There are no Solr classes in the > > > stacktrace, only Jetty and Java classes. I won't try to tell you that a > > > bug in Solr can't be the root cause, because it definitely can. The > > > threads appear to be created by Jetty, but the supplied info doesn't > > > indicate WHY it's happening. > > > > > > Presumably there's a previous version you've used where this problem did > > > NOT happen. What version would that be? > > > > > > Can you share the solr.log file from both nodes when this happens? > > > There might be a clue there. > > > > > > It sounds like you probably have a small number of collections in the > > > dev cluster. Can you confirm that? > > > > > > Thanks, > > > Shawn > > > > > > > > >
