Could you please share your Tomcat web server configuration? Are the JVM parameters good enough?
Deepak "The greatness of a nation can be judged by the way its animals are treated - Mahatma Gandhi" +91 73500 12833 [email protected] Facebook: https://www.facebook.com/deicool LinkedIn: www.linkedin.com/in/deicool "Plant a Tree, Go Green" Make In India : http://www.makeinindia.com/home On Mon, May 30, 2022 at 4:41 PM Clemens WYSS (Helbling Technik) < [email protected]> wrote: > Ciao Vincenzo, > > Why do you think that this problem is caused > >by the wrong usage of the clients? > * during the "peak" the solr server itself was not really "under pressure" > * stressing the solr server with k6 showed that the Solr-Server(setup) can > handle the amount of expected concurrent requests > > So to me it's either within tomcat, i.e. the HttpClient "connection > handling" (e.g. MaxConnections, MaxConnectionsPerHost, ...), or the "tomcat > host server setup" (limit of outgoing connections, ephemeral ports, > ulimit?)? > > https://hc.apache.org/httpclient-legacy/performance.html > Does HttpSolrClient make use of MultiThreadedHttpConnectionManager? > > https://hc.apache.org/httpclient-legacy/threading.html > Other things HttpSolrClient may need to consider > > Thx > - Clemens > -----Ursprüngliche Nachricht----- > Von: Vincenzo D'Amore <[email protected]> > Gesendet: Montag, 30. Mai 2022 09:12 > An: [email protected] > Betreff: Re: SolrJ connection handling, best practice(s)? > > Hi Clemens, Solrj clients are thread-safe singletons, from my experience > the strategy you are using works well even under high load, when calls to > Solr take seconds to answer usually it happens because of the number of the > queries, the kind of queries (grouping or too many facets) or even for > garbage collection problems. > Why do you think that this problem is caused by the wrong usage of the > clients? > > On Mon, May 30, 2022 at 8:30 AM Clemens WYSS (Helbling Technik) < > [email protected]> wrote: > > > We have a non clustered single (8.x) Solr server setup and use SolrJ > > to connect our tomcat to the Solr server. Under high load (e.g. last > > black > > friday) we face a bottleneck due to our webapp-search-requests. Calls > > to solr take seconds, dangling http-connecitons, ... > > Firing the same ( or even way more ) search requests directly onto our > > Solr server showed, as expected, no problem. Therefore my implication > > is that SolrJ/HttpSolrClient/HttpClient is not setup as expected. > > Our search API is setup alike: > > > > private final Map<String, SolrClient> coreConnections; // one > > SolrClient per core ... > > > > // create a new SolrClient for a core if not existant > > HttpSolrClient.Builder builder = new HttpSolrClient.Builder( coreUrl > > ); solrClient = builder.build(); coreConnections.put( key, solrClient > > ); > > > > As you can see > > * we keep one SolrClient-connection per core in a map and make use of > > these connections > > * making use of HttpSolrClient ( Http2SolrClient is denoted > "experimental" > > and has cause some problem ) > > * no special tweaking of underneath HttpClient > > > > Any reason for this minimal setup not being able to stem many many > > concurrent requests? What is the proposed SolrJ based solr client > > setup (for a single Solr server)? > > > > Thx > > - Clemens > > > > > -- > Vincenzo D'Amore >
