Could you please share your Tomcat web server configuration?
Deepak "The greatness of a nation can be judged by the way its animals are treated - Mahatma Gandhi" +91 73500 12833 [email protected] Facebook: https://www.facebook.com/deicool LinkedIn: www.linkedin.com/in/deicool "Plant a Tree, Go Green" Make In India : http://www.makeinindia.com/home On Mon, May 30, 2022 at 4:56 PM Clemens WYSS (Helbling Technik) < [email protected]> wrote: > > Are the JVM parameters good enough? > Non search-related http requests run smoothly as long as not all AJPs are > blocked waiting for Solr-resonses 😉 > > -----Ursprüngliche Nachricht----- > Von: Deepak Goel <[email protected]> > Gesendet: Montag, 30. Mai 2022 13:16 > An: [email protected] > Betreff: Re: SolrJ connection handling, best practice(s)? > > Could you please share your Tomcat web server configuration? Are the JVM > parameters good enough? > > Deepak > "The greatness of a nation can be judged by the way its animals are treated > - Mahatma Gandhi" > > +91 73500 12833 > [email protected] > > Facebook: > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2Fdeicool&data=05%7C01%7CClemens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b0f0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DMh0j1aYYos3wMw9Dcgfh46yHIU5HYjklrmX4TSFPt0%3D&reserved=0 > LinkedIn: > https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdeicool&data=05%7C01%7CClemens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b0f0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=xoALkzrXnivcwVxZdBx3pTXCc%2BJShGEkeSSRvlj%2BU3M%3D&reserved=0 > > "Plant a Tree, Go Green" > > Make In India : > https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.makeinindia.com%2Fhome&data=05%7C01%7CClemens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b0f0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=xLgg5Q8i85NNCe7M2sC7Bd9y8C1MhuGubxt2jAaOg0U%3D&reserved=0 > > > On Mon, May 30, 2022 at 4:41 PM Clemens WYSS (Helbling Technik) < > [email protected]> wrote: > > > Ciao Vincenzo, > > > Why do you think that this problem is caused by the wrong usage of > > >the clients? > > * during the "peak" the solr server itself was not really "under > pressure" > > * stressing the solr server with k6 showed that the Solr-Server(setup) > > can handle the amount of expected concurrent requests > > > > So to me it's either within tomcat, i.e. the HttpClient "connection > > handling" (e.g. MaxConnections, MaxConnectionsPerHost, ...), or the > > "tomcat host server setup" (limit of outgoing connections, ephemeral > > ports, ulimit?)? > > > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhc.a > > pache.org%2Fhttpclient-legacy%2Fperformance.html&data=05%7C01%7CCl > > emens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b > > 0f0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFp > > bGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn > > 0%3D%7C3000%7C%7C%7C&sdata=LAbOw8pr0D7Ut2kVm%2F%2FZMjXIrcQMS3toLZ5 > > 4tTHNKcI%3D&reserved=0 Does HttpSolrClient make use of > > MultiThreadedHttpConnectionManager? > > > > https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhc.a > > pache.org%2Fhttpclient-legacy%2Fthreading.html&data=05%7C01%7CClem > > ens.Wyss%40helbling.ch%7Ccaf45aa7c3294dbc7b3708da422dd127%7Cbcbab93b0f > > 0d4747b4a4bc57e6753bb4%7C0%7C0%7C637895061793187418%7CUnknown%7CTWFpbG > > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0% > > 3D%7C3000%7C%7C%7C&sdata=iTNWeMzrY2jHzUS6ZXCvXBl9zanMTJ4jLpRXuDBEz > > LY%3D&reserved=0 Other things HttpSolrClient may need to consider > > > > Thx > > - Clemens > > -----Ursprüngliche Nachricht----- > > Von: Vincenzo D'Amore <[email protected]> > > Gesendet: Montag, 30. Mai 2022 09:12 > > An: [email protected] > > Betreff: Re: SolrJ connection handling, best practice(s)? > > > > Hi Clemens, Solrj clients are thread-safe singletons, from my > > experience the strategy you are using works well even under high load, > > when calls to Solr take seconds to answer usually it happens because > > of the number of the queries, the kind of queries (grouping or too > > many facets) or even for garbage collection problems. > > Why do you think that this problem is caused by the wrong usage of the > > clients? > > > > On Mon, May 30, 2022 at 8:30 AM Clemens WYSS (Helbling Technik) < > > [email protected]> wrote: > > > > > We have a non clustered single (8.x) Solr server setup and use SolrJ > > > to connect our tomcat to the Solr server. Under high load (e.g. last > > > black > > > friday) we face a bottleneck due to our webapp-search-requests. > > > Calls to solr take seconds, dangling http-connecitons, ... > > > Firing the same ( or even way more ) search requests directly onto > > > our Solr server showed, as expected, no problem. Therefore my > > > implication is that SolrJ/HttpSolrClient/HttpClient is not setup as > expected. > > > Our search API is setup alike: > > > > > > private final Map<String, SolrClient> coreConnections; // one > > > SolrClient per core ... > > > > > > // create a new SolrClient for a core if not existant > > > HttpSolrClient.Builder builder = new HttpSolrClient.Builder( coreUrl > > > ); solrClient = builder.build(); coreConnections.put( key, > > > solrClient ); > > > > > > As you can see > > > * we keep one SolrClient-connection per core in a map and make use > > > of these connections > > > * making use of HttpSolrClient ( Http2SolrClient is denoted > > "experimental" > > > and has cause some problem ) > > > * no special tweaking of underneath HttpClient > > > > > > Any reason for this minimal setup not being able to stem many many > > > concurrent requests? What is the proposed SolrJ based solr client > > > setup (for a single Solr server)? > > > > > > Thx > > > - Clemens > > > > > > > > > -- > > Vincenzo D'Amore > > >
