Fyi: the CommonsHttpSolrServer already has method to do all of those
things:
/** set connectionTimeout on the underlying
MultiThreadedHttpConnectionManager */
public void setConnectionTimeout(int timeout) {
_connectionManager.getParams().setConnectionTimeout(timeout);
}
/** set maxConnectionsPerHost on the underlying
MultiThreadedHttpConnectionManager */
public void setDefaultMaxConnectionsPerHost(int connections) {
_connectionManager.getParams().setDefaultMaxConnectionsPerHost(connections);
}
/** set maxTotalConnection on the underlying
MultiThreadedHttpConnectionManager */
public void setMaxTotalConnections(int connections) {
_connectionManager.getParams().setMaxTotalConnections(connections);
}
You can also get the underlying connection factory if you want to do other
crazier stuff.
public MultiThreadedHttpConnectionManager getConnectionManager() {
return _connectionManager;
}
- will
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Yonik Seeley
Sent: Monday, December 17, 2007 9:18 PM
To: [email protected]
Subject: Re: Resource contention problem in Solrj
Excellent! Thanks for diagnosing this!
-Yonik
On Dec 17, 2007 9:00 PM, climbingrose <[EMAIL PROTECTED]> wrote:
> There seems to be resource contention problem with Solrj under load. To
> reproduce the problem: set up a sample webapp with solrj connect to a HTTP
> Solr instance and hammer the webapp with Apache ab (say 10 concurrent
> connection with 100 requests). You'll notice that the webapp's servlet
> container quickly consumes 100% CPU and stays there unless you restart it.
I
> can confirm that this happens with both Tomcat and Jetty. Meanwhile, the
> server that Solr is deployed on seems to be running fine.
>
> From this observation, I suspect that Solrj has connection contention
> problem. And this seems to be the case if you look at
CommonHttpSolrServer.
> This class uses MultiThreadedHttpConnectionManager which has
> maxConnectionsPerHost set to 2 by default. When the number of thread
> increases, this is obviously not enough and leads to connection contention
> problem. I quickly solve problem by adding another constructor to
> CommonHttpSolrServer that allows setting maxConnectionsPerHost and
> maxTotalConnections:
>
> public CommonsHttpSolrServer(int maxConsPerHost, int maxTotalCons, String
> solrServerUrl) throws MalformedURLException {
> this(solrServerUrl);
> this.maxConsPerHost = maxConsPerHost;
> this.maxTotalCons = maxTotalCons;
> HttpConnectionManagerParams params = new
HttpConnectionManagerParams();
> params.setDefaultMaxConnectionsPerHost(maxConsPerHost);
> params.setMaxTotalConnections(maxTotalCons);
> _connectionManager.setParams(params);
> }
>
> Hope this information would help others.
>
> --
> Regards,
>
> Cuong Hoang
>
