Re: Max http connections in CloudSolrServer
On 18-Apr-2013, at 9:43 PM, Shawn Heisey wrote: > Are you using the Jetty included with Solr, or a Jetty installed separately? I am using the Jetty that comes with Solr. > The Jetty included with Solr has a maxThreads value of 1 in its config. > The default would be closer to 200, and a single request from a Cloud client > likely uses multiple Jetty threads. The default maxThreads is 1 and minThreads is 10. ./zahoor
Re: Max http connections in CloudSolrServer
On 4/18/2013 6:42 AM, J Mohamed Zahoor wrote: I dont yet know if this is the reason... I am looking if jetty has some limit on accepting connections.. Are you using the Jetty included with Solr, or a Jetty installed separately? The Jetty included with Solr has a maxThreads value of 1 in its config. The default would be closer to 200, and a single request from a Cloud client likely uses multiple Jetty threads. Thanks, Shawn
Re: Max http connections in CloudSolrServer
I dont yet know if this is the reason...
I am looking if jetty has some limit on accepting connections..
./zahoor
On 18-Apr-2013, at 12:52 PM, J Mohamed Zahoor wrote:
>
> Thanks for this.
> The reason i asked this was.. when i fire 30 queries simultaneously from 30
> threads using the same CloudSolrServer instance,
> some queries gets fired after a delay.. sometime the delay is 30-50 seconds...
>
> In solr logs i can see.. 20+ queries get fired almost immediately... but some
> of them gets fired late..
>
> i increased the connections per host from 32 to 200.. still no respite...
>
> ./zahoor
>
> On 18-Apr-2013, at 12:20 AM, Shawn Heisey wrote:
>
>> ModifiableSolrParams params = new ModifiableSolrParams();
>> params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 1000);
>> params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 200);
>> HttpClient client = HttpClientUtil.createClient(params);
>> LBHttpSolrServer lbServer = new LBHttpSolrServer
>>(client, "http://localhost/solr";);
>> lbServer.removeSolrServer("http://localhost/solr";);
>> SolrServer server = new CloudSolrServer(zkHost, lbServer);
>
Re: Max http connections in CloudSolrServer
Thanks for this.
The reason i asked this was.. when i fire 30 queries simultaneously from 30
threads using the same CloudSolrServer instance,
some queries gets fired after a delay.. sometime the delay is 30-50 seconds...
In solr logs i can see.. 20+ queries get fired almost immediately... but some
of them gets fired late..
i increased the connections per host from 32 to 200.. still no respite...
./zahoor
On 18-Apr-2013, at 12:20 AM, Shawn Heisey wrote:
> ModifiableSolrParams params = new ModifiableSolrParams();
> params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 1000);
> params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 200);
> HttpClient client = HttpClientUtil.createClient(params);
> LBHttpSolrServer lbServer = new LBHttpSolrServer
>(client, "http://localhost/solr";);
> lbServer.removeSolrServer("http://localhost/solr";);
> SolrServer server = new CloudSolrServer(zkHost, lbServer);
Re: Max http connections in CloudSolrServer
On 4/17/2013 3:21 PM, Chris Hostetter wrote: I think the name comes from the effect it has on the underlying HttpClient code ... it's possible to configure a HttpConnectionManager such that it has different number of max connections per host -- ie: host1 has max connections of 23, host2 has max connections of 45, etc i believe that method just changes the "default" when there isn't something specifically set for an individual host.. That puts it into complete perspective, so changing the name is a bad idea. I do think it might be a good idea to include HttpSolrServer's convenience methods in the other classes, even if there is a working HttpClient -> LBHttpSolrServer -> CloudSolrServer way to change things. Thanks, Shawn
Re: Max http connections in CloudSolrServer
: Side issue: shouldn't that be setMaxConnectionsPerHost instead of including : the word Default? If there's no objection, I would plan on adding the renamed : method and using a typical deprecation procedure for the old one. I think the name comes from the effect it has on the underlying HttpClient code ... it's possible to configure a HttpConnectionManager such that it has different number of max connections per host -- ie: host1 has max connections of 23, host2 has max connections of 45, etc i believe that method just changes the "default" when there isn't something specifically set for an individual host.. -Hoss
Re: Max http connections in CloudSolrServer
On 4/17/2013 3:46 AM, J Mohamed Zahoor wrote:
Hi
I am pumping parallel select queries using CloudSolrServer.
It looks like it can handle only certain no of max connections...
my Question is,
How many concurrent queries does a CloudSolrServer can handle?
Looking into the code for 4.x versions, I found that the default max
number of connections is 128, and the default max number of connections
per host is 32. This is set in the HttpSolrServer constructor.
An old thread tries to answer this by asking to give our own instance of
LBHttpSolrServer...
But it looks like there is no way from LBHttpSolrServer to up the maxConnection
of the httpClient it has...
Can someone let me know how to bump up the maxConnections and
maxConnectionsPerHost parameter for the httpCLient used by cloudSolrServer?
You should be able to create an instance of LBHttpSolrServer, which in
turn lets you use a custom HttpClient, where you could set the
connection limits. Then you could use the LBHttpSolrServer object to
create an instance of CloudSolrServer.
The following code MIGHT work (probably in a try block), but I haven't
tried to actually use it, so it might be horribly broken.
ModifiableSolrParams params = new ModifiableSolrParams();
params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 1000);
params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 200);
HttpClient client = HttpClientUtil.createClient(params);
LBHttpSolrServer lbServer = new LBHttpSolrServer
(client, "http://localhost/solr";);
lbServer.removeSolrServer("http://localhost/solr";);
SolrServer server = new CloudSolrServer(zkHost, lbServer);
I would argue that CloudSolrServer (and therefore LBHttpSolrServer)
should have many of the setters available on HttpSolrServer, including
setDefaultMaxConnectionsPerHost and setMaxTotalConnections. Perhaps not
all of them, some of them are things that most people would never really
need, and some of them might not make sense for a clustered server object.
Side issue: shouldn't that be setMaxConnectionsPerHost instead of
including the word Default? If there's no objection, I would plan on
adding the renamed method and using a typical deprecation procedure for
the old one.
Thanks,
Shawn
Max http connections in CloudSolrServer
Hi I am pumping parallel select queries using CloudSolrServer. It looks like it can handle only certain no of max connections... my Question is, How many concurrent queries does a CloudSolrServer can handle? An old thread tries to answer this by asking to give our own instance of LBHttpSolrServer... But it looks like there is no way from LBHttpSolrServer to up the maxConnection of the httpClient it has... Can someone let me know how to bump up the maxConnections and maxConnectionsPerHost parameter for the httpCLient used by cloudSolrServer? ./zahoor
