Re: PropagateServer Implementation for Solr
Ok, in the scenario where the calling app uses SolrJ and creates a CloudSolrServer to send all its requests in. In that case, yes I can see the logic that says CloudSolrServer shouldn't load balance that (its not that type of request), it should forward it on to all the servers in the cloud. What will happen to the responses, do you get N (independent) responses back or do you plan to do some kind of aggregation? I confess we don't use SolrJ (our clients are C++), so we just manually send the request to all the servers in the cloud (will integrate with ZK when we work out that interface) so it would be nice if HTTP callers could do the same (maybe something like distrib=true|false on the LukeRequest as a shot in the dark, caller can request details from 1 server, or from the cloud as a whole?) Is there a way to send the Threads (/admin/threads) and stats requests (/admin/mbeans)? We also use them for monitoring (we can't deploy the web-based monitoring tools for various internal reasons which I won't bore you with!), but I can't see a request in SolrJ that would map to them? On 3 July 2013 22:08, Furkan KAMACI furkankam...@gmail.com wrote: Hi; I've written an e-mail at dev list and I want to share same e-mail here. I've opened two issues at Jira and I want to get feedback of community. First issue is: https://issues.apache.org/jira/browse/SOLR-4995 Currently Solr servers are interacting with only one Solr node. I think that there should be an implementation that propagates requests into multiple Solr nodes. For example when Solr is used as SolrCloud sending a LukeRequest should be made to one node at each shard. First patch will be related to implementing a PropagateServer for Solr. Second issue is related to first one: https://issues.apache.org/jira/browse/SOLR-4996 Let's assume that you are using Solr as SolrCloud and you have more than one shard. Let's assume that there are 20 docs at shard_1 and 15 docs at shard_2. When using CloudSolrServer if you make a LukeRequest it uses LBHttpSolrServer internally and it sends request to just one Solr Node (via HttpSolrServer) as round robin. So you may get 20 docs as a result at first request and if you send same request you may get 15 docs as a result too. Using a PropagateServer inside CloudSolrServer will fix that bug. I've made initial patchs for them and I will change/add code to them after getting feedback from community (i.e. first patch does not make multi threaded requests at PropagateServer, I just want to get feedbacks of community after that I will add other features) Thanks; Furkan KAMACI
Re: PropagateServer Implementation for Solr
Here is an example how I use PropagateServer inside CloudSolrServer:
public static ListCloudStatistics customListStatistics(CloudSolrServer
solrServer) {
NamedListObject namedList = new SimpleOrderedMapObject();
try {
namedList = solrServer.request(new LukeRequest());
} catch (SolrServerException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
ListNamedListObject all = (ListNamedListObject)
namedList.get(all);
ListCloudStatistics cloudStatisticsList = new
ArrayListCloudStatistics();
for (NamedListObject namedListSlice : all) {
cloudStatisticsList.add(new CloudStatistics((NamedListObject)
namedListSlice.get(index)));
}
return cloudStatisticsList;
}
PS: CloudStatistics is a class implemented by me and holds statistics
metrics.
2013/7/4 Daniel Collins danwcoll...@gmail.com
Ok, in the scenario where the calling app uses SolrJ and creates a
CloudSolrServer to send all its requests in. In that case, yes I can see
the logic that says CloudSolrServer shouldn't load balance that (its not
that type of request), it should forward it on to all the servers in the
cloud. What will happen to the responses, do you get N (independent)
responses back or do you plan to do some kind of aggregation?
I confess we don't use SolrJ (our clients are C++), so we just manually
send the request to all the servers in the cloud (will integrate with ZK
when we work out that interface) so it would be nice if HTTP callers could
do the same (maybe something like distrib=true|false on the LukeRequest
as a shot in the dark, caller can request details from 1 server, or from
the cloud as a whole?)
Is there a way to send the Threads (/admin/threads) and stats requests
(/admin/mbeans)? We also use them for monitoring (we can't deploy the
web-based monitoring tools for various internal reasons which I won't bore
you with!), but I can't see a request in SolrJ that would map to them?
On 3 July 2013 22:08, Furkan KAMACI furkankam...@gmail.com wrote:
Hi;
I've written an e-mail at dev list and I want to share same e-mail here.
I've opened two issues at Jira and I want to get feedback of community.
First issue is: https://issues.apache.org/jira/browse/SOLR-4995
Currently Solr servers are interacting with only one Solr node. I think
that there should be an implementation that propagates requests into
multiple Solr nodes. For example when Solr is used as SolrCloud sending a
LukeRequest should be made to one node at each shard. First patch will be
related to implementing a PropagateServer for Solr.
Second issue is related to first one:
https://issues.apache.org/jira/browse/SOLR-4996
Let's assume that you are using Solr as SolrCloud and you have more than
one shard. Let's assume that there are 20 docs at shard_1 and 15 docs at
shard_2. When using CloudSolrServer if you make a LukeRequest it uses
LBHttpSolrServer internally and it sends request to just one Solr Node
(via
HttpSolrServer) as round robin. So you may get 20 docs as a result at
first
request and if you send same request you may get 15 docs as a result too.
Using a PropagateServer inside CloudSolrServer will fix that bug.
I've made initial patchs for them and I will change/add code to them
after
getting feedback from community (i.e. first patch does not make multi
threaded requests at PropagateServer, I just want to get feedbacks of
community after that I will add other features)
Thanks;
Furkan KAMACI
