Re: partialResults, distributed search SOLR-502
I don't think this patch is working yet. If I take a shard out of rotation (even just one out of four), I get an error: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused org.apache.solr.common.SolrException: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:256) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1156) from http://localhost:8983/solr/select/?shards=10.0.16.181:8983,10.0.16.182:8983,10.0.16.183:8983,10.0.16.184:8983/solrtimeAllowed=1000q=cancer%0D%0Aversion=2.2start=0rows=10indent=on where .181 is down but .183-.184 are up. On Fri, Aug 15, 2008 at 1:23 PM, Brian Whitman [EMAIL PROTECTED] wrote: I was going to file a ticket like this: A SOLR-303 query with shards=host1,host2,host3 when host3 is down returns an error. One of the advantages of a shard implementation is that data can be stored redundantly across different shards, either as direct copies (e.g. when host1 and host3 are snapshooter'd copies of each other) or where there is some data RAID that stripes indexes for redundancy. But then I saw SOLR-502, which appears to be committed. If I have the above scenario (host1,host2,host3 where host3 is not up) and set a timeAllowed, will I still get a 400 or will it come back with partial results? If not, can we think of a way to get this to work? It's my understanding already that duplicate docIDs are merged in the SOLR-303 response, so other than building in some this host isn't working, just move on and report it and of course the work to index redundantly, we wouldn't need anything to achieve a good redundant shard implementation. B -- Regards, Ian Connor
Re: partialResults, distributed search SOLR-502
Hi, I have traced this as far as I can figure. It does seem as though the patch is in the trunk. I can see that timeAllowed is certainly being set and the lucene class TimeLimitedCollector is being used when the param is there. However, I have tried to trace RequestHandlerBase from this stack through to SearchHandler and get lost when the Shard is submitted. I can see it creates a CommonsHttpSolrServer to make the request and that at least at this point the timeAllowed param is alive and well. However, when I try to dive into the QueryRequest and SolrServer I realize my java is a little rusty. Can anyone explain how the QueryRequest here uses the code that is found in SolrIndexSearcher? On Mon, Aug 18, 2008 at 9:31 AM, Ian Connor [EMAIL PROTECTED] wrote: I don't think this patch is working yet. If I take a shard out of rotation (even just one out of four), I get an error: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused org.apache.solr.common.SolrException: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:256) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1156) from http://localhost:8983/solr/select/?shards=10.0.16.181:8983,10.0.16.182:8983,10.0.16.183:8983,10.0.16.184:8983/solrtimeAllowed=1000q=cancer%0D%0Aversion=2.2start=0rows=10indent=on where .181 is down but .183-.184 are up. On Fri, Aug 15, 2008 at 1:23 PM, Brian Whitman [EMAIL PROTECTED] wrote: I was going to file a ticket like this: A SOLR-303 query with shards=host1,host2,host3 when host3 is down returns an error. One of the advantages of a shard implementation is that data can be stored redundantly across different shards, either as direct copies (e.g. when host1 and host3 are snapshooter'd copies of each other) or where there is some data RAID that stripes indexes for redundancy. But then I saw SOLR-502, which appears to be committed. If I have the above scenario (host1,host2,host3 where host3 is not up) and set a timeAllowed, will I still get a 400 or will it come back with partial results? If not, can we think of a way to get this to work? It's my understanding already that duplicate docIDs are merged in the SOLR-303 response, so other than building in some this host isn't working, just move on and report it and of course the work to index redundantly, we wouldn't need anything to achieve a good redundant shard implementation. B -- Regards, Ian Connor -- Regards, Ian Connor
Re: partialResults, distributed search SOLR-502
On Aug 18, 2008, at 11:51 AM, Ian Connor wrote: On Mon, Aug 18, 2008 at 9:31 AM, Ian Connor [EMAIL PROTECTED] wrote: I don't think this patch is working yet. If I take a shard out of rotation (even just one out of four), I get an error: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused It's my understanding that SOLR-502 is really only concerned with queries timing out (i.e. they connect but take over N seconds to return) If the connection gets refused then a non-solr java connection exception is thrown. Something would have to get put in that (optionally) catches connection errors and still builds the response from the shards that did respond. On Fri, Aug 15, 2008 at 1:23 PM, Brian Whitman [EMAIL PROTECTED] wrote: I was going to file a ticket like this: A SOLR-303 query with shards=host1,host2,host3 when host3 is down returns an error. One of the advantages of a shard implementation is that data can be stored redundantly across different shards, either as direct copies (e.g. when host1 and host3 are snapshooter'd copies of each other) or where there is some data RAID that stripes indexes for redundancy. But then I saw SOLR-502, which appears to be committed. If I have the above scenario (host1,host2,host3 where host3 is not up) and set a timeAllowed, will I still get a 400 or will it come back with partial results? If not, can we think of a way to get this to work? It's my understanding already that duplicate docIDs are merged in the SOLR-303 response, so other than building in some this host isn't working, just move on and report it and of course the work to index redundantly, we wouldn't need anything to achieve a good redundant shard implementation. B -- Regards, Ian Connor -- Regards, Ian Connor -- http://variogr.am/
Re: partialResults, distributed search SOLR-502
Yes, as far as I know, what Brian said is correct. Also, as far as I know, there is nothing that gracefully handles problematic Solr instances during distributed search. Solr 1.4 request? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Brian Whitman [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Monday, August 18, 2008 11:57:23 AM Subject: Re: partialResults, distributed search SOLR-502 On Aug 18, 2008, at 11:51 AM, Ian Connor wrote: On Mon, Aug 18, 2008 at 9:31 AM, Ian Connor wrote: I don't think this patch is working yet. If I take a shard out of rotation (even just one out of four), I get an error: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused It's my understanding that SOLR-502 is really only concerned with queries timing out (i.e. they connect but take over N seconds to return) If the connection gets refused then a non-solr java connection exception is thrown. Something would have to get put in that (optionally) catches connection errors and still builds the response from the shards that did respond. On Fri, Aug 15, 2008 at 1:23 PM, Brian Whitman wrote: I was going to file a ticket like this: A SOLR-303 query with shards=host1,host2,host3 when host3 is down returns an error. One of the advantages of a shard implementation is that data can be stored redundantly across different shards, either as direct copies (e.g. when host1 and host3 are snapshooter'd copies of each other) or where there is some data RAID that stripes indexes for redundancy. But then I saw SOLR-502, which appears to be committed. If I have the above scenario (host1,host2,host3 where host3 is not up) and set a timeAllowed, will I still get a 400 or will it come back with partial results? If not, can we think of a way to get this to work? It's my understanding already that duplicate docIDs are merged in the SOLR-303 response, so other than building in some this host isn't working, just move on and report it and of course the work to index redundantly, we wouldn't need anything to achieve a good redundant shard implementation. B -- Regards, Ian Connor -- Regards, Ian Connor -- http://variogr.am/
Re: partialResults, distributed search SOLR-502
When I put logging into SolrIndexSearcher just to see if we get there, I don't see any messages. However, I do see logging without a problem in QueryRequest and above. My issue is that I just cannot understand how SolrIndexSearcher comes into play here. On Mon, Aug 18, 2008 at 11:57 AM, Brian Whitman [EMAIL PROTECTED] wrote: On Aug 18, 2008, at 11:51 AM, Ian Connor wrote: On Mon, Aug 18, 2008 at 9:31 AM, Ian Connor [EMAIL PROTECTED] wrote: I don't think this patch is working yet. If I take a shard out of rotation (even just one out of four), I get an error: org.apache.solr.client.solrj.SolrServerException: java.net.ConnectException: Connection refused It's my understanding that SOLR-502 is really only concerned with queries timing out (i.e. they connect but take over N seconds to return) If the connection gets refused then a non-solr java connection exception is thrown. Something would have to get put in that (optionally) catches connection errors and still builds the response from the shards that did respond. On Fri, Aug 15, 2008 at 1:23 PM, Brian Whitman [EMAIL PROTECTED] wrote: I was going to file a ticket like this: A SOLR-303 query with shards=host1,host2,host3 when host3 is down returns an error. One of the advantages of a shard implementation is that data can be stored redundantly across different shards, either as direct copies (e.g. when host1 and host3 are snapshooter'd copies of each other) or where there is some data RAID that stripes indexes for redundancy. But then I saw SOLR-502, which appears to be committed. If I have the above scenario (host1,host2,host3 where host3 is not up) and set a timeAllowed, will I still get a 400 or will it come back with partial results? If not, can we think of a way to get this to work? It's my understanding already that duplicate docIDs are merged in the SOLR-303 response, so other than building in some this host isn't working, just move on and report it and of course the work to index redundantly, we wouldn't need anything to achieve a good redundant shard implementation. B -- Regards, Ian Connor -- Regards, Ian Connor -- http://variogr.am/ -- Regards, Ian Connor 1 Leighton St #605 Cambridge, MA 02141 Direct Line: +1 (978) 672 Call Center Phone: +1 (714) 239 3875 (24 hrs) Mobile Phone: +1 (312) 218 3209 Fax: +1(770) 818 5697 Suisse Phone: +41 (0) 22 548 1664 Skype: ian.connor
partialResults, distributed search SOLR-502
I was going to file a ticket like this: A SOLR-303 query with shards=host1,host2,host3 when host3 is down returns an error. One of the advantages of a shard implementation is that data can be stored redundantly across different shards, either as direct copies (e.g. when host1 and host3 are snapshooter'd copies of each other) or where there is some data RAID that stripes indexes for redundancy. But then I saw SOLR-502, which appears to be committed. If I have the above scenario (host1,host2,host3 where host3 is not up) and set a timeAllowed, will I still get a 400 or will it come back with partial results? If not, can we think of a way to get this to work? It's my understanding already that duplicate docIDs are merged in the SOLR-303 response, so other than building in some this host isn't working, just move on and report it and of course the work to index redundantly, we wouldn't need anything to achieve a good redundant shard implementation. B
