Re: how to optimize same query with different start values

2013-01-15 Thread Andre Bois-Crettez
It looks like a use case for using Solrj with queryAndStreamResponse ? http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/client/solrj/SolrServer.html#queryAndStreamResponse%28org.apache.solr.common.params.SolrParams,%20org.apache.solr.client.solrj.StreamingResponseCallback%29 André O

Re: how to optimize same query with different start values

2013-01-15 Thread Mikhail Khludnev
It's a well know search engines limitation. This post will help you get into the core problem http://www.searchworkings.org/blog/-/blogs/lucene-solr-and-deep-paging . it seems that the solution is contributed into Lucene, but not yet for Solr. On Tue, Jan 15, 2013 at 6:36 PM, Upayavira wrote: >

Re: how to optimize same query with different start values

2013-01-15 Thread Upayavira
You are setting yourself up for disaster. If you ask Solr for documents 1000 to 1010, it needs to sort documents 1 to 1010, and discard the first 1000, which causes horrible performance. I'm curious to hear if others have strategies to extract content sequentially from an index. I suspect a new S