What does this have to to with removing the
int32 limit? It's still the same problem if you have
a high start parameter, it's the deep paging
issue that's part of Solr.
I know there has been work on this (you'll have to
search the JIRAs), the basic idea is that you pass
enough information that
Oh, P.S. Solr is a great search engine, but it's
certainly not the perfect answer to all problems.
Mayhap you've hit on a case where it isn't the
best solution
On Sat, Jul 6, 2013 at 8:22 AM, Erick Erickson erickerick...@gmail.comwrote:
What does this have to to with removing the
int32
Eric,
We did not have any RAM problems, but just the following official
limitation makes our life too miserable to use the shards:
Makes it more inefficient to use a high start parameter. For example,
if you request start=50rows=25 on an index with 500,000+ docs per
shard, this will
What are you doing that start=50 is normal? --wunder
On Jul 5, 2013, at 1:28 PM, Valery Giner wrote:
Eric,
We did not have any RAM problems, but just the following official limitation
makes our life too miserable to use the shards:
Makes it more inefficient to use a high start
As a simplest example, just write a query result into a file for
processing by external programs (the programs are out of our control,
and the result could contain millions of docs)
Thanks,
Val
On 07/05/2013 04:41 PM, Walter Underwood wrote:
What are you doing that start=50 is normal?
Does https://issues.apache.org/jira/browse/SOLR-2112 help?
Otis
--
Solr ElasticSearch Support -- http://sematext.com/
Performance Monitoring -- http://sematext.com/spm
On Fri, Jul 5, 2013 at 5:57 PM, Valery Giner valgi...@research.att.com wrote:
As a simplest example, just write a query
My off the cuff thought is that there are significant costs trying to
do this that would be paid by 99.999% of setups out there. Also,
usually you'll run into other issues (RAM etc) long before you come
anywhere close to 2^31 docs.
Lucene/Solr often allocates int[maxDoc] for various operations.
The Integer.MAX_VALUE-1 limit is set by Lucene. As hardware capacity and
performance continues to advance, I think it's only a matter of time before
Lucene (and then Solr) relaxes the limit, but I don't imagine it will
happened real soon. Maybe in Lucene/Solr 6.0?
-- Jack Krupansky
$100 for anyone who gets me a working Long.MAX_VALUE branch! ;-)
I know that for many of the SOLR with faceting use cases, things will
not scale to Long documents, but there are a number of more
straightforward use cases, where SOLR/Lucene will scale to Long. Like
simple searches, small numbers
Val,
Haven't seen this mentioned in a while...
I'm curious...what sort of index, queries, hardware, and latency
requirements do you have?
Otis
Solr ElasticSearch Support
http://sematext.com/
On May 1, 2013 4:36 PM, Valery Giner valgi...@research.att.com wrote:
Dear Solr Developers,
I've
Otis,
The documents themselves are relatively small, tens of fields, only a
few of them could be up to a hundred bytes.
Lunix Servers with relatively large RAM (256),
Minutes on the searches are fine for our purposes, adding a few tens of
millions of records in tens of minutes are also fine.
Dear Solr Developers,
I've been unable to find an answer to the question in the subject line
of this e-mail, except of a vague one.
We need to be able to index over 2bln+ documents. We were doing well
without sharding until the number of docs hit the limit ( 2bln+). The
performance was
12 matches
Mail list logo
