Re: Info on document number limitations

2020-02-14 Thread Adrien Grand
Lucene has a limit of 2^31-1-128 documents per index, see IndexWriter.MAX_DOCS. Users don't often run into this limit but I've seen it happen multiple times. I think that it's unlikely that Lucene will ever remove this limit on a per-segment basis, however there have been some discussions about

Re: Info on document number limitations

2020-02-09 Thread Erick Erickson
Also, given how people use search, they hit performance issues long before running out of document IDs. Usually. Although that said I do know of one user who’s running in the 1.0-1.5B range per replica so 2B is just around the corner. Of course they have to be _very_ careful how they use Solr.

Re: Info on document number limitations

2020-02-08 Thread Tim Casey
Hi Doug, I don't know the specific limits. But the document limits are going to be around an int, probably signed. This comes out to mean about 2 billion documents per lucene index. This is fairly embedded into the lucene code. The way the collective we have solved this is through forms of

Info on document number limitations

2020-02-07 Thread Doug Tarr
Hi! I'm working on a team that is building a lucene based search platform. I've been lurking on this list for a while as we are spooling up on learning the various components of Lucene. Thank you all for your amazing work! I'm interested in learning more about what work has been done around