Shawn: Thanks! Obviously a bad cut/paste on my part, you're absolutely correctl
On Wed, Apr 16, 2014 at 11:35 AM, Toke Eskildsen <t...@statsbiblioteket.dk> wrote: > mukh....@gmail.com [mukh....@gmail.com] On Behalf Of Mukesh Jha > [me.mukesh....@gmail.com] wrote: >> In my solr cluster I've multiple shards and each shard containing >> ~500,000,000 documents total index size being ~1 TB. > >> I was just wondering how much more can I keep on adding to the shard before >> we reach a tipping point and the performance starts to degrade? > > Looking isolated at the number of documents, there are no hard corners: More > documents just means worse performance, with the relative performance > deterioration per document getting smaller and smaller. For index size, there > is a bump when it blows the disk cache and machines using spinning drives are > quite hurt by that. But it seems that you are way past that so adding more > documents will just give you slightly worse performance. > > If you are not already doing so, do log index size and performance as your > data grows. You should soon be able to get an idea about how performance > deteriorates as a function of index size and with that you can see when it > will be too bad for your needs. > >> Also as best practice what is the recomended no of docs / size of shards . > > None. It depends on your data and your usage. > > - Toke Eskildsen
