tomas.kalas kala...@email.cz wrote:
Existing some hardware or software limits for indexing data?
The only really hard Solr limit is 2 billion X per shard, where X is document
count, unique values in a DocValues String field and other things like that.
There are some softer limits, after which performance degrades markedly: Number
of fields (hundreds are fine, millions are unrealistic), number of shards
(avoid going into the thousands). Having a Java heap of hundreds of gigabytes
is possible, but requires tweaking to avoid very long garbage collection
pauses. I do not know of a byte size limit for shards: Shards of 1-2 TB works
without problems on fitting hardware.
And is some maximum of indexed documents?
While the limit is 2 billion per single shard, SolrCloud does not have this
limitation. A soft limit before doing some custom multi-level setup would thus
be around 2000 billion documents, divided across 1000 shards.
- Toke Eskildsen
