I'm inferring that at the end of the day, all your docs fit in a single index, correct? SolrCloud won't be a magic bullet, and I'd strongly advise if you _do_ go to SolrCloud to use SlolrJ or similar to feed docs as DIH runs on a single server.
However, all that aside if I can restate your problem to be sure I have it right: When you replicate out from your master to your 100 slaves the master is overburdened. Have you consider the "repeater" strategy? See: https://cwiki.apache.org/confluence/display/solr/Index+Replication#IndexReplication-SettingUpaRepeaterwiththeReplicationHandler The idea is that the master serves the index to, say, 10 slaves and each of those slaves in turn serves 10 _other_ slaves. Best, Erick On Thu, Mar 30, 2017 at 6:12 AM, santosh sidnal <sidnal.sant...@gmail.com> wrote: > Hi All, > > I have a problem with scalability on my project. we are running almost > close of 100 cores which are having documents of ~25000 each and the total > size of the index files being 7.5 GB. > > > Also, we have the staging server where we build index files using data > importer and using replication we are pushing data to data LIVE servers > which are used for serving the live application. because we are initiating > index pulling from live servers currently 5 servers against one stage > server. Stage server is overburden and will not be able to respond properly > either to indexing job or to other services (very minimal). > > > So now the problem statement is I understand my current SOLR architecture > is not able to handle all my needs so we are thinking to upgrade to next > levels but I am confused with below questions, > > > 1. Can I use index Sharding for my problem? it is recommended to use if > my have more than ~1million docs in a core but I have in only 25000 in one > core and having 100 cores. > 2. Can I consider SOLR cloud? if yes please let me know why? > 3. How about using apache zookeeper and maintain only LIVE servers by > dividing every 20 cores to each server? so that I can make only 25 snap > pull request to stage server instead of 100 snap-puller per server as > current. > > Any new suggestions or reply to this email is greatly appreciated. Thanks > in advance. > > > > -- > Regards, > Santosh Sidnal
