Hi Erick, thanks. I have about 40 shards. repFactor=2. The cause of slower shards is very interesting, and this is the main approach we took. Note that in every query, it is another shard which is the slowest. In 20% of the queries, the slowest shard takes about 4 times more than the average shard qtime. While continuing investigation, remember it might be the virtualization / storage-access / network / gc /..., so I thought that reducing the effect of the slow shards might be a good (temporary or permanent) solution.
I thought it should be an almost trivial code change (for proving the concept). Isn't it? On Sat, Jul 27, 2013 at 6:11 PM, Erick Erickson <erickerick...@gmail.com>wrote: > This has been suggested, but so far it's not been implemented > as far as I know. > > I'm curious though, how many shards are you dealing with? I > wonder if it would be a better idea to try to figure out _why_ > you so often have a slow shard and whether the problem could > be cured with, say, better warming queries on the shards... > > Best > Erick > > On Fri, Jul 26, 2013 at 8:23 AM, Isaac Hebsh <isaac.he...@gmail.com> > wrote: > > Hi! > > > > When SolrClound executes a query, it creates shard requests, which is > sent > > to one replica of each shard. Total QTime is determined by the slowest > > shard response (plus some extra time). [For simplicity, let's assume that > > no stored fields are requested.] > > > > I suffer from a situation where in every query, some shards are much > slower > > than others. > > > > We might consider a different approach, which sends the shard request to > > *ALL* replicas of each shard. Solr will continue when responses are got > > from at least one replica of each shard. > > > > Of course, the amount of work that is wasted is big (multiplied by > > replicationFactor), but in my case, there are very few concurrent > queries, > > and the most important performance is the qtime. Such a solution might > > improve qtime significantly. > > > > > > Did someone tried this before? > > Any tip from where should I start in the code? >
