If you request 1000 docs from each shard, then aggregator is really fetching 30,000 total documents, which then it must merge (re-sort results, and take top 1000 to return to client). Its possible that SOLR merging implementation needs optimized, but it does not seem like it could be that slow. How big are the documents you return (how many fields, avg KB per doc, etc.)? I would take a look at network to make sure that is not some bottleneck, and also to make sure there is not some underlying issue making 30 concurrent HTTP requests from the aggregator. I am not an expert in Java, but under .NET there is a setting that limits concurrent out-going HTTP requests from a process that must be over-ridden via configuration, otherwise by default is very limiting.
Does performance get much better if you only request top 100, or top 10 documents instead of top 1000? What if you only request a couple fields, instead of fl=*? What if you only search 10 shards instead of 30? I would collect those numbers and try to determine if time increases linearly or not as you increase shards and/or # of docs. On Wed, Nov 23, 2011 at 9:55 AM, Artem Lokotosh <arco...@gmail.com> wrote: >> If the response time from each shard shows decent figures, then aggregator> >> seems to be a bottleneck. Do you btw have a lot of concurrent users?For now >> is not a problem, but we expect from 1K to 10K of concurrent users and maybe >> more > On Wed, Nov 23, 2011 at 4:43 PM, Dmitry Kan <dmitry....@gmail.com> wrote: >> If the response time from each shard shows decent figures, then aggregator >> seems to be a bottleneck. Do you btw have a lot of concurrent users? >> >> On Wed, Nov 23, 2011 at 4:38 PM, Artem Lokotosh <arco...@gmail.com> wrote: >> >>> > Is this log from the frontend SOLR (aggregator) or from a shard? >>> from aggregator >>> >>> > Can you merge, e.g. 3 shards together or is it much effort for your team? >>> Yes, we can merge. We'll try to do this and review how it will works >>> Thanks, Dmitry >>> >>> Any another ideas? >>> > > -- > Best regards, > Artem Lokotosh mailto:arco...@gmail.com >
