shalinmangar opened a new pull request #2086: URL: https://github.com/apache/lucene-solr/pull/2086
# Description HttpClusterStateProvider fetches and caches Aliases and Live Nodes for 5 seconds. But all calls to getState are live. These collection states are supposed to be cached inside BaseSolrCloudClient but BaseSolrCloudClient only caches collection state if it is lazy which is never the case for states returned by HttpClusterStateProvider. The BaseSolrCloudClient calls getState for each collection mentioned in the request thereby making a live http call. It also calls getClusterProperties() for each live node! So overall, at least 4 HTTP calls are made to fetch cluster state for each update request when using HttpClusterStateProvider. There may be more if aliases are involved or if more than one collection is specified in the request. Similar problems exist on the query path as well. Due to these reasons, using HttpClusterStateProvider causes horrible latencies and throughput for update and search requests. # Solution This PR fixes BaseCloudSolrClient to cache collection states returned by HttpClusterStateProvider, reduce the number of calls to getClusterProperty in case of admin requests, replace usage of getClusterStateProvider().getState() with getDocCollection() which caches the collection state. Therefore the number of clusterstatus calls are reduced from 4 for each query/indexing to either one at max and usually 0 (if data is cached already). # Tests A new CountingHttpClusterStateProvider test class is added which can track the number of http calls made. This class is used in the HttpClusterStateProviderTest to track and assert the number of http calls made by HttpClusterStateProvider. # Checklist Please review the following and check all that apply: - [X] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [X] I have created a Jira issue and added the issue ID to my pull request title. - [X] I have developed this patch against the `master` branch. - [X] I have run `./gradlew check`. - [X] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
