[ https://issues.apache.org/jira/browse/SOLR-16487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Risden updated SOLR-16487: -------------------------------- Status: Patch Available (was: Open) > Improve efficiency of pull replicas > ----------------------------------- > > Key: SOLR-16487 > URL: https://issues.apache.org/jira/browse/SOLR-16487 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: replication (java) > Affects Versions: 9.0, 8.11.2 > Reporter: Justin Sweeney > Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Currently there are a few inefficiencies related to pull replicas that can > cause increased resource usage on a Solr cluster, particularly if there are a > large number of cores. > The following are the inefficiencies identified: > # The > [RecoveryStrategy.replicate()|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java#L219] > method makes a call to commit to on the leader. This happens whenever a > replica is reloaded. For PULL replicas in particular this isn't necessary > since we can just pull down whatever the latest data is and rely on other > mechanisms to be consistently committing the leader. (As an aside, it seems > like forcing a commit on the leader might never be necessary, but for this > I've limited it to focusing on PULL replicas). > # In a case where the leader has no data yet (index version is 0), then a > non-leader replica will consistently delete and recreate its core due to this > case in IndexFetcher: > [https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/handler/IndexFetcher.java#L549.] > This can cause unnecessary CPU usage until the leader has data indexed to it. > # The polling for replication is fairly simply, but can lead to polling too > often. As an example if you had the following config for commits: > {code:java} > <autoCommit> > <maxTime>15000</maxTime> > <openSearcher>false</openSearcher> > </autoCommit> > <autoSoftCommit> > <maxTime>60000</maxTime> > </autoSoftCommit> {code} > The current logic would setup polling to be half of the autoCommit time, so > poll every 7.5 seconds. However since a new searcher isn't opened, there will > only be changes reflected every 60 seconds on the leader. We can make this > logic a bit smarter knowing that the replication handler won't reflect > changes until a new searcher is opened. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org