[ https://issues.apache.org/jira/browse/SOLR-14613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17186938#comment-17186938 ]
Ilan Ginzburg commented on SOLR-14613: -------------------------------------- I would definitely love to have more eyeballs on this [PR1684|https://github.com/apache/lucene-solr/pull/1684]. To make it easier, a way to approach it is as follows: [{{Assign.createAssignStrategy()}}|https://github.com/apache/lucene-solr/pull/1684/files#diff-3fd945fa4f7d44dfc97595494bb8ea18R552] picks between existing placement strategies and the new plugin one. [{{PlacementPluginAssignStrategy.assign()}}|https://github.com/apache/lucene-solr/pull/1684/files#diff-5d6e5294192d97f9b0a4d6baa22d25e2R56] is the glue between existing Solr placement infra and the plugin code, transforming Solr abstractions into the plugin abstractions and vice versa. A sample initial (toy) plugin example can be found in {{[SamplePluginMinimizeCores|https://github.com/apache/lucene-solr/pull/1684/files#diff-9e00a78f3810f5e5c3907723fa9d187c]}}. It does implement the placement logic and computation (in a suboptimal way, WIP) and shows how it interacts with the placement plugin interfaces defined in this PR. There are 5 families of interfaces that a plugin interacts with: 1. Observing existing cluster topology: {{[Cluster|https://github.com/apache/lucene-solr/pull/1684/files#diff-b69b4e75815b3c67aae07fc92f91608d]}}, {{[Node|https://github.com/apache/lucene-solr/pull/1684/files#diff-9e130453afd9b39c1a421906e28a4c4a]}}, {{[SolrCollection|https://github.com/apache/lucene-solr/pull/1684/files#diff-9cb21e61da6a3b1e537de35b78eedb1a]}}, {{[Shard|https://github.com/apache/lucene-solr/pull/1684/files#diff-de4d5a286497ffa09c92901e5282fd6a]}}, {{[Replica|https://github.com/apache/lucene-solr/pull/1684/files#diff-b236471170ce1c259c1a3fbc0aeb5c13]}}. 2. Receiving a placement request, for example: {{[AddReplicasPlacementRequest|https://github.com/apache/lucene-solr/pull/1684/files#diff-2066869526c1471e9ec46d71faec1826]}} 3. Preparing to fetch additional data from the cluster by creating {{PropertyKey}} instances using {{[PropertyKeyFactory|https://github.com/apache/lucene-solr/pull/1684/files#diff-5d65928113495143e29323a88ee6bf9f]}} 4. Fetching the additional data using {{[PropertyValueFetcher|https://github.com/apache/lucene-solr/pull/1684/files#diff-a55062703fa3cf875dea2542e930e82e]}} and getting the corresponding {{[PropertyValue|https://github.com/apache/lucene-solr/pull/1684/files#diff-d4ace968d26586330e5b20a0b698e9a7]}} (for example {{[CoresCountPropertyValue|https://github.com/apache/lucene-solr/pull/1684/files#diff-955a2dda6d825451ceb86a5b4e452d3b]}}) 5. Creating the placement decisions and returning them as plugin output, using {{[PlacementPlanFactory|https://github.com/apache/lucene-solr/pull/1684/files#diff-f416298dbc7b2fc4a569737da5174fc5]}} > Provide a clean API for pluggable replica assignment implementations > -------------------------------------------------------------------- > > Key: SOLR-14613 > URL: https://issues.apache.org/jira/browse/SOLR-14613 > Project: Solr > Issue Type: Improvement > Components: AutoScaling > Reporter: Andrzej Bialecki > Assignee: Ilan Ginzburg > Priority: Major > Time Spent: 20h 40m > Remaining Estimate: 0h > > As described in SIP-8 the current autoscaling Policy implementation has > several limitations that make it difficult to use for very large clusters and > very large collections. SIP-8 also mentions the possible migration path by > providing alternative implementations of the placement strategies that are > less complex but more efficient in these very large environments. > We should review the existing APIs that the current autoscaling engine uses > ({{SolrCloudManager}} , {{AssignStrategy}} , {{Suggester}} and related > interfaces) to see if they provide a sufficient and minimal API for plugging > in alternative autoscaling placement strategies, and if necessary refactor > the existing APIs. > Since these APIs are internal it should be possible to do this without > breaking back-compat. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org