[
https://issues.apache.org/jira/browse/SOLR-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nitin Sharma updated SOLR-9241:
-------------------------------
Attachment: (was: SOLR-9241-4.6.1.patch)
> Rebalance API for SolrCloud
> ---------------------------
>
> Key: SOLR-9241
> URL: https://issues.apache.org/jira/browse/SOLR-9241
> Project: Solr
> Issue Type: New Feature
> Components: SolrCloud
> Affects Versions: 4.6.1
> Environment: Ubuntu, Mac OsX
> Reporter: Nitin Sharma
> Labels: Cluster, SolrCloud
> Fix For: 4.6.1
>
> Attachments: Redistribute_After.jpeg, Redistribute_Before.jpeg,
> Redistribute_call.jpeg, Replace_After.jpeg, Replace_Before.jpeg,
> Replace_Call.jpeg, SOLR-9241-6.1.patch
>
> Original Estimate: 2,016h
> Remaining Estimate: 2,016h
>
> This is the v1 of the patch for Solrcloud Rebalance api (as described in
> http://engineering.bloomreach.com/solrcloud-rebalance-api/) , built at
> Bloomreach by Nitin Sharma and Suruchi Shah. The goal of the API is to
> provide a zero downtime mechanism to perform data manipulation and efficient
> core allocation in solrcloud. This API was envisioned to be the base layer
> that enables Solrcloud to be an auto scaling platform. (and work in unison
> with other complementing monitoring and scaling features).
> Patch Status:
> ===============
> The patch is work in progress and incremental. We have done a few rounds of
> code clean up. We wanted to get the patch going first to get initial feed
> back. We will continue to work on making it more open source friendly and
> easily testable.
> Deployment Status:
> ====================
> The platform is deployed in production at bloomreach and has been battle
> tested for large scale load. (millions of documents and hundreds of
> collections).
> Internals:
> =============
> The internals of the API and performance :
> http://engineering.bloomreach.com/solrcloud-rebalance-api/
> It is built on top of the admin collections API as an action (with various
> flavors). At a high level, the rebalance api provides 2 constructs:
> Scaling Strategy: Decides how to move the data. Every flavor has multiple
> options which can be reviewed in the api spec.
> Re-distribute - Move around data in the cluster based on capacity/allocation.
> Auto Shard - Dynamically shard a collection to any size.
> Smart Merge - Distributed Mode - Helps merging data from a larger shard setup
> into smaller one. (the source should be divisible by destination)
> Scale up - Add replicas on the fly
> Scale Down - Remove replicas on the fly
> Allocation Strategy: Decides where to put the data. (Nodes with least
> cores, Nodes that do not have this collection etc). Custom implementations
> can be built on top as well. One other example is Availability Zone aware.
> Distribute data such that every replica is placed on different availability
> zone to support HA.
> Detailed API Spec:
> ====================
> https://github.com/bloomreach/solrcloud-rebalance-api
> Contributors:
> =====================
> Nitin Sharma
> Suruchi Shah
> Questions/Comments:
> =====================
> You can reach me at nitin.sha...@bloomreach.com
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
