[
https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749274#comment-15749274
]
ASF GitHub Bot commented on BOOKKEEPER-950:
-------------------------------------------
GitHub user rithin-shetty opened a pull request:
https://github.com/apache/bookkeeper/pull/93
BOOKKEEPER-950: Ledger placement policy to accomodate different stora…
…ge capacity of bookies
This change introduces Disk weight based ledger placement. Currently free
disk space is the only supported
weight for a bookie. This change also introduces a new protocol message
between bk client and server
called GET_BOOKIE_INFO. This message is used by the client to retrieve the
free disk space info from
all the bookies. The existing placement policies: DefaultPlacementPolicy
and RackAwarePlacementPolicy
have been enhanced to make use of the weight while selecting bookies. New
test cases have been added to
test RackawarePlacement with weights. A new test class has been added to
test the weight based selection
algorithm in a stand alone fashion.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rithin-shetty/bookkeeper
weightBasedPlacementDec13
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bookkeeper/pull/93.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #93
----
commit e78b11d46ae4c3bdd58082d2b4ffa6e9ce06c68d
Author: Rithin <rithin.she...@salesforce.com>
Date: 2016-12-14T04:56:15Z
BOOKKEEPER-950: Ledger placement policy to accomodate different storage
capacity of bookies
This change introduces Disk weight based ledger placement. Currently free
disk space is the only supported
weight for a bookie. This change also introduces a new protocol message
between bk client and server
called GET_BOOKIE_INFO. This message is used by the client to retrieve the
free disk space info from
all the bookies. The existing placement policies: DefaultPlacementPolicy
and RackAwarePlacementPolicy
have been enhanced to make use of the weight while selecting bookies. New
test cases have been added to
test RackawarePlacement with weights. A new test class has been added to
test the weight based selection
algorithm in a stand alone fashion.
----
> Ledger placement policy to accomodate different storage capacity of bookies
> ---------------------------------------------------------------------------
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
> Issue Type: New Feature
> Reporter: Rithin Shetty
> Assignee: Rithin Shetty
> Fix For: 4.5.0
>
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with
> different storage capacity: some will have 1TB others might have 3TB. Also,
> our ledgers are likely going to be long lived. The current ledger placement
> policy selects the bookies randomly leading to uniform distribution. This
> would cause some of bookies to reach high utilization while the rest would be
> underutilized. We need a new ledger placement policy that has higher
> probability of selecting bookies with higher free disk space than the ones
> with lower disk free space.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
