[
https://issues.apache.org/jira/browse/HBASE-9501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Honghua updated HBASE-9501:
--------------------------------
Attachment: HBASE-9501-trunk_v2.patch
[~jdcryans] : Thanks very much for the review. New patch attached according to
your review feedback. The throttling logic is refactored out to be a helper
method which computes the sleep time(>0 if needed) according to the current
context which are passed as parameters, the real sleep occurs in shipEdits
after calling this helper function. Now the newly added unit test
testApplyThrottling in TestReplicationSmallTests just checks the sleep time
computed by this helper function, but without real sleep operation/call. The
original added testThrottling is also run several times to verify this refactor
is fine before being removed.
> Provide throttling for replication
> ----------------------------------
>
> Key: HBASE-9501
> URL: https://issues.apache.org/jira/browse/HBASE-9501
> Project: HBase
> Issue Type: Improvement
> Components: Replication
> Reporter: Feng Honghua
> Assignee: Feng Honghua
> Attachments: HBASE-9501-trunk_v0.patch, HBASE-9501-trunk_v1.patch,
> HBASE-9501-trunk_v2.patch
>
>
> When we disable a peer for a time of period, and then enable it, the
> ReplicationSource in master cluster will push the accumulated hlog entries
> during the disabled interval to the re-enabled peer cluster at full speed.
> If the bandwidth of the two clusters is shared by different applications, the
> push at full speed for replication can use all the bandwidth and severely
> influence other applications.
> Though there are two config replication.source.size.capacity and
> replication.source.nb.capacity to tweak the batch size each time a push
> delivers, but if decrease these two configs, the number of pushes increase,
> and all these pushes proceed continuously without pause. And no obvious help
> for the bandwidth throttling.
> From bandwidth-sharing and push-speed perspective, it's more reasonable to
> provide a bandwidth up limit for each peer push channel, and within that
> limit, peer can choose a big batch size for each push for bandwidth
> efficiency.
> Any opinion?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
