[
https://issues.apache.org/jira/browse/HBASE-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598568#comment-14598568
]
Lars Hofhansl commented on HBASE-12636:
---------------------------------------
Sorry... Missed this.
The patch's approach is to perform the replication and simply not log every
advance.
If I understand the issue correctly this is for the case we write into the
cluster at a medium pace and hence each check on whether there's something to
replicate would only find a few edits. Hence another approach is to simply slow
down replication a bit. I.e. if we only have a few edits we wait a bit longer
to edits to come in, only then we replicate the data and advance the pointer in
ZK. That would also improve replication efficiency.
> Avoid too many write operations on zookeeper in replication
> -----------------------------------------------------------
>
> Key: HBASE-12636
> URL: https://issues.apache.org/jira/browse/HBASE-12636
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.94.11
> Reporter: Liu Shaohui
> Assignee: Liu Shaohui
> Labels: replication
> Fix For: 2.0.0
>
> Attachments: HBASE-12635-v2.diff, HBASE-12636-v1.diff
>
>
> In our production cluster, we found there are about over 1k write operations
> per second on zookeeper from hbase replication. The reason is that the
> replication source will write the log position to zookeeper for every edit
> shipping. If the current replicating WAL is just the WAL that regionserver is
> writing to, each skipping will be very small but the frequency is very high,
> which causes many write operations on zookeeper.
> A simple solution is that writing log position to zookeeper when position
> diff or skipped edit number is larger than a threshold, not every edit
> shipping.
> Suggestions are welcomed, thx~
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
