[ https://issues.apache.org/jira/browse/HBASE-12636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245821#comment-14245821 ]
stack commented on HBASE-12636: ------------------------------- bq. Is it really OK to double replicate data? It's not doing that now, right? The duplicate will just overwrite. Counters will be slightly off. I was thinking this only repercussion. Better than buckling your local zk ensemble with updates I'd say. Fine holding on commit till gets a bit more attention. > Avoid too many write operations on zookeeper in replication > ----------------------------------------------------------- > > Key: HBASE-12636 > URL: https://issues.apache.org/jira/browse/HBASE-12636 > Project: HBase > Issue Type: Improvement > Affects Versions: 0.94.11 > Reporter: Liu Shaohui > Assignee: Liu Shaohui > Labels: replication > Fix For: 1.0.0 > > Attachments: HBASE-12635-v2.diff, HBASE-12636-v1.diff > > > In our production cluster, we found there are about over 1k write operations > per second on zookeeper from hbase replication. The reason is that the > replication source will write the log position to zookeeper for every edit > shipping. If the current replicating WAL is just the WAL that regionserver is > writing to, each skipping will be very small but the frequency is very high, > which causes many write operations on zookeeper. > A simple solution is that writing log position to zookeeper when position > diff or skipped edit number is larger than a threshold, not every edit > shipping. > Suggestions are welcomed, thx~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)