[ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727711#comment-14727711 ]
Ashish Singhi commented on HBASE-13153: --------------------------------------- bq. There seem to be two types of notifications from the peer cluster No, only one. Sorry if I confused you. At a very high level the sequence of operation will be something like below for a positive cycle, (1) doBulkLoad(HFiles, source cluster IDs) (2) notify(BulkLoadListeners) (3) ZK(add HFile entries only if no cyclic loop) \[return to the client\] (4) ReplicateEntries(entries, source cluster ids) --> Back to step (1). c1->c2 Complete bulk load process at cluster c1, (1) doBulkLoad(HFiles) (2) notify(BulkLoadListeners) (3) ZK(add HFile entries, no cyclic loop -> replicate to c2) \[return to the client\] (4) ReplicateEntries(entries, source cluster ids) Replication process at cluster c2, (1) doBulkLoad(HFiles,{c1\}) (2) notify(BulkLoadListeners) \[return to the client, client will delete the znode\] For cyclic replication case, c1->c2->c3->c1 Complete bulk load process at cluster c1, (1) doBulkLoad(HFiles) (2) notify(BulkLoadListeners) (3) ZK(add HFile entries, no cyclic loop -> replicate to c2) \[return to the client\] (4) ReplicateEntries(entries, source cluster ids) Replication process at cluster c2, (1) doBulkLoad(HFiles,{c1\}) (2) notify(BulkLoadListeners) (3) ZK(add HFile entries, no cyclic loop -> replicate to c3) \[return to the client, client will delete the znode\] (4) ReplicateEntries(entries, source cluster ids) --> Back to step (1). Replication process at cluster c3, (1) doBulkLoad(HFiles,{c1,c2\}) (2) notify(BulkLoadListeners) (3) ZK(add HFile entries, cyclic loop detected) \[return to the client, client will delete the znode\] Cyclic loop ends here Hope I am able to clear your questions. > enable bulkload to support replication > -------------------------------------- > > Key: HBASE-13153 > URL: https://issues.apache.org/jira/browse/HBASE-13153 > Project: HBase > Issue Type: New Feature > Components: Replication > Reporter: sunhaitao > Assignee: Ashish Singhi > Fix For: 2.0.0 > > Attachments: HBase Bulk Load Replication.pdf > > > Currently we plan to use HBase Replication feature to deal with disaster > tolerance scenario.But we encounter an issue that we will use bulkload very > frequently,because bulkload bypass write path, and will not generate WAL, so > the data will not be replicated to backup cluster. It's inappropriate to > bukload twice both on active cluster and backup cluster. So i advise do some > modification to bulkload feature to enable bukload to both active cluster and > backup cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)