[ https://issues.apache.org/jira/browse/HBASE-26913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629946#comment-17629946 ]
Rushabh Shah commented on HBASE-26913: -------------------------------------- > After looking at the current PR, I think maybe we could introduce a special >region info, for handling the region server level markers, for example, let's >call the table 'hbase:replication_marker_placeholder', and we will always use >this table's first region info, i.e, creating by >RegionInfoBuilder.newBuilder(tableName).build(), to write region server level >markers. And it will be replicated to remote peers, but when splitting, we >will just drop it, which is almost the same with the current implementation. In this case also, we will create an edit with a region which will not reside on any region server. In the first proposal, we were re-using an existing table we created for this framework instead of creating yet another table. [~zhangduo] Am I missing something? > Replication Observability Framework > ----------------------------------- > > Key: HBASE-26913 > URL: https://issues.apache.org/jira/browse/HBASE-26913 > Project: HBase > Issue Type: New Feature > Components: regionserver, Replication > Reporter: Rushabh Shah > Assignee: Rushabh Shah > Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4 > > > In our production clusters, we have seen cases where data is present in > source cluster but not in the sink cluster and 1 case where data is present > in sink cluster but not in source cluster. > We have internal tools where we take incremental backup every day on both > source and sink clusters and we compare the hash of the data in both the > backups. We have seen many cases where hash doesn't match which means data is > not consistent between source and sink for that given day. The Mean Time To > Detect (MTTD) these inconsistencies is atleast 2 days and requires lot of > manual debugging. > We need some tool where we can reduce MTTD and requires less manual debugging. > I have attached design doc. Huge thanks to [~bharathv] to come up with this > design at my work place. -- This message was sent by Atlassian Jira (v8.20.10#820010)