The problem for putting more things in meta is that, the row key pattern are different. For example, when re-implementing the serial replication feature, the 'replication barrier', which is actually the sequence of the open sequence number for a region, is stored in meta as the row key can the region name, so it is OK.
But for other stuffs for replication, such as the WAL files we need to replicate, the row key will be a peer id, and multiplies the server name, and maybe also the file name? This is a completely different thing. If we put this in meta, the row key will be messed up... 2018-03-16 3:01 GMT+08:00 Stack <st...@duboce.net>: > On Wed, Mar 14, 2018 at 11:55 PM, OpenInx <open...@gmail.com> wrote: > > > Hi : > > > > (Paste from https://issues.apache.org/jira/browse/HBASE-20166? > > focusedCommentId=16399886&page=com.atlassian.jira. > > plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16399886) > > > > There's a really big problem here if we use table based replication to > > start a hbase cluster: > > > > For HMaster process, it works as following: > > 1. Start active master initialization . > > 2. Master wait rs report in . > > 3. Master assign meta region to one of the region servers . > > 4. Master create hbase:replication table if not exist. > > > > > We have to have a new system table? Can't we add a column family on > hbase:meta that keeps offsets? We've done the work to make sure hbase:meta > is up before everything else. It has its own WALs so we can split these > ahead of user-space WALs, and so on. We've not done the work to for > hbase:replication or hbase:namespace, hbase:acl... etc. > > Means more loading on hbase:meta and it is going to get bigger but I'd > rather work on splitting meta than on figuring how to preassign > miscellaneous system tables; one-per-feature. > > > > > But the RS need to finish initialize the replication source & sink before > > finish startup( and the initialization of replication source & sink must > > finish before opening region, because we need to listen the wal event, > > otherwise our replication may lost data), and when initialize the source > & > > sink , we need to read hbase:replication table which hasn't been avaiable > > because our master is waiting rs to be OK, and the rs is waiting > > hbase:replication to be OK ... a dead loop happen again ... > > > > After discussed with Guanghao Zhang offline, I'm considering that try to > > assign all system table to a rs which only accept regions of system table > > assignment (The rs will skip to initialize the replication source or sink > > )... > > > > > Can we avoid this sort of special-casing? > > St.Ack > > > > > I've tried to start a mini cluster by setting > > hbase.balancer.tablesOnMaster.systemTablesOnly=true > > & hbase.balancer.tablesOnMaster=true , it seems not work. because > > currently > > we initialize the master logic firstly, then region logic for the HMaster > > process, and it should be ... > > > > > > Any suggestion ? > > >
