For more information on how replication works, please see http://hbase.apache.org/replication.html
Also regarding the tool, I don't have time right now to work on it, so don't expect something soon unless you want to work on it :) Have a good weekend, J-D On Fri, Jul 26, 2013 at 1:29 PM, Demai Ni <[email protected]> wrote: > JD, thanks for the jira and further explanation. for some reason, I was > always thinking about 'pull' while considering the solution. Certainly > 'push' the natural way to address this on Apache's hbase. I got your > points now. Appreciate it... Demai > > > On Fri, Jul 26, 2013 at 1:08 PM, Jean-Daniel Cryans > <[email protected]>wrote: > >> https://issues.apache.org/jira/browse/HBASE-9047 >> >> On Fri, Jul 26, 2013 at 12:59 PM, Jean-Daniel Cryans >> <[email protected]> wrote: >> > I guess I didn't explain my ideas clearly. >> > >> > So, first, replication in HBase is master-push, so you don't want to >> > reverse the process. It means that this tool needs to run on the >> > master cluster. >> > >> > Then I don't think you need to specify a timestamp since the >> > replication state is in ZK. Basically that tool we're talking about >> > would be able to read the replication state of each master region >> > server, finish replicating what's missing, and then clear that state >> > in zookeeper. >> > >> > The code that handles replication does most of that already. Check >> > ReplicationSourceManager and ReplicationSource. Basically when >> > ReplicationSourceManager.init() is called, it will check all the >> > queues in ZK and try to grab those that aren't attached to a region >> > server. If the whole cluster is down, it will grab all of them. >> > >> > The beautiful thing here is that you could start that tool on all your >> > machines and the load will be spread out, but that might not be a big >> > concern if replication wasn't lagging since it would take a few >> > seconds to finish replicating the missing data for each region server. >> > >> > I'll open a jira. >> > >> > J-D >> > >> > On Fri, Jul 26, 2013 at 11:50 AM, Demai Ni <[email protected]> wrote: >> >> JD, >> >> >> >> yeah. that sounds what I will need to do. a tool like this >> >> [slave_cluster]$tool_to_syncup master_ZKquorum table_name >> start_timestamp >> >> >> >> so two tasks for me: >> >> 1) identify the start_timestamp >> >> 2) write the tool_to_syncup which will reach to master_ZK, copy the >> HLOGs >> >> from makster, replay the HLOGs on Slave. >> >> >> >> are you aware of some example code for the 2) task that I can leverage? >> >> thanks >> >> >> >> Demai >>
