-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/76/
-----------------------------------------------------------
(Updated 2010-05-28 16:02:44.219338)
Review request for hbase.
Changes
-------
The remaining issues and some resolutions:
ReplicationSink.java
- I think it would be good to document the fact that this method will typically
be called from another thread than the thread that executes `run' so that other
people reading the code will quickly get a good grasp of what are the
concurrency / locking requirements.
I chatted with Benoit on this issue, the problem with ReplicationSink at the
moment is it's a mix of two solutions: sync and async. It's sync'ed because the
thread that writes the log file has to wait for the ReplicationSink to apply
it… but instead it could work like the ReplicationSource and leave it up to
ZK-based management of log files. It currently work as is, but it's not
efficient. I can fix this in the scope of this jira, which is still named
"Handle 10min+ network partitions between clusters", or in a later one.
- So Delete operations are "unbuffered" unlike Put operations, which you
"buffer" in the `puts' list. Does that mean that a Delete can be executed
before the Put that was creating the data in the first place, and that the
Delete will fail first and the Put will survive second?
Created a test that inserts 2 rows, deletes the second one, and adds two more.
I expect to see the row as deleted, and it works (yeah!). Included in this patch
// Should we log rejected edits in a file for replay?
- I vote yes
This will be in HBASE-2626.
ReplicationSource.java
- This `try' block is massive, would it be possible to refactor it using a
private method to make the code a bit more readable?
Did some refactoring, but still not satisfied.
Summary
-------
This is HBASE-2223 AKA Replication 2.0, it is currently only a "preview patch"
as it's pretty much feature complete, works on a cluster, has unit tests and
whatnot, but it could use a lot more testing and cleaning and ideas from others.
This addresses bug HBASE-2223.
http://issues.apache.org/jira/browse/HBASE-2223
Diffs (updated)
-----
bin/replication/add_peer.rb PRE-CREATION
bin/replication/copy_tables_desc.rb PRE-CREATION
pom.xml 0a009cf
src/main/java/org/apache/hadoop/hbase/HConstants.java 13aff26
src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 4cbe52a
src/main/java/org/apache/hadoop/hbase/master/ServerManager.java a197b8f
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java b5ff43a
src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 12a3cd8
src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java 7c1184c
src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeperHelper.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/replication/package.html PRE-CREATION
src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
PRE-CREATION
src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
PRE-CREATION
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ed8709f
src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
PRE-CREATION
src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java
PRE-CREATION
src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSource.java
PRE-CREATION
src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSink.java
PRE-CREATION
src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
PRE-CREATION
Diff: http://review.hbase.org/r/76/diff
Testing
-------
Thanks,
Jean-Daniel
