[ https://issues.apache.org/jira/browse/HBASE-5869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264372#comment-13264372 ]
jirapos...@reviews.apache.org commented on HBASE-5869: ------------------------------------------------------ ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4926/ ----------------------------------------------------------- Review request for hbase and Jimmy Xiang. Summary ------- Convert two zk users to pb: distributed log splitting and regions in transition. Refactored distributed log splitting so we only serialize/deserialize in one location. Less changes needed to do same for regions in transition. Moves serialization/deserialization out of the ZKAssign, ZKSplit and into the classes themselves so can encapsulate how serialization is done into one place (try to make the ZK* classes just deal in bytes -- about 90% done). Moved classes used by various packages up to top level to minimize imports that are across package (zookeeper into protobuf and/or into regionserver and/or master packages, etc). A src/main/java/org/apache/hadoop/hbase/DeserializationException.java New generic deserialization exception. A src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java D src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java Moved under zookeeper package. A src/main/java/org/apache/hadoop/hbase/HBaseException.java New base hbase exception as suggested by hbase-5796. New DeserializationException inherits from this. A src/main/java/org/apache/hadoop/hbase/RegionTransition.java State of a region in transition. Top-level because used by a few top-level packages. Encapsulates pb serialization/deserialization. M src/main/java/org/apache/hadoop/hbase/ServerName.java Add method to deserialize a ServeName, etc. Encapsulates pb'ing. M src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java Counters used by distributed log splitting. A SplitLogTask Class that encapsulates log splitting state. Also encapsulates pb'ing. M src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java Implement code for state. Added functions to go from code to state and vice versa. Used serializing. M src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java Remove unused imports. D src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java Removed. Replaced by RegionTransition moved to package top-level. M src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java Use new DeserializationException. Move to using new RegionTransition from RegionTransitionData class. Pass deserialized class rather than byte array. Remove duplicated code. M src/main/java/org/apache/hadoop/hbase/master/HMaster.java Use new ServerName parse method rather than ZKUtil one. M src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java M src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java Redo to use new SplitLogTask and SplitLogCounter classes. M src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java expectPBMagicPrefix added M src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java Use new RegionTransition in place of RegionTransitionData. M src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java Define moved from ZKSplitLog to SplitLogManager. M src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java M src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java Changed method name from getZNodeData to toByteArray to match how we've named it elsewhere. Use new DeserializationException M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java Use new RegionTransion class M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java Moved stuff that was in here up into SplitLogManager where better belongs. Also moved serialization/deserialization up into the class itself: SplitLogTask. Moved counters out to SplitLogCounter class. M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java Moved deserialization of ServerName out of here and up into ServerName. M src/main/protobuf/ZooKeeper.proto Add two new classes, RegionTransition and SplitLogTask. This addresses bug HBASE-5869. https://issues.apache.org/jira/browse/HBASE-5869 Diffs ----- src/main/java/org/apache/hadoop/hbase/DeserializationException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java 9881ec2 src/main/java/org/apache/hadoop/hbase/HBaseException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/RegionTransition.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/ServerName.java 8fdb624 src/main/java/org/apache/hadoop/hbase/SplitLogCounters.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/SplitLogTask.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 4121508 src/main/java/org/apache/hadoop/hbase/executor/ExecutorService.java 06ca377 src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java 35d7b70 src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 47e3bd6 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java f56127d src/main/java/org/apache/hadoop/hbase/master/HMaster.java 81e9023 src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 919c65f src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 994cb76 src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 8457bdc src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java 8ea342f src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java ea12da4 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java 587386c src/main/java/org/apache/hadoop/hbase/zookeeper/EmptyWatcher.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java f9575af src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java babde80 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java e94b672 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java 30d7fe9 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 46a6fde src/main/protobuf/ZooKeeper.proto 961ab65 src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java c3a1889 src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java d496d48 src/test/java/org/apache/hadoop/hbase/master/Mocking.java 676d6bb src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 36046f8 src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 2669876 src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 14cdb90 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java f8029ba src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java 0f7d54e src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 26b9865 src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 75b5aea src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestCloseRegionHandler.java 07f8fc4 src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java 55a8c4a src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java 4314572 Diff: https://reviews.apache.org/r/4926/diff Testing ------- Thanks, Michael > Move SplitLogManager splitlog taskstate and AssignmentManager > RegionTransitionData znode datas to pb > ----------------------------------------------------------------------------------------------------- > > Key: HBASE-5869 > URL: https://issues.apache.org/jira/browse/HBASE-5869 > Project: HBase > Issue Type: Task > Reporter: stack > Attachments: firstcut.txt, secondcut.txt, v4.txt, v5.txt, v6.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira