[ https://issues.apache.org/jira/browse/HBASE-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152757#comment-13152757 ]
Hudson commented on HBASE-2742: ------------------------------- Integrated in HBase-0.92 #143 (See [https://builds.apache.org/job/HBase-0.92/143/]) HBASE-2742 Provide strong authentication with a secure RPC engine garyh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/conf/hbase-policy.xml * /hbase/branches/0.92/pom.xml * /hbase/branches/0.92/security * /hbase/branches/0.92/security/src * /hbase/branches/0.92/security/src/main * /hbase/branches/0.92/security/src/main/java * /hbase/branches/0.92/security/src/main/java/org * /hbase/branches/0.92/security/src/main/java/org/apache * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/ipc * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureClient.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureConnectionHeader.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureServer.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/AccessDeniedException.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/HBasePolicyProvider.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcClient.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcServer.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationKey.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationProtocol.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenIdentifier.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenSecretManager.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token/AuthenticationTokenSelector.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token/TokenProvider.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java * /hbase/branches/0.92/security/src/main/java/org/apache/hadoop/hbase/security/token/ZKSecretWatcher.java * /hbase/branches/0.92/security/src/test * /hbase/branches/0.92/security/src/test/java * /hbase/branches/0.92/security/src/test/java/org * /hbase/branches/0.92/security/src/test/java/org/apache * /hbase/branches/0.92/security/src/test/java/org/apache/hadoop * /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase * /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security * /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/token * /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java * /hbase/branches/0.92/security/src/test/java/org/apache/hadoop/hbase/security/token/TestZKSecretWatcher.java * /hbase/branches/0.92/security/src/test/resources * /hbase/branches/0.92/security/src/test/resources/hbase-site.xml * /hbase/branches/0.92/src/assembly/all.xml * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HServerAddress.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/ConnectionHeader.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcMetrics.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/RequestContext.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/security/KerberosInfo.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/security/TokenInfo.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/security/User.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKLeaderManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java * /hbase/branches/0.92/src/main/resources/hbase-default.xml * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/zookeeper/TestZKLeaderManager.java > Provide strong authentication with a secure RPC engine > ------------------------------------------------------ > > Key: HBASE-2742 > URL: https://issues.apache.org/jira/browse/HBASE-2742 > Project: HBase > Issue Type: Improvement > Components: ipc > Reporter: Gary Helmling > Assignee: Gary Helmling > Priority: Critical > Fix For: 0.92.0 > > Attachments: HBASE-2742_10.patch > > > The HBase RPC code (org.apache.hadoop.hbase.ipc.*) was originally forked off > of Hadoop RPC classes, with some performance tweaks added. Those > optimizations have come at a cost in keeping up with Hadoop RPC changes > however, both bug fixes and improvements/new features. > In particular, this impacts how we implement security features in HBase (see > HBASE-1697 and HBASE-2016). The secure Hadoop implementation (HADOOP-4487) > relies heavily on RPC changes to support client authentication via kerberos > and securing and mutual authentication of client/server connections via SASL. > Making use of the built-in Hadoop RPC classes will gain us these pieces for > free in a secure HBase. > So, I'm proposing that we drop the HBase forked version of RPC and convert to > direct use of Hadoop RPC, while working to contribute important fixes back > upstream to Hadoop core. Based on a review of the HBase RPC changes, the key > divergences seem to be: > HBaseClient: > - added use of TCP keepalive (HBASE-1754) > - made connection retries and sleep configurable (HBASE-1815) > - prevent NPE if socket == null due to creation failure (HBASE-2443) > HBaseRPC: > - mapping of method names <-> codes (removed in HBASE-2219) > HBaseServer: > - use of TCP keep alives (HBASE-1754) > - OOME in server does not trigger abort (HBASE-1198) > HbaseObjectWritable: > - allows List<> serialization > - includes it's own class <-> code mapping (HBASE-328) > Proposed process is: > 1. open issues with patches on Hadoop core for important fixes/adjustments > from HBase RPC (HBASE-1198, HBASE-1815, HBASE-1754, HBASE-2443, plus a > pluggable ObjectWritable implementation in RPC.Invocation to allow use of > HbaseObjectWritable). > 2. ship a Hadoop version with RPC patches applied -- ideally we should avoid > another copy-n-paste code fork, subject to ability to isolate changes from > impacting Hadoop internal RPC wire formats > 3. if all Hadoop core patches are applied we can drop back to a plain vanilla > Hadoop version > I realize there are many different opinions on how to proceed with HBase RPC, > so I'm hoping this issue will kick off a discussion on what the best approach > might be. My own motivation is maximizing re-use of the authentication and > connection security work that's already gone into Hadoop core. I'll put > together a set of patches around #1 and #2, but obviously we need some > consensus around this to move forward. If I'm missing other differences > between HBase and Hadoop RPC, please list as well. Discuss! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira