----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29386/ -----------------------------------------------------------
(Updated Dec. 29, 2014, 10:11 p.m.) Review request for accumulo. Changes ------- Addressed some more things from the existing reviews. Found a place in TServerUtils that wasn't using the UGIAssumingProcessor as it should've been which was found after changing the TCredentialsUpdatingInvocationHandler with Keith's suggestions (good call). I found an issue with the serializing of the KerberosToken and SecurityOperations that I'm currently looking into. More work incoming, but wanted to update with changes that I already made. Bugs: ACCUMULO-2815 https://issues.apache.org/jira/browse/ACCUMULO-2815 Repository: accumulo Description ------- ACCUMULO-2815 Initial support for Kerberos client authentication. Leverage SASL transport provided by Thrift which can speak GSSAPI, which Kerberos implements. Introduced... * An Accumulo KerberosToken which is an AuthenticationToken to validate users. * Custom thrift processor and invocation handler to ensure server RPCs have a valid KRB identity and Accumulo authentication. * A KerberosAuthenticator which extends ZKAuthenticator to support Kerberos identities seamlessly. * New ClientConf variables to use SASL transport and pass Kerberos server principal * Updated ClientOpts and Shell opts to transparently use a KerberosToken when SASL is enabled (no extra client work). I believe this is the "bare minimum" for Kerberos support. They are also grossly lacking in unit and integration tests. I believe that I might have somehow broken the client address string in the server (I saw log messages with client: null, but I'm not sure if it's due to these changes or not). A necessary limitation in the Thrift server used is that, like the SSL transport, the SASL transport cannot presently be used with the TFramedTransport, which means none of the [half]async thrift servers will function with this -- we're stuck with the TThreadPoolServer. Performed some contrived benchmarks on my laptop (while still using it myself) to get at big-picture view of the performance impact against "normal" operation and Kerberos alone. Each "run" was the duration to ingest 100M records using continuous-ingest, timed with `time`, using 'real'. THsHaServer (our default), 6 runs: Avg: 10m7.273s (607.273s) Min: 9m43.395s Max: 10m52.715s TThreadPoolServer (no SASL), 5 runs: Avg: 11m16.254s (676.254s) Min: 10m30.987s Max: 12m24.192s TThreadPoolServer+SASL/GSSAPI (these changes), 6 runs: Avg: 13m17.187s (797.187s) Min: 10m52.997s Max: 16m0.975s The general takeway is that there's about 15% performance degredation in its initial state which is in the realm of what I expected (~10%). Diffs (updated) ----- core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java f6ea934 core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java 6fe61a5 core/src/main/java/org/apache/accumulo/core/client/impl/ClientContext.java e75bec6 core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java f481cc3 core/src/main/java/org/apache/accumulo/core/client/impl/ThriftTransportKey.java 6dc846f core/src/main/java/org/apache/accumulo/core/client/impl/ThriftTransportPool.java 5da803b core/src/main/java/org/apache/accumulo/core/client/security/tokens/KerberosToken.java PRE-CREATION core/src/main/java/org/apache/accumulo/core/conf/Property.java e054a5f core/src/main/java/org/apache/accumulo/core/rpc/FilterTransport.java PRE-CREATION core/src/main/java/org/apache/accumulo/core/rpc/SaslConnectionParams.java PRE-CREATION core/src/main/java/org/apache/accumulo/core/rpc/TTimeoutTransport.java 6eace77 core/src/main/java/org/apache/accumulo/core/rpc/ThriftUtil.java 09bd6c4 core/src/main/java/org/apache/accumulo/core/rpc/UGIAssumingTransport.java PRE-CREATION core/src/main/java/org/apache/accumulo/core/rpc/UGIAssumingTransportFactory.java PRE-CREATION core/src/main/java/org/apache/accumulo/core/security/Credentials.java 525a958 proxy/src/main/java/org/apache/accumulo/proxy/Proxy.java 4b048eb server/base/src/main/java/org/apache/accumulo/server/AccumuloServerContext.java 09ae4f4 server/base/src/main/java/org/apache/accumulo/server/init/Initialize.java 046cfb5 server/base/src/main/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingInvocationHandler.java PRE-CREATION server/base/src/main/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingWrapper.java PRE-CREATION server/base/src/main/java/org/apache/accumulo/server/rpc/TServerUtils.java 641c0bf server/base/src/main/java/org/apache/accumulo/server/rpc/ThriftServerType.java PRE-CREATION server/base/src/main/java/org/apache/accumulo/server/security/SecurityOperation.java 5e81018 server/base/src/main/java/org/apache/accumulo/server/security/SystemCredentials.java a59d57c server/base/src/main/java/org/apache/accumulo/server/security/handler/KerberosAuthenticator.java PRE-CREATION server/base/src/main/java/org/apache/accumulo/server/thrift/UGIAssumingProcessor.java PRE-CREATION server/base/src/test/java/org/apache/accumulo/server/security/SystemCredentialsTest.java 4202a7e server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 93a9a49 server/gc/src/test/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogsTest.java f98721f server/gc/src/test/java/org/apache/accumulo/gc/SimpleGarbageCollectorTest.java 99558b8 server/gc/src/test/java/org/apache/accumulo/gc/replication/CloseWriteAheadLogReferencesTest.java cad1e01 server/master/src/main/java/org/apache/accumulo/master/Master.java 12195fa server/tracer/src/main/java/org/apache/accumulo/tracer/TraceServer.java 7e33300 server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServer.java d5c1d2f shell/src/main/java/org/apache/accumulo/shell/Shell.java 58308ff shell/src/main/java/org/apache/accumulo/shell/ShellOptionsJC.java 8167ef8 test/src/main/java/org/apache/accumulo/test/functional/ZombieTServer.java eb84533 test/src/main/java/org/apache/accumulo/test/performance/thrift/NullTserver.java 2ebc2e3 test/src/test/java/org/apache/accumulo/server/security/SystemCredentialsIT.java fb71f5f Diff: https://reviews.apache.org/r/29386/diff/ Testing ------- Ensure existing unit tests still function. Accumulo is functional and ran continuous ingest multiple times using a client with only a Kerberos identity (no user/password provided). Used MIT Kerberos with Apache Hadoop 2.6.0 and Apache ZooKeeper 3.4.5. Thanks, Josh Elser