> For Kerberos though it isn't clear to me how to do good > integration testing since we need a KDC to test against and it isn't clear > how that happens in the test environment except possibly manually (which is > not ideal). How do other projects handle this?
Actually it’s not that hard. Hadoop is providing small KDC implementation for testing purpose called MiniKdc. It’s super easy to use, check it out in Sentry: https://github.com/apache/incubator-sentry/blob/master/sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/thrift/SentryMiniKdcTestcase.java Jarcec On Oct 9, 2014, at 1:44 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > Hey Gwen, > > Your absolutely right about these. I added the ticket for ZK authentication > and Hadoop delegation tokens. > > For the Hadoop case I actually don't understand Hadoop security very well. > Maybe you could fill in some of the details on what needs to happen for > that to work? > > For testing, we should probably discuss the best way to test security. I > think this is a fairly critical thing, if we are going to say we have > security we really need to have good tests in place to ensure we do. This > will require some thought. I think we should be able to test TLS fairly > easily using junit integration test that just starts the server and > connects using TLS. For Kerberos though it isn't clear to me how to do good > integration testing since we need a KDC to test against and it isn't clear > how that happens in the test environment except possibly manually (which is > not ideal). How do other projects handle this? > > -Jay > > On Tue, Oct 7, 2014 at 5:25 PM, Gwen Shapira <gshap...@cloudera.com> wrote: > >> I think we need to add: >> >> * Authentication of Kafka brokers with a secured ZooKeeper >> * Kafka should be able to generate delegation tokens for MapReduce / >> Spark / Yarn jobs. >> * Extend systest framework to allow testing secured kafka >> >> Gwen >> >> On Tue, Oct 7, 2014 at 5:15 PM, Jay Kreps <jay.kr...@gmail.com> wrote: >>> Hey guys, >>> >>> As promised, I added a tree of JIRAs for the stuff in the security wiki ( >>> https://cwiki.apache.org/confluence/display/KAFKA/Security): >>> >>> https://issues.apache.org/jira/browse/KAFKA-1682 >>> >>> I tried to break it into reasonably standalone pieces. I think many of >> the >>> tickets could actually be done in parallel. Since there were many people >>> interested in this area this may help parallelize the work a bit. >>> >>> I added some strawman details on implementation to each ticket. We can >>> discuss and refine further on the individual tickets. >>> >>> Please take a look and let me know if this breakdown seems reasonable. >>> >>> Cheers, >>> >>> -Jay >>
