Following on from the discussions in the KIP meeting today, the suggestion is to implement a cut-down version of KIP-43 for 0.10.0.0 with a follow-on KIP after the release to address support for custom mechanisms.
Changes to be removed from KIP-43: 1. Remove the configuration for CallbackHandler. The callback handler implementation in Kafka will support Kerberos, PLAIN and Digest-MD5. It will not support custom or more complex mechanisms which require additional callbacks. 2. Remove the configuration for Login. The Login implementation in Kafka will support Kerberos and any other mechanism (PLAIN, Digest-MD5 etc) that doesn't require functionality like token refresh. Changes included in KIP-43: 1. Configurable mechanism 2. Support for multiple mechanisms in the broker 3. Implementation of SASL/PLAIN If there are no objections to this, I can update the KIP and the PR by tomorrow. And move the support for custom mechanisms into another KIP and PR for review after the release of 0.10.0.0. On Mon, Mar 14, 2016 at 7:48 AM, Rajini Sivaram < rajinisiva...@googlemail.com> wrote: > Harsha, > > You are right, we don't expect to override callback handler or login for > Digest-MD5. > > Pluggable CallbackHandler and Login modules enable custom SASL mechanisms > to be implemented without modifying Kafka. For instance, it would enable > KIP-44 ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-44+-+Allow+Kafka+to+have+a+customized+security+protocol) > to be implemented without making the whole security protocol pluggable. Tao > Xiao has already confirmed earlier in this discussion thread that the > proposed callback handler and login interfaces are suitable for their > custom authentication. > > > > On Sun, Mar 13, 2016 at 6:59 PM, Harsha <ka...@harsha.io> wrote: > >> Agree with Gwen here. I feel like these additional pluggable Login >> Modules are making this KIP complex. Since the main goal of the KIP is >> to enable additional mechanism , can we limit the scope to that and If >> we feel necessary for pluggable Login and callback handler classes we >> can address in another JIRA. >> >> Adding digest-md5 ,password callbacks can be done to existing >> callbackhandler without expose it as pluggable class. It would be >> useful to have broker support multiple mechanisms. I haven't seen >> anyone using more than this in hadoop . It might be different for Kafka >> but I personally haven't seen anyone asking for this yet. >> >> Thanks, >> Harsha >> >> >> On Thu, Mar 10, 2016, at 01:44 AM, Rajini Sivaram wrote: >> > Gwen, >> > >> > Just to be clear, the alternative would be: >> > >> > *jaas.conf:* >> > >> > GssapiKafkaServer { >> > >> > com.ibm.security.auth.module.Krb5LoginModule required >> > credsType=both >> > useKeytab="file:/kafka/key.tab" >> > principal="kafka/localh...@example.com <http://example.com/>"; >> > >> > }; >> > >> > SmartcardKafkaServer { >> > >> > example.SmartcardLoginModule required >> > >> > cardNumber=123; >> > >> > }; >> > >> > >> > *KafkaConfig* >> > >> > >> > >> > - login.context.map={"GSSAPI="GssapiKafkaServer", >> > "SMARTCARD"=SmartcardKafkaServer} >> > - login.class.map={"GSSAPI=GssapiLogin.class, >> > "SMARTCARD"=SmartcardLogin.class} >> > - >> callback.handler.class.map={"GSSAPI"=GssapiCallbackHandler.class, >> > "SMARTCARD"=SmartcardCallbackHandler.class} >> > >> > *Client Config * >> > Same as the server, but with only one entry allowed in each map and >> > jaas.conf >> > >> > >> > >> > This is a different model from the Java standard for supporting multiple >> > logins. As a developer, I am inclined to stick with approaches that are >> > widely in use like JSSE. But this alternative can be made to work if the >> > Kafka community feels it is more appropriate for Kafka. If you know of >> > other systems which use this approach, that would be helpful. >> > >> > >> > >> > On Thu, Mar 10, 2016 at 2:07 AM, Gwen Shapira <g...@confluent.io> >> wrote: >> > >> > > What I'm hearing is that: >> > > >> > > 1. In order to support authentication mechanisms that were not written >> > > specifically with Kafka in mind, someone will need to write the >> > > integration between the mechanism and Kafka. This may include Login >> > > and CallbackHandler classes. This can be the mechanism vendor, the >> > > user or a 3rd party vendor. >> > > 2. If someone wrote the code to support a mechanism in Kafka, and a >> > > user will want to use more than one mechanism, they will still need to >> > > write a wrapper. >> > > 3. In reality, #2 will not be necessary ("edge-case") because Kafka >> > > will actually already provide the callback needed (and presumably also >> > > the code to load the LoginModule provided by Example.com)? >> > > >> > > Tradeoff #1 sounds reasonable. >> > > #2 and #3 do not sound reasonable considering one of the goals of the >> > > patch is to support multiple mechanisms. I don't think we should force >> > > our users to write code just to avoid writing it ourselves. >> > > Configuring security is complex enough as is. >> > > Furthermore, if we believe that "Smartcard is likely to use standard >> > > NameCallback and PasswordCallback already implemented in Kafka" - why >> > > do we even provide configuration for Login and CallbackHandler >> > > classes? Either we support multiple mechanisms written by different >> > > vendors, or we don't. >> > > >> > > Gwen >> > > >> > > >> > > On Wed, Mar 9, 2016 at 12:32 AM, Rajini Sivaram >> > > <rajinisiva...@googlemail.com> wrote: >> > > > I am not saying that the developer at Example Inc. would develop a >> Login >> > > > implementation that combines Smartcard and Kerberos because >> Retailer uses >> > > > both. I am saying that Example Inc develops the LoginModule >> (similar to >> > > JVM >> > > > security providers developing Kerberos modules). But there is no >> standard >> > > > interface for Login to allow ticket refresh. So, it is very >> unlikely that >> > > > Example Inc would develop a Login implementation that works with an >> > > Apache >> > > > Kafka defined interface ( Kafka developers wrote this code for >> Kerberos). >> > > > For a custom integration, the user (i.e. Retailer) would be >> expected to >> > > > develop this code if required. >> > > > >> > > > You could imagine that Smartcard is a commonly used mechanism and a >> 3rd >> > > > party develops code for integrating Smartcard with Kafka and makes >> the >> > > > integration code (Login and CallbackHandler implementation) widely >> > > > available, If Retailer wants to use clients or a broker with just >> > > Smartcard >> > > > enabled in their broker, they configure Kafka to use the 3rd party >> code, >> > > > with no additional code development. But to combine Smartcard and >> > > Kerberos, >> > > > Retailer needs to write a few lines of code to incorporate both >> Smartcard >> > > > and Kerberos. I believe this is an edge case. >> > > > >> > > > Smartcard is likely to use standard NameCallback and >> PasswordCallback >> > > > already implemented in Kafka and Kerberos support exists in Kafka. >> So it >> > > is >> > > > very likely that Retailer doesn't need to override Login or >> > > CallbackHandler >> > > > in this case. And it would just be a matter of configuring the >> > > mechanisms. >> > > > >> > > > On Wed, Mar 9, 2016 at 12:48 AM, Gwen Shapira <g...@confluent.io> >> wrote: >> > > > >> > > >> "Since smart card logins are not built into Kafka (or the JDK), you >> > > need a >> > > >> developer to build the login module. So the developer implements >> > > >> example.SmartcardLoginModule. In addition, the developer may also >> > > implement >> > > >> callback handlers for the SASL client or server and a login class >> to >> > > keep >> > > >> this login refreshed. The callback handlers and login >> implementation >> > > >> support all the mechanisms that the organisation supports - in >> this case >> > > >> Kerberos and smart card." >> > > >> >> > > >> In this case, the developer works for Example Inc (which develops >> > > >> SmartCard authentication modules), while I work for Retailer and >> need >> > > >> to use his module. >> > > >> You assume that developer from Example Inc knows about all the >> > > >> mechanisms that I have enabled in my Kafka cluster and he is >> capable >> > > >> of developing a Login class and maybe a callback handler that >> supports >> > > >> all of them. I think this is an unreasonable expectation. >> > > >> >> > > >> I think that allowing plug in of Login method and single Callback >> > > >> handler is not a good extensible solution for multiple mechanisms. >> We >> > > >> should probably supply a Kafka-based LoginContext and >> CallbackHandler >> > > >> that will be able to delegate to implementations of LoginModules >> and >> > > >> Callback Handlers based on mechanism out of a list supplied by a >> user. >> > > >> I agree it is more complex, but otherwise we are asking of too much >> > > >> coordination from whoever implements CallbackHandlers and Login >> > > >> methods. >> > > >> >> > > >> Gwen >> > > >> >> > > >> On Tue, Mar 8, 2016 at 3:35 PM, Rajini Sivaram >> > > >> <rajinisiva...@googlemail.com> wrote: >> > > >> > As an admin, you would configure login modules using standard >> JAAS >> > > >> > configuration. So, if you are supporting Kerberos and smart card >> login >> > > >> > modules within a single broker, in your jaas.conf you would >> specify: >> > > >> > >> > > >> > KafkaServer { >> > > >> > >> > > >> > com.ibm.security.auth.module.Krb5LoginModule required >> > > >> > credsType=both >> > > >> > useKeytab="file:/kafka/key.tab" >> > > >> > principal="kafka/localh...@example.com"; >> > > >> > >> > > >> > example.SmartcardLoginModule required >> > > >> > >> > > >> > cardNumber=123 >> > > >> > >> > > >> > }; >> > > >> > >> > > >> > >> > > >> > Since smart card logins are not built into Kafka (or the JDK), >> you >> > > need a >> > > >> > developer to build the login module. So the developer implements >> > > >> > example.SmartcardLoginModule. In addition, the developer may also >> > > >> implement >> > > >> > callback handlers for the SASL client or server and a login >> class to >> > > >> keep >> > > >> > this login refreshed. The callback handlers and login >> implementation >> > > >> > support all the mechanisms that the organisation supports - in >> this >> > > case >> > > >> > Kerberos and smart card. >> > > >> > >> > > >> > As an admin, if you are using mechanisms built into Kafka, you >> simply >> > > >> > specify a JAAS config with the combination of mechanisms you >> want and >> > > you >> > > >> > are done. If you are adding new mechanisms, it clearly can't be >> done >> > > >> > without some additional code to implement that mechanism. The >> callback >> > > >> and >> > > >> > login interfaces are configurable to ensure that (as a >> developer), you >> > > >> can >> > > >> > build custom mechanisms without modifying Kafka. SASL was >> designed >> > > >> > precisely to do this. But at the moment, Kafka restricts what >> type of >> > > >> > authentication you can perform with SASL. >> > > >> > >> > > >> > Back to the questions: >> > > >> > 1. *How does the proposed Login class interact with LoginContext >> and * >> > > >> > *LoginModule.* >> > > >> > Login class performs *LoginContext.login(). *This performs login >> for >> > > all >> > > >> > the login modules of that context (this is standard Java >> function). >> > > Login >> > > >> > is cached with a reference counter to keep it alive. >> > > >> > >> > > >> > 2. *Do we really need a pluggable Login class, when users >> > > >> **(administrators, >> > > >> > not developers) are more likely to want to specify LoginModules?* >> > > >> > Administrators specify *LoginModules. *Developers implement new >> > > >> mechanisms. >> > > >> > The pluggable *Login* class enables the developer to implement >> new >> > > >> > mechanisms without changing Kafka. If Kafka didn't have a default >> > > >> Kerberos >> > > >> > implementation and a user wanted to use Kerberos, pluggability of >> > > >> *Login* class >> > > >> > enables the user to include Kerberos without changing Kafka. Not >> that >> > > we >> > > >> > want another Kerberos, but Tao had a requirement to enable a >> custom >> > > >> > mechanism with functionality similar to Kerberos (see KIP-44). >> > > Pluggable >> > > >> > *Login* supports this use case without replacing the entire >> > > >> authentication >> > > >> > mechanism in Kafka. >> > > >> > >> > > >> > *3. How will users get to specify multiple LoginModules?* >> > > >> > Standard Java JAAS configuration - see example above. >> > > >> > >> > > >> > >> > > >> > *4. It sounds like different LoginModules may need >> > > >> > different CallbackHandlers. Since we only allow a single >> > > >> > pluggableCallbackHandler class in the KIP-43 configuration, how >> will >> > > this >> > > >> > be handled?* >> > > >> > One LoginContext, one subject, one callback handler, multiple >> login >> > > >> > modules. This is the Java model for login. And it works well >> because >> > > >> > callback handlers tend to be fairly standard. Even if the smart >> card >> > > SASL >> > > >> > callbacks needed to do something wildly different and you wanted >> a >> > > broker >> > > >> > that combined this with Kerberos, you would implement a callback >> that >> > > >> > looked like the code below. >> > > >> > >> > > >> > private CallbackHandler handler; >> > > >> > >> > > >> > public void configure(Map<String, ?> configs, Mode mode, >> > > Subject >> > > >> > subject, String mechanism) {public void configure(Map<String, ?> >> > > configs, >> > > >> > Mode mode, Subject subject, String mechanism) { >> > > >> > >> > > >> > switch (mechanism) { >> > > >> > >> > > >> > case "GSSAPI" : handler = new >> GssapiCallbackHandler(); >> > > >> break >> > > >> > ; >> > > >> > >> > > >> > case "SMARTCARD" : handler = new >> > > >> SmartcardCallbackHandler(); >> > > >> > break; >> > > >> > >> > > >> > } >> > > >> > >> > > >> > } >> > > >> > >> > > >> > >> > > >> > public void handle(Callback[] callbacks) throws >> IOException, >> > > >> > UnsupportedCallbackException { >> > > >> > >> > > >> > handler.handle(callbacks); >> > > >> > >> > > >> > } >> > > >> > >> > > >> > But I don't see this as just an admin configuration. This is >> some code >> > > >> > developed alongside a new mechanism. Yes, you can add a new >> > > configuration >> > > >> > type in Kafka that maps mechanism to callback handler class to >> avoid >> > > the >> > > >> > switch statement. But that doesn't remove the need for new code >> > > >> altogether >> > > >> > since you still need to implement SmartcardCallbackHandler. >> Multiple >> > > >> > conflicting callback handlers within one broker to support >> complete >> > > >> > different behaviour is an edge case. Since the design supports >> > > >> > full flexibility in terms of functionality and you have to write >> new >> > > code >> > > >> > for new mechanisms anyway, this feels like the simplest solution. >> > > >> > >> > > >> > >> > > >> > >> > > >> > On Tue, Mar 8, 2016 at 8:23 PM, Gwen Shapira <g...@confluent.io> >> > > wrote: >> > > >> > >> > > >> >> I guess what is unclear to me is: >> > > >> >> >> > > >> >> 1. How does the proposed Login class interact with LoginContext >> and >> > > >> >> LoginModule. >> > > >> >> 2. Do we really need a pluggable Login class, when users >> > > >> >> (administrators, not developers) are more likely to want to >> specify >> > > >> >> LoginModules? >> > > >> >> 3. How will users get to specify multiple LoginModules? >> > > >> >> 4. It sounds like different LoginModules may need different >> > > >> >> CallbackHandlers. Since we only allow a single pluggable >> > > >> >> CallbackHandler class in the KIP-43 configuration, how will >> this be >> > > >> >> handled? >> > > >> >> >> > > >> >> In other words, suppose I want my Kafka installation to support >> both >> > > >> >> Kerberos and SmartCard mechanisms, can you tell me how to >> configure >> > > >> >> Kafka? (as admin, not developer)? >> > > >> >> >> > > >> >> Gwen >> > > >> >> >> > > >> >> >> > > >> >> >> > > >> >> On Tue, Mar 8, 2016 at 11:46 AM, Rajini Sivaram >> > > >> >> <rajinisiva...@googlemail.com> wrote: >> > > >> >> > Gwen, >> > > >> >> > >> > > >> >> > I am not sure I understand your concern. Java login contexts >> > > support >> > > >> >> > multiple login modules precisely to support this type of >> real-world >> > > >> >> > scenario. >> > > >> >> > >> > > >> >> > From Javadoc of *javax.security.auth.login.LoginContext*: >> > > >> >> > >> > > >> >> > *In addition to supporting pluggable authentication, this >> class >> > > also >> > > >> >> > supports the notion of stacked authentication. Applications >> may be >> > > >> >> > configured to use more than one LoginModule. For example, one >> could >> > > >> >> > configure both a Kerberos LoginModule and a smart card >> LoginModule >> > > >> under >> > > >> >> an >> > > >> >> > application. * >> > > >> >> > >> > > >> >> > * A typical caller instantiates a LoginContext with a name >> and a >> > > >> >> > CallbackHandler. LoginContext uses the name as the index into >> a >> > > >> >> > Configuration to determine which LoginModules should be used, >> and >> > > >> which >> > > >> >> > ones must succeed in order for the overall authentication to >> > > succeed. >> > > >> The >> > > >> >> > CallbackHandler is passed to the underlying LoginModules so >> they >> > > may >> > > >> >> > communicate and interact with users (prompting for a username >> and >> > > >> >> password >> > > >> >> > via a graphical user interface, for example).* >> > > >> >> > >> > > >> >> > If it is good enough for Java, surely it must be good enough >> for >> > > >> Kafka. >> > > >> >> We >> > > >> >> > don't expect users to directly use existing classes when they >> want >> > > to >> > > >> >> > support complex new mechanisms. Login is not a standard >> interface, >> > > >> and we >> > > >> >> > want to make callback handlers configurable. So we do require >> > > users to >> > > >> >> wrap >> > > >> >> > their existing classes in our interface. And the interfaces >> make it >> > > >> >> > possible to plugin any number of mechanisms. Am I missing >> > > something? >> > > >> >> > >> > > >> >> > >> > > >> >> > >> > > >> >> > On Tue, Mar 8, 2016 at 4:49 PM, Gwen Shapira < >> g...@confluent.io> >> > > >> wrote: >> > > >> >> > >> > > >> >> >> Yes, I understand that. >> > > >> >> >> >> > > >> >> >> However, because the current configuration only allows a >> single >> > > >> >> >> callback and a single login module, I need to supply a class >> that >> > > >> >> >> supports callbacks or logins for every single mechanism that >> I >> > > need >> > > >> to >> > > >> >> >> support. I question whether this is applicable in real-world >> > > scenario >> > > >> >> >> where a user may need to support mechanisms that were >> written by >> > > >> >> >> different vendors and where a single class that implements >> all the >> > > >> >> >> necessary modules or callsbacks is very unlikely to exist. >> > > >> >> >> >> > > >> >> >> I agree that supporting a list of classes and dynamically >> using >> > > the >> > > >> >> >> correct callbacks and modules based on configuration is >> complex, >> > > but >> > > >> I >> > > >> >> >> don't think this feature is useful enough without it. >> > > >> >> >> >> > > >> >> >> Gwen >> > > >> >> >> >> > > >> >> >> On Tue, Mar 8, 2016 at 2:40 AM, Rajini Sivaram >> > > >> >> >> <rajinisiva...@googlemail.com> wrote: >> > > >> >> >> > Gwen, >> > > >> >> >> > >> > > >> >> >> > Sorry if this is not clear in the KIP. The goal of the KIP >> is to >> > > >> >> enable >> > > >> >> >> new >> > > >> >> >> > mechanisms to be added without any changes to Apache >> Kafka. The >> > > >> >> >> > configuration proposed by the KIP is based on the >> > > configurability >> > > >> of >> > > >> >> the >> > > >> >> >> > Java SASL API and what it would take to add a new mechanism >> > > >> similar to >> > > >> >> >> > Kerberos/PLAIN/Digest-MD5 and the requirements that Tao >> raised >> > > for >> > > >> >> custom >> > > >> >> >> > authentication. >> > > >> >> >> > >> > > >> >> >> > The PR includes a test for Digest-MD5 which is not >> included in >> > > >> Kafka, >> > > >> >> but >> > > >> >> >> > is tested using the configuration options that allow new >> > > mechanisms >> > > >> >> to be >> > > >> >> >> > added. >> > > >> >> >> > >> > > >> >> >> > If you want to add a more complex mechanism which requires >> > > >> integration >> > > >> >> >> with >> > > >> >> >> > some external authentication libraries for login or >> callback >> > > >> handlers, >> > > >> >> >> you >> > > >> >> >> > can do that by configuring your own callback handler and >> login >> > > >> >> >> > implementation. The login class allows you to add >> additional >> > > >> threads >> > > >> >> (for >> > > >> >> >> > instance for token refresh), while the callback handler >> extends >> > > the >> > > >> >> >> > standard Java CallbackHandler interface, enabling you to >> > > override >> > > >> any >> > > >> >> of >> > > >> >> >> > the default callbacks in Kafka to provide your own >> behaviour. If >> > > >> you >> > > >> >> want >> > > >> >> >> > to add your own login modules, SASL server or SASL client >> > > >> >> implementation >> > > >> >> >> > for a mechanism that is not supported in your JDK, you can >> do >> > > that >> > > >> by >> > > >> >> >> > adding new Java security modules, using the built-in >> > > pluggability >> > > >> of >> > > >> >> Java >> > > >> >> >> > security. >> > > >> >> >> > >> > > >> >> >> > And finally, if you want to enable multiple mechanisms in >> your >> > > >> Kafka >> > > >> >> >> > brokers, you can specify a callback handler class that >> either >> > > >> provides >> > > >> >> >> the >> > > >> >> >> > handlers for all the mechanisms (as the PR does for >> Kerberos and >> > > >> >> PLAIN) >> > > >> >> >> or >> > > >> >> >> > a delegator that calls different callback handlers based >> on the >> > > >> >> >> mechanism. >> > > >> >> >> > And for login, you provide a single or a compound class >> that >> > > >> provides >> > > >> >> any >> > > >> >> >> > additional login logic for all the login modules that you >> have >> > > >> >> specified >> > > >> >> >> > for KafkaServer. >> > > >> >> >> > >> > > >> >> >> > Hope that helps. >> > > >> >> >> > >> > > >> >> >> > >> > > >> >> >> > On Tue, Mar 8, 2016 at 1:17 AM, Gwen Shapira < >> g...@confluent.io >> > > > >> > > >> >> wrote: >> > > >> >> >> > >> > > >> >> >> >> Can you explain the process for a adding a new mechanism >> based >> > > on >> > > >> >> >> current >> > > >> >> >> >> KIP? >> > > >> >> >> >> >> > > >> >> >> >> My thought is that if it requires modifying Apache Kafka >> code, >> > > it >> > > >> is >> > > >> >> >> >> not pluggable enough. >> > > >> >> >> >> >> > > >> >> >> >> On Mon, Mar 7, 2016 at 4:04 PM, Rajini Sivaram >> > > >> >> >> >> <rajinisiva...@googlemail.com> wrote: >> > > >> >> >> >> > Gwen, >> > > >> >> >> >> > >> > > >> >> >> >> > In cases where you want completely different callbacks >> for >> > > >> >> different >> > > >> >> >> >> > mechanisms, I was thinking that the choice would be >> between a >> > > >> map >> > > >> >> of >> > > >> >> >> >> > classes (mechanism -> callbackHandler class) or a >> delegator >> > > >> class >> > > >> >> that >> > > >> >> >> >> > chooses the appropriate callback handler class based on >> > > >> mechanism. >> > > >> >> I >> > > >> >> >> >> chose >> > > >> >> >> >> > the latter since it makes it easier to configure in >> Kafka. >> > > >> Since we >> > > >> >> >> >> create >> > > >> >> >> >> > a callback handler for each channel and configure it >> with the >> > > >> >> >> >> > client-selected mechanism, it is straightforward to >> have one >> > > >> >> wrapper >> > > >> >> >> >> class >> > > >> >> >> >> > that delegates to the right mechanism-specific class to >> > > handle >> > > >> >> >> callbacks. >> > > >> >> >> >> > In many cases, a single class may be sufficient (the PR >> uses >> > > a >> > > >> >> single >> > > >> >> >> >> > callback class for Kerberos and PLAIN). I do see your >> point >> > > >> about >> > > >> >> the >> > > >> >> >> >> > flexibility that multiple classes would provide, but >> since >> > > you >> > > >> need >> > > >> >> >> to be >> > > >> >> >> >> > able to associate the callback with a mechanism for >> this to >> > > be >> > > >> >> >> useful, I >> > > >> >> >> >> am >> > > >> >> >> >> > not sure if just a list would add value. >> > > >> >> >> >> > >> > > >> >> >> >> > Login class is slightly different since the proposal is >> to >> > > use a >> > > >> >> >> single >> > > >> >> >> >> > login context with multiple login modules to handle >> multiple >> > > >> >> >> mechanisms. >> > > >> >> >> >> In >> > > >> >> >> >> > this case, you want to perform login for all the >> mechanisms >> > > that >> > > >> >> are >> > > >> >> >> >> > enabled. And you want to call loginContext.login() only >> once. >> > > >> >> Again, >> > > >> >> >> you >> > > >> >> >> >> > can delegate to multiple classes if you wish to add some >> > > complex >> > > >> >> >> >> > mechanism-specific logic, but a single login class >> makes the >> > > >> >> mapping >> > > >> >> >> to a >> > > >> >> >> >> > single login context and the login cache more obvious >> (the PR >> > > >> has a >> > > >> >> >> test >> > > >> >> >> >> > that includes Kerberos and PLAIN). >> > > >> >> >> >> > >> > > >> >> >> >> > Thoughts? >> > > >> >> >> >> > >> > > >> >> >> >> > On Mon, Mar 7, 2016 at 9:57 PM, Gwen Shapira < >> > > g...@confluent.io >> > > >> > >> > > >> >> >> wrote: >> > > >> >> >> >> > >> > > >> >> >> >> >> Hi, >> > > >> >> >> >> >> >> > > >> >> >> >> >> Before I can vote on this KIP, I have two additional >> > > questions >> > > >> / >> > > >> >> >> >> >> comments on the new configuration: >> > > >> >> >> >> >> >> > > >> >> >> >> >> 1. sasl.callback.handler.class - it looks like we want >> a >> > > single >> > > >> >> class >> > > >> >> >> >> >> that implements all mechanisms. I think this will make >> it >> > > >> >> difficult >> > > >> >> >> to >> > > >> >> >> >> >> extend since the only way I can add a mechanism will >> be by >> > > >> >> >> >> >> implementing every single existing mechanism (otherwise >> > > >> customers >> > > >> >> >> will >> > > >> >> >> >> >> need to choose between new and existing when selecting >> which >> > > >> >> class to >> > > >> >> >> >> >> use). If Microsoft releases a proprietary "AD >> Mechanism" and >> > > >> >> Oracle >> > > >> >> >> >> >> releases "OID mechanism", there will be no class that >> > > >> implements >> > > >> >> >> both. >> > > >> >> >> >> >> Can we make it a list of classes instead? I realize >> this >> > > >> >> complicates >> > > >> >> >> >> >> the code a bit (some kind of factory will be required >> to >> > > choose >> > > >> >> the >> > > >> >> >> >> >> right class to use), but important IMO. >> > > >> >> >> >> >> 2. similar for sasl.login.class - if I have a class for >> > > >> Kerberos >> > > >> >> >> (with >> > > >> >> >> >> >> refresh thread) and a class for "plain", we need to be >> able >> > > to >> > > >> >> load >> > > >> >> >> >> >> both. >> > > >> >> >> >> >> >> > > >> >> >> >> >> Gwen >> > > >> >> >> >> >> >> > > >> >> >> >> >> On Wed, Mar 2, 2016 at 12:30 AM, Rajini Sivaram >> > > >> >> >> >> >> <rajinisiva...@googlemail.com> wrote: >> > > >> >> >> >> >> > Jun, >> > > >> >> >> >> >> > >> > > >> >> >> >> >> > Thanks, I have added a note to the KIP. I will add a >> > > comment >> > > >> in >> > > >> >> the >> > > >> >> >> >> >> > implementation and also add a unit test to ensure >> that >> > > >> conflicts >> > > >> >> >> are >> > > >> >> >> >> >> > avoided when version number is modified. >> > > >> >> >> >> >> > >> > > >> >> >> >> >> > On Tue, Mar 1, 2016 at 5:43 PM, Jun Rao < >> j...@confluent.io >> > > > >> > > >> >> wrote: >> > > >> >> >> >> >> > >> > > >> >> >> >> >> >> Rajini, >> > > >> >> >> >> >> >> >> > > >> >> >> >> >> >> Thanks for the explanation. For 1, this implies >> that we >> > > have >> > > >> >> to be >> > > >> >> >> >> >> careful >> > > >> >> >> >> >> >> with changing the 2-byte version in the future to >> avoid >> > > >> >> conflict. >> > > >> >> >> >> Could >> > > >> >> >> >> >> you >> > > >> >> >> >> >> >> document this in the KIP and also in the >> implementation? >> > > >> >> >> >> >> >> >> > > >> >> >> >> >> >> Jun >> > > >> >> >> >> >> >> >> > > >> >> >> >> >> >> On Tue, Mar 1, 2016 at 2:47 AM, Rajini Sivaram < >> > > >> >> >> >> >> >> rajinisiva...@googlemail.com >> > > >> >> >> >> >> >> > wrote: >> > > >> >> >> >> >> >> >> > > >> >> >> >> >> >> > Jun, >> > > >> >> >> >> >> >> > >> > > >> >> >> >> >> >> > Thank you for the review. >> > > >> >> >> >> >> >> > >> > > >> >> >> >> >> >> > >> > > >> >> >> >> >> >> > 1. With GSSAPI, the first context establishment >> > > packet >> > > >> >> starts >> > > >> >> >> >> with >> > > >> >> >> >> >> the >> > > >> >> >> >> >> >> > byte 0x60 (tag for APPLICATION-0) followed by a >> > > >> >> >> variable-length >> > > >> >> >> >> >> >> encoded >> > > >> >> >> >> >> >> > size, followed by various tags and contents. >> And the >> > > >> >> packet >> > > >> >> >> also >> > > >> >> >> >> >> >> > contains a >> > > >> >> >> >> >> >> > checksum. This is completely different from the >> > > >> mechanism >> > > >> >> >> packet >> > > >> >> >> >> >> from >> > > >> >> >> >> >> >> > Kafka >> > > >> >> >> >> >> >> > clients which start with a two-byte version >> set to >> > > zero >> > > >> >> >> >> currently, >> > > >> >> >> >> >> >> > followed >> > > >> >> >> >> >> >> > by just a String mechanism. >> > > >> >> >> >> >> >> > 2. Agreed, I have removed the version from the >> > > server >> > > >> >> >> response >> > > >> >> >> >> in >> > > >> >> >> >> >> the >> > > >> >> >> >> >> >> > KIP. Thanks. >> > > >> >> >> >> >> >> > >> > > >> >> >> >> >> >> > >> > > >> >> >> >> >> >> > On Tue, Mar 1, 2016 at 2:33 AM, Jun Rao < >> > > j...@confluent.io >> > > >> > >> > > >> >> >> wrote: >> > > >> >> >> >> >> >> > >> > > >> >> >> >> >> >> > > Rajini, >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> > > Thanks for the updates. Just a couple of minor >> > > comments. >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> > > 1. With the default GSSAPI, what's the first >> packet >> > > that >> > > >> >> the >> > > >> >> >> >> client >> > > >> >> >> >> >> >> sends >> > > >> >> >> >> >> >> > > to the server? Is that completely different >> from the >> > > >> packet >> > > >> >> >> >> format >> > > >> >> >> >> >> that >> > > >> >> >> >> >> >> > we >> > > >> >> >> >> >> >> > > will use for non-GSSAPI mechanisms? >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> > > 2. In the server response, it doesn't seem that >> we >> > > need >> > > >> to >> > > >> >> >> >> include >> > > >> >> >> >> >> the >> > > >> >> >> >> >> >> > > version since the client knows the version of >> the >> > > >> request >> > > >> >> >> that it >> > > >> >> >> >> >> >> sends. >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> > > Jun >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> > > On Mon, Feb 29, 2016 at 10:14 AM, Rajini >> Sivaram < >> > > >> >> >> >> >> >> > > rajinisiva...@googlemail.com> wrote: >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> > > > Harsha, >> > > >> >> >> >> >> >> > > > >> > > >> >> >> >> >> >> > > > Thank you for the review. I will wait another >> day >> > > to >> > > >> see >> > > >> >> if >> > > >> >> >> >> there >> > > >> >> >> >> >> is >> > > >> >> >> >> >> >> > more >> > > >> >> >> >> >> >> > > > feedback and then start a voting thread. >> > > >> >> >> >> >> >> > > > >> > > >> >> >> >> >> >> > > > Rajini >> > > >> >> >> >> >> >> > > > >> > > >> >> >> >> >> >> > > > On Mon, Feb 29, 2016 at 2:51 PM, Harsha < >> > > >> ka...@harsha.io >> > > >> >> > >> > > >> >> >> >> wrote: >> > > >> >> >> >> >> >> > > > >> > > >> >> >> >> >> >> > > > > Rajini, >> > > >> >> >> >> >> >> > > > > Thanks for the changes to the >> KIP. >> > > It >> > > >> >> looks >> > > >> >> >> >> good >> > > >> >> >> >> >> to >> > > >> >> >> >> >> >> > me. I >> > > >> >> >> >> >> >> > > > > think we can move to voting. >> > > >> >> >> >> >> >> > > > > Thanks, >> > > >> >> >> >> >> >> > > > > Harsha >> > > >> >> >> >> >> >> > > > > >> > > >> >> >> >> >> >> > > > > On Mon, Feb 29, 2016, at 12:43 AM, Rajini >> Sivaram >> > > >> >> wrote: >> > > >> >> >> >> >> >> > > > > > I have added some more detail to the KIP >> based >> > > on >> > > >> the >> > > >> >> >> >> >> discussion >> > > >> >> >> >> >> >> in >> > > >> >> >> >> >> >> > > the >> > > >> >> >> >> >> >> > > > > > last KIP meeting to simplify support for >> > > multiple >> > > >> >> >> >> mechanisms. >> > > >> >> >> >> >> >> Have >> > > >> >> >> >> >> >> > > also >> > > >> >> >> >> >> >> > > > > > changed the property names to reflect >> this. >> > > >> >> >> >> >> >> > > > > > >> > > >> >> >> >> >> >> > > > > > Also updated the PR in >> > > >> >> >> >> >> >> > > > >> https://issues.apache.org/jira/browse/KAFKA-3149 >> > > >> >> >> >> >> >> > > > > > to >> > > >> >> >> >> >> >> > > > > > reflect the KIP. >> > > >> >> >> >> >> >> > > > > > >> > > >> >> >> >> >> >> > > > > > Any feedback is appreciated. >> > > >> >> >> >> >> >> > > > > > >> > > >> >> >> >> >> >> > > > > > >> > > >> >> >> >> >> >> > > > > > On Tue, Feb 23, 2016 at 9:36 PM, Rajini >> > > Sivaram < >> > > >> >> >> >> >> >> > > > > > rajinisiva...@googlemail.com> wrote: >> > > >> >> >> >> >> >> > > > > > >> > > >> >> >> >> >> >> > > > > > > I have updated the KIP based on the >> > > discussion >> > > >> in >> > > >> >> the >> > > >> >> >> KIP >> > > >> >> >> >> >> >> meeting >> > > >> >> >> >> >> >> > > > > today. >> > > >> >> >> >> >> >> > > > > > > >> > > >> >> >> >> >> >> > > > > > > Comments and feedback are welcome. >> > > >> >> >> >> >> >> > > > > > > >> > > >> >> >> >> >> >> > > > > > > On Wed, Feb 3, 2016 at 7:20 PM, Rajini >> > > Sivaram < >> > > >> >> >> >> >> >> > > > > > > rajinisiva...@googlemail.com> wrote: >> > > >> >> >> >> >> >> > > > > > > >> > > >> >> >> >> >> >> > > > > > >> Hi Harsha, >> > > >> >> >> >> >> >> > > > > > >> >> > > >> >> >> >> >> >> > > > > > >> Thank you for the review. Can you >> clarify - >> > > I >> > > >> >> think >> > > >> >> >> you >> > > >> >> >> >> are >> > > >> >> >> >> >> >> > saying >> > > >> >> >> >> >> >> > > > > that >> > > >> >> >> >> >> >> > > > > > >> the client should send its mechanism >> over >> > > the >> > > >> >> wire to >> > > >> >> >> >> the >> > > >> >> >> >> >> >> > server. >> > > >> >> >> >> >> >> > > Is >> > > >> >> >> >> >> >> > > > > that >> > > >> >> >> >> >> >> > > > > > >> correct? The exchange is slightly >> different >> > > in >> > > >> the >> > > >> >> >> KIP >> > > >> >> >> >> >> (the PR >> > > >> >> >> >> >> >> > > > > matches the >> > > >> >> >> >> >> >> > > > > > >> KIP) from the one you described to >> enable >> > > >> >> >> >> interoperability >> > > >> >> >> >> >> >> with >> > > >> >> >> >> >> >> > > > > 0.9.0.0. >> > > >> >> >> >> >> >> > > > > > >> >> > > >> >> >> >> >> >> > > > > > >> >> > > >> >> >> >> >> >> > > > > > >> On Wed, Feb 3, 2016 at 1:56 PM, Harsha >> < >> > > >> >> >> m...@harsha.io> >> > > >> >> >> >> >> >> wrote: >> > > >> >> >> >> >> >> > > > > > >> >> > > >> >> >> >> >> >> > > > > > >>> Rajini, >> > > >> >> >> >> >> >> > > > > > >>> I looked at the PR you >> have. I >> > > >> think >> > > >> >> its >> > > >> >> >> >> better >> > > >> >> >> >> >> >> with >> > > >> >> >> >> >> >> > > > your >> > > >> >> >> >> >> >> > > > > > >>> earlier approach rather >> than >> > > >> extending >> > > >> >> >> the >> > > >> >> >> >> >> >> protocol. >> > > >> >> >> >> >> >> > > > > > >>> What I was thinking initially is, >> Broker >> > > has a >> > > >> >> >> config >> > > >> >> >> >> >> option >> > > >> >> >> >> >> >> of >> > > >> >> >> >> >> >> > > say >> > > >> >> >> >> >> >> > > > > > >>> sasl.mechanism = GSSAPI, PLAIN >> > > >> >> >> >> >> >> > > > > > >>> and the client can have similar >> config of >> > > >> >> >> >> >> >> sasl.mechanism=PLAIN. >> > > >> >> >> >> >> >> > > > > Client >> > > >> >> >> >> >> >> > > > > > >>> can send its sasl mechanism before the >> > > >> handshake >> > > >> >> >> starts >> > > >> >> >> >> >> and >> > > >> >> >> >> >> >> if >> > > >> >> >> >> >> >> > > the >> > > >> >> >> >> >> >> > > > > > >>> broker accepts that particular >> mechanism >> > > than >> > > >> it >> > > >> >> >> can go >> > > >> >> >> >> >> ahead >> > > >> >> >> >> >> >> > > with >> > > >> >> >> >> >> >> > > > > > >>> handshake otherwise return a error >> saying >> > > that >> > > >> >> the >> > > >> >> >> >> >> mechanism >> > > >> >> >> >> >> >> > not >> > > >> >> >> >> >> >> > > > > > >>> allowed. >> > > >> >> >> >> >> >> > > > > > >>> >> > > >> >> >> >> >> >> > > > > > >>> Thanks, >> > > >> >> >> >> >> >> > > > > > >>> Harsha >> > > >> >> >> >> >> >> > > > > > >>> >> > > >> >> >> >> >> >> > > > > > >>> On Wed, Feb 3, 2016, at 04:58 AM, >> Rajini >> > > >> Sivaram >> > > >> >> >> wrote: >> > > >> >> >> >> >> >> > > > > > >>> > A slightly different approach for >> > > supporting >> > > >> >> >> >> different >> > > >> >> >> >> >> SASL >> > > >> >> >> >> >> >> > > > > mechanisms >> > > >> >> >> >> >> >> > > > > > >>> > within a broker is to allow the same >> > > >> "*security >> > > >> >> >> >> >> protocol*" >> > > >> >> >> >> >> >> to >> > > >> >> >> >> >> >> > > be >> > > >> >> >> >> >> >> > > > > used >> > > >> >> >> >> >> >> > > > > > >>> on >> > > >> >> >> >> >> >> > > > > > >>> > different ports with different >> > > configuration >> > > >> >> >> >> options. An >> > > >> >> >> >> >> >> > > > advantage >> > > >> >> >> >> >> >> > > > > of >> > > >> >> >> >> >> >> > > > > > >>> > this >> > > >> >> >> >> >> >> > > > > > >>> > approach is that it extends the >> > > >> >> configurability of >> > > >> >> >> >> not >> > > >> >> >> >> >> just >> > > >> >> >> >> >> >> > > SASL, >> > > >> >> >> >> >> >> > > > > but >> > > >> >> >> >> >> >> > > > > > >>> any >> > > >> >> >> >> >> >> > > > > > >>> > protocol. For instance, it would >> enable >> > > the >> > > >> >> use of >> > > >> >> >> >> SSL >> > > >> >> >> >> >> with >> > > >> >> >> >> >> >> > > > mutual >> > > >> >> >> >> >> >> > > > > > >>> client >> > > >> >> >> >> >> >> > > > > > >>> > authentication on one port or >> different >> > > >> >> >> certificate >> > > >> >> >> >> >> chains >> > > >> >> >> >> >> >> on >> > > >> >> >> >> >> >> > > > > another. >> > > >> >> >> >> >> >> > > > > > >>> > And >> > > >> >> >> >> >> >> > > > > > >>> > it avoids the need for SASL >> mechanism >> > > >> >> negotiation. >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > Kafka would have the same "*security >> > > >> protocols" >> > > >> >> >> >> >> *defined as >> > > >> >> >> >> >> >> > > > today, >> > > >> >> >> >> >> >> > > > > but >> > > >> >> >> >> >> >> > > > > > >>> > with >> > > >> >> >> >> >> >> > > > > > >>> > (a single) configurable SASL >> mechanism. >> > > To >> > > >> have >> > > >> >> >> >> >> different >> > > >> >> >> >> >> >> > > > > > >>> configurations >> > > >> >> >> >> >> >> > > > > > >>> > of >> > > >> >> >> >> >> >> > > > > > >>> > a protocol within a broker, users >> can >> > > define >> > > >> >> new >> > > >> >> >> >> >> protocol >> > > >> >> >> >> >> >> > names >> > > >> >> >> >> >> >> > > > > which >> > > >> >> >> >> >> >> > > > > > >>> are >> > > >> >> >> >> >> >> > > > > > >>> > configured versions of existing >> > > protocols, >> > > >> >> perhaps >> > > >> >> >> >> using >> > > >> >> >> >> >> >> just >> > > >> >> >> >> >> >> > > > > > >>> > configuration >> > > >> >> >> >> >> >> > > > > > >>> > entries and no additional code. >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > For example: >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > A single mechanism broker would be >> > > >> configured >> > > >> >> as: >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > listeners=SASL_SSL://:9092 >> > > >> >> >> >> >> >> > > > > > >>> > sasl.mechanism=GSSAPI >> > > >> >> >> >> >> >> > > > > > >>> > sasl.kerberos.class.name=kafka >> > > >> >> >> >> >> >> > > > > > >>> > ... >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > And a multi-mechanism broker would >> be >> > > >> >> configured >> > > >> >> >> as: >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> listeners=gssapi://:9092,plain://:9093,custom://:9094 >> > > >> >> >> >> >> >> > > > > > >>> > gssapi.security.protocol=SASL_SSL >> > > >> >> >> >> >> >> > > > > > >>> > gssapi.sasl.mechanism=GSSAPI >> > > >> >> >> >> >> >> > > > > > >>> > gssapi.sasl.kerberos.class.name >> =kafka >> > > >> >> >> >> >> >> > > > > > >>> > ... >> > > >> >> >> >> >> >> > > > > > >>> > plain.security.protocol=SASL_SSL >> > > >> >> >> >> >> >> > > > > > >>> > plain.sasl.mechanism=PLAIN >> > > >> >> >> >> >> >> > > > > > >>> > .. >> > > >> >> >> >> >> >> > > > > > >>> > >> custom.security.protocol=SASL_PLAINTEXT >> > > >> >> >> >> >> >> > > > > > >>> > custom.sasl.mechanism=CUSTOM >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> custom.sasl.callback.handler.class=example.CustomCallbackHandler >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > This is still a big change because >> it >> > > >> affects >> > > >> >> the >> > > >> >> >> >> >> currently >> > > >> >> >> >> >> >> > > fixed >> > > >> >> >> >> >> >> > > > > > >>> > enumeration of security protocol >> > > >> definitions, >> > > >> >> but >> > > >> >> >> one >> > > >> >> >> >> >> that >> > > >> >> >> >> >> >> is >> > > >> >> >> >> >> >> > > > > perhaps >> > > >> >> >> >> >> >> > > > > > >>> > more >> > > >> >> >> >> >> >> > > > > > >>> > flexible than defining every new >> SASL >> > > >> mechanism >> > > >> >> >> as a >> > > >> >> >> >> new >> > > >> >> >> >> >> >> > > security >> > > >> >> >> >> >> >> > > > > > >>> > protocol. >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > Thoughts? >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > On Tue, Feb 2, 2016 at 12:20 PM, >> Rajini >> > > >> >> Sivaram < >> > > >> >> >> >> >> >> > > > > > >>> > rajinisiva...@googlemail.com> >> wrote: >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> >> >> >> >> >> > > > > > >>> > > As Ismael has said, we do not >> have a >> > > >> >> >> requirement to >> > > >> >> >> >> >> >> support >> > > >> >> >> >> >> >> > > > > multiple >> > > >> >> >> >> >> >> > > > > > >>> > > protocols in a broker. But I >> agree with >> > > >> Jun's >> > > >> >> >> >> >> observation >> > > >> >> >> >> >> >> > > that >> > > >> >> >> >> >> >> > > > > some >> > > >> >> >> >> >> >> > > > > > >>> > > companies might want to support a >> > > >> different >> > > >> >> >> >> >> >> authentication >> > > >> >> >> >> >> >> > > > > mechanism >> > > >> >> >> >> >> >> > > > > > >>> for >> > > >> >> >> >> >> >> > > > > > >>> > > internal users or partners. For >> > > instance, >> > > >> we >> > > >> >> do >> > > >> >> >> use >> > > >> >> >> >> >> two >> > > >> >> >> >> >> >> > > > different >> > > >> >> >> >> >> >> > > > > > >>> > > authentication mechanisms, it >> just so >> > > >> happens >> > > >> >> >> that >> > > >> >> >> >> we >> > > >> >> >> >> >> are >> > > >> >> >> >> >> >> > > able >> > > >> >> >> >> >> >> > > > > to use >> > > >> >> >> >> >> >> > > > > > >>> > > certificate-based authentication >> for >> > > >> internal >> > > >> >> >> >> users, >> > > >> >> >> >> >> and >> > > >> >> >> >> >> >> > > hence >> > > >> >> >> >> >> >> > > > > don't >> > > >> >> >> >> >> >> > > > > > >>> > > require multiple SASL mechanisms >> in a >> > > >> broker. >> > > >> >> >> >> >> >> > > > > > >>> > > >> > > >> >> >> >> >> >> > > > > > >>> > > As Tao has pointed out, mechanism >> > > >> negotiation >> > > >> >> >> is a >> > > >> >> >> >> >> common >> > > >> >> >> >> >> >> > > usage >> > > >> >> >> >> >> >> > > > > > >>> pattern. >> > > >> >> >> >> >> >> > > > > > >>> > > Many existing protocols that >> support >> > > SASL >> > > >> do >> > > >> >> >> >> already >> > > >> >> >> >> >> use >> > > >> >> >> >> >> >> > this >> > > >> >> >> >> >> >> > > > > > >>> pattern. AMQP >> > > >> >> >> >> >> >> > > > > > >>> > > ( >> > > >> >> >> >> >> >> > > > > > >>> > > >> > > >> >> >> >> >> >> > > > > > >>> >> > > >> >> >> >> >> >> > > > > >> > > >> >> >> >> >> >> > > > >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> > >> > > >> >> >> >> >> >> >> > > >> >> >> >> >> >> > > >> >> >> >> >> > > >> >> >> >> > > >> >> >> > > >> >> > > >> http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-security-v1.0-os.html#type-sasl-mechanisms >> > > >> >> >> >> >> >> > > > > > >>> ), >> > > >> >> >> >> >> >> > > > > > >>> > > which, as a messaging protocol >> maybe >> > > >> closer >> > > >> >> to >> > > >> >> >> >> Kafka >> > > >> >> >> >> >> in >> > > >> >> >> >> >> >> use >> > > >> >> >> >> >> >> > > > cases >> > > >> >> >> >> >> >> > > > > > >>> than >> > > >> >> >> >> >> >> > > > > > >>> > > Zookeeper, is an example. Other >> > > examples >> > > >> >> where >> > > >> >> >> the >> > > >> >> >> >> >> client >> > > >> >> >> >> >> >> > > > > negotiates >> > > >> >> >> >> >> >> > > > > > >>> or >> > > >> >> >> >> >> >> > > > > > >>> > > sends SASL mechanism to server >> include >> > > >> ACAP >> > > >> >> >> that is >> > > >> >> >> >> >> used >> > > >> >> >> >> >> >> as >> > > >> >> >> >> >> >> > > an >> > > >> >> >> >> >> >> > > > > > >>> example in >> > > >> >> >> >> >> >> > > > > > >>> > > the SASL RFCs, POP3, LDAP, SMTP >> etc. >> > > This >> > > >> is >> > > >> >> >> not to >> > > >> >> >> >> >> say >> > > >> >> >> >> >> >> > that >> > > >> >> >> >> >> >> > > > > Kafka >> > > >> >> >> >> >> >> > > > > > >>> > > shouldn't use a different type of >> > > >> mechanism >> > > >> >> >> >> selection >> > > >> >> >> >> >> >> that >> > > >> >> >> >> >> >> > > fits >> > > >> >> >> >> >> >> > > > > > >>> better with >> > > >> >> >> >> >> >> > > > > > >>> > > the existing Kafka design. Just >> that >> > > >> >> negotiation >> > > >> >> >> >> is a >> > > >> >> >> >> >> >> > common >> > > >> >> >> >> >> >> > > > > pattern >> > > >> >> >> >> >> >> > > > > > >>> and >> > > >> >> >> >> >> >> > > > > > >>> > > since we typically turn on >> > > >> javax.net.debug to >> > > >> >> >> debug >> > > >> >> >> >> >> TLS >> > > >> >> >> >> >> >> > > > > negotiation >> > > >> >> >> >> >> >> > > > > > >>> issues, >> > > >> >> >> >> >> >> > > > > > >>> > > having to use Kafka logging to >> debug >> > > SASL >> > > >> >> >> >> negotiation >> > > >> >> >> >> >> >> > issues >> > > >> >> >> >> >> >> > > is >> > > >> >> >> >> >> >> > > > > not >> > > >> >> >> >> >> >> > > > > > >>> that >> > > >> >> >> >> >> >> > > > > > >>> > > dissimilar. >> > > >> >> >> >> >> >> > > > > > >>> > > >> > > >> >> >> >> >> >> > > > > > >>> > > >> > > >> >> >> >> >> >> > > > > > >>> > > On Tue, Feb 2, 2016 at 6:12 AM, >> tao >> > > xiao < >> > > >> >> >> >> >> >> > > xiaotao...@gmail.com >> > > >> >> >> >> >> >> > > > > >> > > >> >> >> >> >> >> > > > > > >>> wrote: >> > > >> >> >> >> >> >> > > > > > >>> > > >> > > >> >> >> >> >> >> > > > > > >>> > >> I am the author of KIP-44. I >> hope my >> > > use >> > > >> >> case >> > > >> >> >> will >> > > >> >> >> >> >> add >> > > >> >> >> >> >> >> > some >> > > >> >> >> >> >> >> > > > > values >> > > >> >> >> >> >> >> > > > > > >>> to this >> > > >> >> >> >> >> >> > > > > > >>> > >> discussion. The reason I raised >> KIP44 >> > > is >> > > >> >> that I >> > > >> >> >> >> want >> > > >> >> >> >> >> to >> > > >> >> >> >> >> >> be >> > > >> >> >> >> >> >> > > > able >> > > >> >> >> >> >> >> > > > > to >> > > >> >> >> >> >> >> > > > > > >>> > >> implement a custom security >> protocol >> > > that >> > > >> >> can >> > > >> >> >> >> fulfill >> > > >> >> >> >> >> >> the >> > > >> >> >> >> >> >> > > need >> > > >> >> >> >> >> >> > > > > of my >> > > >> >> >> >> >> >> > > > > > >>> > >> company. As pointed out by Ismael >> > > KIP-43 >> > > >> now >> > > >> >> >> >> >> supports a >> > > >> >> >> >> >> >> > > > > pluggable >> > > >> >> >> >> >> >> > > > > > >>> way to >> > > >> >> >> >> >> >> > > > > > >>> > >> inject custom security provider >> to >> > > SASL I >> > > >> >> think >> > > >> >> >> >> it is >> > > >> >> >> >> >> >> > enough >> > > >> >> >> >> >> >> > > > to >> > > >> >> >> >> >> >> > > > > > >>> cover the >> > > >> >> >> >> >> >> > > > > > >>> > >> use case I have and address the >> > > concerns >> > > >> >> >> raised in >> > > >> >> >> >> >> >> KIP-44. >> > > >> >> >> >> >> >> > > > > > >>> > >> >> > > >> >> >> >> >> >> > > > > > >>> > >> For multiple security protocols >> > > support >> > > >> >> >> >> >> simultaneously >> > > >> >> >> >> >> >> it >> > > >> >> >> >> >> >> > is >> > > >> >> >> >> >> >> > > > not >> > > >> >> >> >> >> >> > > > > > >>> needed in >> > > >> >> >> >> >> >> > > > > > >>> > >> my use case and I don't foresee >> it is >> > > >> >> needed in >> > > >> >> >> >> the >> > > >> >> >> >> >> >> future >> > > >> >> >> >> >> >> > > but >> > > >> >> >> >> >> >> > > > > as i >> > > >> >> >> >> >> >> > > > > > >>> said >> > > >> >> >> >> >> >> > > > > > >>> > >> this is my use case only there >> may be >> > > >> other >> > > >> >> use >> > > >> >> >> >> cases >> > > >> >> >> >> >> >> that >> > > >> >> >> >> >> >> > > > need >> > > >> >> >> >> >> >> > > > > it. >> > > >> >> >> >> >> >> > > > > > >>> But if >> > > >> >> >> >> >> >> > > > > > >>> > >> we want to support it in the >> future I >> > > >> >> prefer to >> > > >> >> >> >> get >> > > >> >> >> >> >> it >> > > >> >> >> >> >> >> > right >> > > >> >> >> >> >> >> > > > at >> > > >> >> >> >> >> >> > > > > the >> > > >> >> >> >> >> >> > > > > > >>> first >> > > >> >> >> >> >> >> > > > > > >>> > >> place given the fact that >> security >> > > >> protocol >> > > >> >> is >> > > >> >> >> an >> > > >> >> >> >> >> ENUM >> > > >> >> >> >> >> >> and >> > > >> >> >> >> >> >> > > if >> > > >> >> >> >> >> >> > > > we >> > > >> >> >> >> >> >> > > > > > >>> stick to >> > > >> >> >> >> >> >> > > > > > >>> > >> that implementation it is very >> hard to >> > > >> >> extend >> > > >> >> >> in >> > > >> >> >> >> the >> > > >> >> >> >> >> >> > future >> > > >> >> >> >> >> >> > > > > when we >> > > >> >> >> >> >> >> > > > > > >>> decide >> > > >> >> >> >> >> >> > > > > > >>> > >> multiple security protocols is >> needed. >> > > >> >> >> >> >> >> > > > > > >>> > >> >> > > >> >> >> >> >> >> > > > > > >>> > >> Protocol negotiation is a very >> common >> > > >> usage >> > > >> >> >> >> pattern >> > > >> >> >> >> >> in >> > > >> >> >> >> >> >> > > > security >> > > >> >> >> >> >> >> > > > > > >>> domain. As >> > > >> >> >> >> >> >> > > > > > >>> > >> suggested in Java SASL doc >> > > >> >> >> >> >> >> > > > > > >>> > >> >> > > >> >> >> >> >> >> > > > > > >>> > >> >> > > >> >> >> >> >> >> > > > > > >>> >> > > >> >> >> >> >> >> > > > > >> > > >> >> >> >> >> >> > > > >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> > >> > > >> >> >> >> >> >> >> > > >> >> >> >> >> >> > > >> >> >> >> >> > > >> >> >> >> > > >> >> >> > > >> >> > > >> http://docs.oracle.com/javase/7/docs/technotes/guides/security/sasl/sasl-refguide.html >> > > >> >> >> >> >> >> > > > > > >>> > >> client >> > > >> >> >> >> >> >> > > > > > >>> > >> first sends out a packet to >> server and >> > > >> >> server >> > > >> >> >> >> >> responds >> > > >> >> >> >> >> >> > with >> > > >> >> >> >> >> >> > > a >> > > >> >> >> >> >> >> > > > > list >> > > >> >> >> >> >> >> > > > > > >>> of >> > > >> >> >> >> >> >> > > > > > >>> > >> mechanisms it supports. This is >> very >> > > >> >> similar to >> > > >> >> >> >> >> SSL/TLS >> > > >> >> >> >> >> >> > > > > negotiation. >> > > >> >> >> >> >> >> > > > > > >>> > >> >> > > >> >> >> >> >> >> > > > > > >>> > >> On Tue, 2 Feb 2016 at 06:39 >> Ismael >> > > Juma < >> > > >> >> >> >> >> >> > ism...@juma.me.uk> >> > > >> >> >> >> >> >> > > > > wrote: >> > > >> >> >> >> >> >> > > > > > >>> > >> >> > > >> >> >> >> >> >> > > > > > >>> > >> > On Mon, Feb 1, 2016 at 7:04 >> PM, Gwen >> > > >> >> Shapira >> > > >> >> >> < >> > > >> >> >> >> >> >> > > > > g...@confluent.io> >> > > >> >> >> >> >> >> > > > > > >>> wrote: >> > > >> >> >> >> >> >> > > > > > >>> > >> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > > Looking at "existing >> solutions", >> > > it >> > > >> >> looks >> > > >> >> >> like >> > > >> >> >> >> >> >> > Zookeeper >> > > >> >> >> >> >> >> > > > > allows >> > > >> >> >> >> >> >> > > > > > >>> > >> plugging >> > > >> >> >> >> >> >> > > > > > >>> > >> > in >> > > >> >> >> >> >> >> > > > > > >>> > >> > > any SASL mechanism, but the >> server >> > > >> will >> > > >> >> >> only >> > > >> >> >> >> >> support >> > > >> >> >> >> >> >> > one >> > > >> >> >> >> >> >> > > > > > >>> mechanism at >> > > >> >> >> >> >> >> > > > > > >>> > >> a >> > > >> >> >> >> >> >> > > > > > >>> > >> > > time. >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> > > >> >> >> >> >> >> > > > > > >>> > >> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > This was the original proposal >> from >> > > >> >> Rajini as >> > > >> >> >> >> that >> > > >> >> >> >> >> is >> > > >> >> >> >> >> >> > > enough >> > > >> >> >> >> >> >> > > > > for >> > > >> >> >> >> >> >> > > > > > >>> their >> > > >> >> >> >> >> >> > > > > > >>> > >> > needs. >> > > >> >> >> >> >> >> > > > > > >>> > >> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > > If this is good enough for >> our >> > > >> use-case >> > > >> >> >> (do we >> > > >> >> >> >> >> >> > actually >> > > >> >> >> >> >> >> > > > > need to >> > > >> >> >> >> >> >> > > > > > >>> > >> support >> > > >> >> >> >> >> >> > > > > > >>> > >> > > multiple mechanisms at >> once?), it >> > > >> will >> > > >> >> >> >> simplify >> > > >> >> >> >> >> >> life a >> > > >> >> >> >> >> >> > > lot >> > > >> >> >> >> >> >> > > > > for >> > > >> >> >> >> >> >> > > > > > >>> us ( >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> > > >> >> >> >> >> >> > > > > > >>> > >> >> > > >> >> >> >> >> >> > > > > > >>> >> > > >> >> >> >> >> >> > > > > >> > > >> >> >> >> >> >> > > >> > > >> >> >> >> >> >> >> > > >> >> >> >> >> >> > > >> >> >> >> >> > > >> >> >> >> > > >> >> >> > > >> >> > > >> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Zookeeper+and+SASL >> > > >> >> >> >> >> >> > > > > > >>> > >> > ) >> > > >> >> >> >> >> >> > > > > > >>> > >> > > >> > > >> >> >> >> >> >> > > > > > >>> > >> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > The current thinking is that it >> > > would >> > > >> be >> > > >> >> >> useful >> > > >> >> >> >> to >> > > >> >> >> >> >> >> > support >> > > >> >> >> >> >> >> > > > > > >>> multiple SASL >> > > >> >> >> >> >> >> > > > > > >>> > >> > mechanisms simultaneously. In >> the >> > > KIP >> > > >> >> >> meeting, >> > > >> >> >> >> Jun >> > > >> >> >> >> >> >> > > mentioned >> > > >> >> >> >> >> >> > > > > that >> > > >> >> >> >> >> >> > > > > > >>> > >> companies >> > > >> >> >> >> >> >> > > > > > >>> > >> > sometimes support additional >> > > >> >> authentication >> > > >> >> >> >> >> mechanisms >> > > >> >> >> >> >> >> > for >> > > >> >> >> >> >> >> > > > > > >>> partners, for >> > > >> >> >> >> >> >> > > > > > >>> > >> > example. It does make things >> more >> > > >> >> complex, as >> > > >> >> >> >> you >> > > >> >> >> >> >> say, >> > > >> >> >> >> >> >> > so >> > > >> >> >> >> >> >> > > we >> > > >> >> >> >> >> >> > > > > need >> > > >> >> >> >> >> >> > > > > > >>> to be >> > > >> >> >> >> >> >> > > > > > >>> > >> > sure the complexity is worth >> it. >> > > >> >> >> >> >> >> > > > > > >>> > >> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > Two more points: >> > > >> >> >> >> >> >> > > > > > >>> > >> > >> > > >> >> >> >> >> >> > > > > > >>> > >> > 1. It has been suggested that >> custom >> > > >> >> security >> > > >> >> >> >> >> protocol >> > > >> >> >> >> >> >> > > > > support is >> > > >> >> >> >> >> >> > > > > > >>> > >> needed by >> > > >> >> >> >> >> >> > > > > > >>> > >> > some (KIP-44). Rajini enhanced >> > > KIP-43 >> > > >> so >> > > >> >> >> that a >> > > >> >> >> >> >> SASL >> > > >> >> >> >> >> >> > > > mechanism >> > > >> >> >> >> > >> > ... > > [Message clipped] -- Regards, Rajini
