Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

FYI: For those not following the VOTE thread I updated the KIP and changed the field "rest.advertised.security.protocol" to "rest.advertised.security.listener" as suggested by Jason. On Fri, Jan 19, 2018 at 11:29 AM, Jakub Scholz wrote: > Hi all, > > I did one more update

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Hi all, I did one more update to the KIP-208. I added the "ssl.endpoint.identification.algorithm" to the list of supported options. It can be used to enable / disable the hostname validation when the Kafka Connect nodes are forwarding the requests to the leader. It is minor but useful change, so

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Hi Randall, Yes the KIP should be up to date. The VOTE thread is actually running already for more than 2 months. So the only thing we need is the votes. I pinged the vote thread so that it gets more attention. Thanks & Regards Jakub On Thu, Jan 18, 2018 at 7:33 PM, Randall Hauch

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Jakub, have you had a chance to update the KIP with the latest changes? Would be great to start the vote today so that it's open long enough to adopt before the deadline on Tuesday. Let me know if I can help. On Wed, Jan 17, 2018 at 1:25 AM, Jakub Scholz wrote: > I have been

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

I have been thinking about this a bit more yesterday while updating the code. I think you are right, we should use only the prefixed values if at least one of them exists. Even I got quite easily confused what setup is actually used when the fields are mixed :-). Randall was also in favour of this

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

On Sun, Jan 14, 2018 at 1:20 PM, Jakub Scholz wrote: > Hi Ewen, > > Thanks for your comments / questions. > > re 1) I was using the valuesWithPrefixOverride(...) method from > AbstractConfig class. That takes the overrides setting by setting. It > should not be hard to change

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

I opened the PR: https://issues.apache.org/jira/browse/KAFKA-4029 It is still work in progress, missing mainly tests, docu etc. I will continue to work on it tomorrow. But it shows the implementation. One of the things which came to my mind - the PR is currently using the HttpURLConnection class

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Hi Ewen, Thanks for your comments / questions. re 1) I was using the valuesWithPrefixOverride(...) method from AbstractConfig class. That takes the overrides setting by setting. It should not be hard to change the code to use only the overrides if one value has changed. But I think that the

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

I'm sorry guys, I'm a bit busy with something else this week. But I will get back to his till the end of the week. Thanks & Regards Jakub On Tue, Jan 9, 2018 at 1:19 AM, Ewen Cheslack-Postava wrote: > On Mon, Jan 8, 2018 at 11:39 AM, Randall Hauch wrote: >

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

On Mon, Jan 8, 2018 at 11:39 AM, Randall Hauch wrote: > Nice feedback, Ewen. Thanks! > > On Thu, Jan 4, 2018 at 5:11 PM, Ewen Cheslack-Postava > wrote: > > > Hey Jakub, > > > > Sorry for not getting to this sooner. Overall the proposal looks good to > > me,

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Nice feedback, Ewen. Thanks! On Thu, Jan 4, 2018 at 5:11 PM, Ewen Cheslack-Postava wrote: > Hey Jakub, > > Sorry for not getting to this sooner. Overall the proposal looks good to > me, I just had a couple of questions. > > 1. For the configs/overrides, does this happen on a

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Hey Jakub, Sorry for not getting to this sooner. Overall the proposal looks good to me, I just had a couple of questions. 1. For the configs/overrides, does this happen on a per-setting basis or if one override is included do we not use any of the original settings? I suspect that if you need to

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

There has been no discussion since my last update week ago. Unless someone has some further comments in the next 48 hours, I will start the voting for this KIP. Thanks & Regards Jakub On Tue, Oct 17, 2017 at 5:54 PM, Jakub Scholz wrote: > Ok, so I updated the KIP according to

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Ok, so I updated the KIP according to what we discussed. Please have a look at the updates. Two points I'm not 100% sure about: 1) Should we mark the rest.host.name and rest.port options as deprecated? 2) I needed to also address the advertised hostname / port. With multiple listeners it is not

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

The broker's configuration options are "listeners" (plural) and "listeners.security.protocol.map". I agree that following the pattern set by the broker is better, so these are really good ideas. However, at this point I don't see a need for the "listeners.security.procotol.map", which for the

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

+1 to this proposal. On Mon, Oct 16, 2017 at 7:49 AM, Jakub Scholz wrote: > I was having some more thoughts about it. We can simply take over what > Kafka broker implements for the listeners: > - We can take over the "listener" and "listener.security.protocol.map" > options to

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

I was having some more thoughts about it. We can simply take over what Kafka broker implements for the listeners: - We can take over the "listener" and "listener.security.protocol.map" options to define multiple REST listeners and the security protocol they should use - The HTTPS interface will by

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

I agree, adding both HTTP and HTTPS is not complicated. I just didn't saw the use case for it. But I can add it. Would you add just support for a single HTTP and single HTTPS interface? Or do you see some value even in allowing more than 2 interfaces (for example one HTTP and two HTTPS with

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

I agree with Randall. Actually I had the same thought during first round of review. On Fri, Oct 13, 2017 at 9:25 AM, Randall Hauch wrote: > Also, do we need these properties to be preceded with `rest`? I'd argue > that we're just configuring the worker's SSL information, and

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Also, do we need these properties to be preceded with `rest`? I'd argue that we're just configuring the worker's SSL information, and that the REST API would just use that. If we added another non-REST API, we'd want to use the same security configuration. It's not that complicated in Jetty to

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

It'd be useful to specify the default values for the configuration properties. On Tue, Oct 10, 2017 at 2:53 AM, Jakub Scholz wrote: > FYI: Based on Ewen's suggestion from the related JIRA, I added a > clarification to the KIP that it doesn't do anything around authorization / >

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

FYI: Based on Ewen's suggestion from the related JIRA, I added a clarification to the KIP that it doesn't do anything around authorization / ACLs. While authorization / ACLs would be for sure valuable feature I would prefer to leave it for different KIP. Jakub On Mon, Oct 9, 2017 at 5:25 PM,

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Sorry, that is a typo. It should be rest.ssl.client.auth and it should correspond to the ssl.client.auth option in the broker. The rest.ssl.client.auth option would support values of required, requested and none (none being the default). It will control whether: * the connecting client is

Re: [DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

For rest.ssl.clientAuth , I don't find counterpart in existing code. Can you add explanation on the KIP ? Thanks On Mon, Oct 9, 2017 at 8:25 AM, Jakub Scholz wrote: > Hi, > > I would like to start a discussion about KIP-208: Add SSL support to Kafka > Connect REST interface (

[DISCUSS] KIP-208: Add SSL support to Kafka Connect REST interface

Hi, I would like to start a discussion about KIP-208: Add SSL support to Kafka Connect REST interface ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-208%3A+Add+SSL+support+to+Kafka+Connect+REST+interface ). I think this would be useful feature to improve the security of Kafka Connect.