Re: [DISCUSS] KIP-519: Make SSL context/engine configuration extensible

2019-10-13 Thread Maulin Vasavada
Hi Clement

1) existing validation code will remain in SslFactory
2) the createEngine() method in SslEngineBuilder will move to SslFactory
and the client/server mode setting will go there (I documented this in the
latest KIP update)

In the current KIP I am proposing (as per the latest updates) to make
SSLContext loading/configuration/creation pluggable. I am not suggesting we
do/repeat anything that is already addressed by the existing Providers for
SSLContext implementation. The createEngine() method (which will move to
SslFactory) will call SslContextFactory.create() to get references to the
SSLContext and then call SSLContext#createEngine(peer, host) and set
client/server mode as it does today. I'll try to put that in a sequence
diagram and update the KIP to make it clearer.

So to your question about SslFactory returning SSLContext - I am saying
register SslContextFactory interface to provide the SSLContext object
instead and keep SslFactory more-or-less as it is today with some
additional responsibility of createEngine() method.

Thanks
Maulin

Thanks
Maulin




On Fri, Oct 11, 2019 at 6:17 AM Pellerin, Clement 
wrote:

> Can you clarify a few points for me?
>
> The two stumbling blocks we have are:
> 1) reuse of the validation code in the existing SslFactory
> 2) the client/server mode on the SSLEngine
>
> How do you deal with those issues in your new proposal?
>
> My use case is to register a custom SslFactory that returns an SSLContext
> previously created elsewhere in the application. Can your new proposal
> handle this use case?
>
> -Original Message-
> From: Maulin Vasavada [mailto:maulin.vasav...@gmail.com]
> Sent: Friday, October 11, 2019 2:13 AM
> To: dev@kafka.apache.org
> Subject: Re: [DISCUSS] KIP-519: Make SSL context/engine configuration
> extensible
>
> Check this out-
>
> https://github.com/apache/httpcomponents-core/blob/master/httpcore5/src/main/java/org/apache/hc/core5/ssl/SSLContextBuilder.java#L349
>
> This is exactly what I mean by using existing provider's SSLContext
> implementation and customizing it with our data points. The similar thing
> Kafka's SslEngineBuilder is doing right now.
>
> On Thu, Oct 10, 2019 at 11:06 PM Maulin Vasavada <
> maulin.vasav...@gmail.com>
> wrote:
>
> > You meant JSSE not JCE right? We are not talking about cryptographic
> > providers we are talking about ssl providers hence JSSE.
> >
> > I do understand how JSSE Providers work and also the impact of multiple
> > JSSE providers with same algorithms in same JVM along with sequencing
> > challenges for the same.
> >
> > Like you said- we need to allow customizing the configuration for
> > SSLContext, so how many ways we have?
> >
> > Option-1: Write a custom JSSE Provider with our SSLContext
> >
> > Option-2: Use whichever SSLContext impl that you get from existing JSSE
> > Provider for SSLContext AND customize data for key material, trust
> material
> > AND secure random.
> >
> > Which one you prefer for this context?
> >
> > I feel we are making it complicated for no reason. It is very simple -
> > When we need to have SSL we need data points like - 1) Keys, 2) Trust
> certs
> > and 3) Secure Random which is feed to SSLContext and we are done. So we
> can
> > keep existing Kafka implementation as is by just making those data points
> > pluggable. Now SecureRandom is already pluggable via
> > 'ssl.secure.random.implementation' so that leaves us with keys and
> trusted
> > certs. For that purpose I raised KIP-486 BUT everybody feels we still
> need
> > higher level of pluggability hence this KIP.
> >
> > I"ve been taking advice from the domain experts and Application security
> > teams and to them it is very straight-forward - Make SSLContext
> > configuration/building pluggable and that's it!
> >
> > Thanks
> > Maulin
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Oct 7, 2019 at 5:47 AM Pellerin, Clement <
> clement_pelle...@ibi.com>
> > wrote:
> >
> >> If I understand correctly, you are proposing to abandon the idea of a
> >> pluggable extension point for SSL in Kafka because we could rely on the
> JCE
> >> provider mechanism.
> >>
> >> I will reiterate that nobody does it that way. That in itself should be
> >> enough but let's discuss some of the reasons why.
> >>
> >> Changing the order of the JCE providers in the java.security file
> affects
> >> all java applications so you probably don't want to do it there.
> Changing
> >> the order of the JCE providers in the JVM instance affects all code it
> >> runs. Your library is not alone in the JVM process and other code will
> want
> >> regular SSLContext instances. That leaves you with the only option of
> >> specifying the provider explicitly when you create the SSLContext
> instance
> >> in Kafka. That would work, as long as your users don't mess things up
> with
> >> the very common configuration approaches above.
> >>
> >> A JCE SSLContext provider is intended to be a mechanism to replace the
> >> SSLContext implementation. Our p

[DISCUSS] KIP-535: Allow state stores to serve stale reads during rebalance

2019-10-13 Thread Navinder Brar
Hi,
Starting a discussion on the KIP to Allow state stores to serve stale reads 
during 
rebalance(https://cwiki.apache.org/confluence/display/KAFKA/KIP-535%3A+Allow+state+stores+to+serve+stale+reads+during+rebalance).
Thanks & Regards,Navinder 
LinkedIn


[jira] [Created] (KAFKA-9035) Improve

2019-10-13 Thread Ivan Yurchenko (Jira)
Ivan Yurchenko created KAFKA-9035:
-

 Summary: Improve 
 Key: KAFKA-9035
 URL: https://issues.apache.org/jira/browse/KAFKA-9035
 Project: Kafka
  Issue Type: Improvement
  Components: KafkaConnect
Affects Versions: 2.2.1, 2.3.0, 2.1.1, 2.2.0, 2.1.0, 2.0.1, 2.0.0
Reporter: Ivan Yurchenko
Assignee: Ivan Yurchenko


Kafka Connect workers in the distributed mode can be set up so that they have 
the same advertised URL (e.g. 
{{[http://127.0.0.1:8083|http://127.0.0.1:8083/]}}). When a request (e.g., for 
connector creation) lands on a worker that is not the leader, it will be 
forwarded to the leader's advertised URL. However, if advertised URLs are all 
the same, it might never reach the leader (due to the limited number of 
forwards). (See https://issues.apache.org/jira/browse/KAFKA-7121 for an 
example.)

I propose to address this by detecting such a situation and warning the user on 
the log.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)