Re: Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

[Darshan]

Hi Jaikiran

My producer is getting *WARN Error while fetching metadata with correlation
id 1 : {Topic4006=TOPIC_AUTHORIZATION_FAILED}
(org.apache.kafka.clients.NetworkClient)* error.

To test it out my producer is the default Kafka console client which I am
trying to use like this: *bin/kafka-console-producer.sh --broker-list
Kafka1:9092 --topic Topic4006* and then I see the above mentioned error
when I type something to send a message.

Here is my server.properties file if that helps.

# ID and basic topic creation
broker.id=1
auto.create.topics.enable=true
delete.topic.enable=true

# LISTERN Settings
listeners=INTERNAL://1.1.1.165:9092,EXTERNAL://172.21.190.176:9093
advertised.listeners=INTERNAL://1.1.1.165:9092,EXTERNAL://17
2.21.190.176:9093
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
inter.broker.listener.name=INTERNAL
host.name=172.21.190.176

# Security Settings
ssl.keystore.location=keystore.jks
ssl.keystore.password=password
ssl.key.password=password
ssl.truststore.location=truststore.jks
ssl.truststore.password=password
ssl.keystore.type=JKS
ssl.truststore.type=JKS
security.protocol=SSL
ssl.client.auth=required
allow.everyone.if.no.acl.found=false
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:CN=Kafka1

Thanks.

On Wed, Dec 20, 2017 at 8:16 PM, Jaikiran Pai 
wrote:

> When you say not able to write to a Kafka broker, do you mean your
> producer isn't able to produce a message? What does your producer configs
> look like? What exact exception, error or DEBUG logs do you see when you
> attempt this?
>
> We do use a similar setup, so I do know that such a configuration works
> fine.
>
> -Jaikiran
>
>
>
> On 21/12/17 1:49 AM, Darshan wrote:
>
>> Hi Jaikiran
>>
>> With that config, my internal kafka client can't write to the Kafka
>> broker.
>> What I am looking for is that internal client can write to Kafka topic
>> without having to have any truststore setup, while external kafka client
>> MUST have certificate, and truststore setup and can read only if ACLs are
>> programmed for that topic.
>>
>> Any idea if such a thing exists ?
>>
>> Thanks.
>>
>>
>> On Tue, Dec 19, 2017 at 10:10 PM, Jaikiran Pai 
>> wrote:
>>
>> What exact issue are you running into with thta configs?
>>>
>>> -Jaikiran
>>>
>>>
>>>
>>> On 20/12/17 7:24 AM, Darshan wrote:
>>>
>>> Anyone ?

 On Mon, Dec 18, 2017 at 7:25 AM, Darshan 
 wrote:

 Hi

> I am wondering if there is a way to run the SSL and PLAINTEXT mode
> together ? I am running Kafka 10.2.1. We want our internal clients to
> use
> the PLAINTEXT mode to write to certain topics, but any external clients
> should use SSL to read messages on those topics. We also want to
> enforce
> ACLs.
>
> To try this out, I modified my server.properties as follows, but
> without
> any luck. Can someone please let me know if it needs any change ?
>
> listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
> advertised.listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://
> 172.1.1.157:9093
> listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
> inter.broker.listener.name=INTERNAL
>
> ssl.keystore.location=/opt/keystores/keystotr.jks
> ssl.keystore.password=ABCDEFGH
> ssl.key.password=ABCDEFGH
> ssl.truststore.location=/opt/keystores/truststore.jks
> ssl.truststore.password=ABCDEFGH
> ssl.keystore.type=JKS
> ssl.truststore.type=JKS
> security.protocol=SSL
> ssl.client.auth=required
> # allow.everyone.if.no.acl.found=false
> allow.everyone.if.no.acl.found=true
> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
> super.users=User:CN=KafkaBroker01
>
> Thanks.
>
> --Darshan
>
>
>
>

Re: Vids Re: Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

[Martin Gainty]

From: svsuj...@gmail.com <svsuj...@gmail.com>
Sent: Sunday, March 11, 2018 4:22 PM
To: users@kafka.apache.org
Cc: Ismael Juma; rajinisiva...@gmail.com
Subject: Vids Re: Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

Chic bhari

Sent from my iPhone
 GC
> On Dec 19, 2017, at 5:54 PM, Darshan <purandare.dars...@gmail.com> wrote:
> Srvy cdhdjtiyyjj
> Anyone ?
> Y. Yum m
> On Mon, Dec 18, 2017 at 7:25 AM, Darshan <purandar...@gmail.com>
> wrote:
>
>> Hi
>>
>> I am wondering if there is a way to know nhj mbiib the SSL and PLAINTEXT mode
>> together ? I am running Kafka 10.2.1. We want our internal clients to use
>> the PLAINTEXT mode to write to certain topics, but any external clients
>> should use SSL to read messages on those topics. We also want to enforce
>> ACLs.ccds
>>
>> To try this out, I modified my server.properties as follows, but without
>> any luck. Can someone please let me know if it needs any change ?
>>
>> listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
MG>where is your need SSL declaration? here is example
MG>listeners=SSL://:9093

>> advertised.listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://
>> 172.1.1.157:9093
>> listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
>> inter.broker.listener.name=INTERNAL
>>
>> ssl.keystore.location=/opt/keystores/keystotr.jks
MG>are you certain the jks file name is keystotr.jks?

>> ssl.keystore.password=ABCDEFGH
>> ssl.key.password=ABCDEFGH
>> ssl.truststore.location=/opt/keystores/truststore.jks
>> ssl.truststore.password=ABCDEFGH
>> ssl.keystore.type=JKS
>> ssl.truststore.type=JKS
>> security.protocol=SSL
>> ssl.client.auth=required
#you are missing the following ssl entries (value on right of = sign is 
placeholder)

ssl.cipher.suites = null
ssl.client.auth = none
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null

ssl.keymanager.algorithm = SunX509

ssl.protocol = TLS

#match ssl.provider listed in $JAVA_HOME/jre/lib/java.security
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX

>> # allow.everyone.if.no.acl.found=false
>> allow.everyone.if.no.acl.found=true
>> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
>> super.users=User:CN=KafkaBroker01
MG>your DN is incomplete.. here is a complete DN example
super.users=User:CN=KafkaBroker01.example.com,OU=Users,O=ConfluentOffice,L=London,ST=London,C=GB
>>
>> Thanks.
>>
>> --Darshan
MG>ismael please confirm
>>

Vids Re: Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

[svsujeet]

Chic bhari 

Sent from my iPhone
 GC 
> On Dec 19, 2017, at 5:54 PM, Darshan  wrote:
> Srvy cdhdjtiyyjj
> Anyone ?
> Y. Yum m
> On Mon, Dec 18, 2017 at 7:25 AM, Darshan 
> wrote:
> 
>> Hi
>> 
>> I am wondering if there is a way to know nhj mbiib the SSL and PLAINTEXT mode
>> together ? I am running Kafka 10.2.1. We want our internal clients to use
>> the PLAINTEXT mode to write to certain topics, but any external clients
>> should use SSL to read messages on those topics. We also want to enforce
>> ACLs.ccds
>> 
>> To try this out, I modified my server.properties as follows, but without
>> any luck. Can someone please let me know if it needs any change ?
>> 
>> listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
>> advertised.listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://
>> 172.1.1.157:9093
>> listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
>> inter.broker.listener.name=INTERNAL
>> 
>> ssl.keystore.location=/opt/keystores/keystotr.jks
>> ssl.keystore.password=ABCDEFGH
>> ssl.key.password=ABCDEFGH
>> ssl.truststore.location=/opt/keystores/truststore.jks
>> ssl.truststore.password=ABCDEFGH
>> ssl.keystore.type=JKS
>> ssl.truststore.type=JKS
>> security.protocol=SSL
>> ssl.client.auth=required
>> # allow.everyone.if.no.acl.found=false
>> allow.everyone.if.no.acl.found=true
>> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
>> super.users=User:CN=KafkaBroker01
>> 
>> Thanks.
>> 
>> --Darshan
>> 

Re: Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

[Jaikiran Pai]

When you say not able to write to a Kafka broker, do you mean your 
producer isn't able to produce a message? What does your producer 
configs look like? What exact exception, error or DEBUG logs do you see 
when you attempt this?


We do use a similar setup, so I do know that such a configuration works 
fine.


-Jaikiran


On 21/12/17 1:49 AM, Darshan wrote:

Hi Jaikiran

With that config, my internal kafka client can't write to the Kafka broker.
What I am looking for is that internal client can write to Kafka topic
without having to have any truststore setup, while external kafka client
MUST have certificate, and truststore setup and can read only if ACLs are
programmed for that topic.

Any idea if such a thing exists ?

Thanks.


On Tue, Dec 19, 2017 at 10:10 PM, Jaikiran Pai 
wrote:


What exact issue are you running into with thta configs?

-Jaikiran



On 20/12/17 7:24 AM, Darshan wrote:


Anyone ?

On Mon, Dec 18, 2017 at 7:25 AM, Darshan 
wrote:

Hi

I am wondering if there is a way to run the SSL and PLAINTEXT mode
together ? I am running Kafka 10.2.1. We want our internal clients to use
the PLAINTEXT mode to write to certain topics, but any external clients
should use SSL to read messages on those topics. We also want to enforce
ACLs.

To try this out, I modified my server.properties as follows, but without
any luck. Can someone please let me know if it needs any change ?

listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
advertised.listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://
172.1.1.157:9093
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
inter.broker.listener.name=INTERNAL

ssl.keystore.location=/opt/keystores/keystotr.jks
ssl.keystore.password=ABCDEFGH
ssl.key.password=ABCDEFGH
ssl.truststore.location=/opt/keystores/truststore.jks
ssl.truststore.password=ABCDEFGH
ssl.keystore.type=JKS
ssl.truststore.type=JKS
security.protocol=SSL
ssl.client.auth=required
# allow.everyone.if.no.acl.found=false
allow.everyone.if.no.acl.found=true
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:CN=KafkaBroker01

Thanks.

--Darshan

Re: Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

[Darshan]

Hi Jaikiran

With that config, my internal kafka client can't write to the Kafka broker.
What I am looking for is that internal client can write to Kafka topic
without having to have any truststore setup, while external kafka client
MUST have certificate, and truststore setup and can read only if ACLs are
programmed for that topic.

Any idea if such a thing exists ?

Thanks.


On Tue, Dec 19, 2017 at 10:10 PM, Jaikiran Pai 
wrote:

> What exact issue are you running into with thta configs?
>
> -Jaikiran
>
>
>
> On 20/12/17 7:24 AM, Darshan wrote:
>
>> Anyone ?
>>
>> On Mon, Dec 18, 2017 at 7:25 AM, Darshan 
>> wrote:
>>
>> Hi
>>>
>>> I am wondering if there is a way to run the SSL and PLAINTEXT mode
>>> together ? I am running Kafka 10.2.1. We want our internal clients to use
>>> the PLAINTEXT mode to write to certain topics, but any external clients
>>> should use SSL to read messages on those topics. We also want to enforce
>>> ACLs.
>>>
>>> To try this out, I modified my server.properties as follows, but without
>>> any luck. Can someone please let me know if it needs any change ?
>>>
>>> listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
>>> advertised.listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://
>>> 172.1.1.157:9093
>>> listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
>>> inter.broker.listener.name=INTERNAL
>>>
>>> ssl.keystore.location=/opt/keystores/keystotr.jks
>>> ssl.keystore.password=ABCDEFGH
>>> ssl.key.password=ABCDEFGH
>>> ssl.truststore.location=/opt/keystores/truststore.jks
>>> ssl.truststore.password=ABCDEFGH
>>> ssl.keystore.type=JKS
>>> ssl.truststore.type=JKS
>>> security.protocol=SSL
>>> ssl.client.auth=required
>>> # allow.everyone.if.no.acl.found=false
>>> allow.everyone.if.no.acl.found=true
>>> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
>>> super.users=User:CN=KafkaBroker01
>>>
>>> Thanks.
>>>
>>> --Darshan
>>>
>>>
>

Re: Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

[Jaikiran Pai]

What exact issue are you running into with thta configs?

-Jaikiran


On 20/12/17 7:24 AM, Darshan wrote:

Anyone ?

On Mon, Dec 18, 2017 at 7:25 AM, Darshan 
wrote:


Hi

I am wondering if there is a way to run the SSL and PLAINTEXT mode
together ? I am running Kafka 10.2.1. We want our internal clients to use
the PLAINTEXT mode to write to certain topics, but any external clients
should use SSL to read messages on those topics. We also want to enforce
ACLs.

To try this out, I modified my server.properties as follows, but without
any luck. Can someone please let me know if it needs any change ?

listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
advertised.listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://
172.1.1.157:9093
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
inter.broker.listener.name=INTERNAL

ssl.keystore.location=/opt/keystores/keystotr.jks
ssl.keystore.password=ABCDEFGH
ssl.key.password=ABCDEFGH
ssl.truststore.location=/opt/keystores/truststore.jks
ssl.truststore.password=ABCDEFGH
ssl.keystore.type=JKS
ssl.truststore.type=JKS
security.protocol=SSL
ssl.client.auth=required
# allow.everyone.if.no.acl.found=false
allow.everyone.if.no.acl.found=true
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:CN=KafkaBroker01

Thanks.

--Darshan

Re: Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

[Darshan]

Anyone ?

On Mon, Dec 18, 2017 at 7:25 AM, Darshan 
wrote:

> Hi
>
> I am wondering if there is a way to run the SSL and PLAINTEXT mode
> together ? I am running Kafka 10.2.1. We want our internal clients to use
> the PLAINTEXT mode to write to certain topics, but any external clients
> should use SSL to read messages on those topics. We also want to enforce
> ACLs.
>
> To try this out, I modified my server.properties as follows, but without
> any luck. Can someone please let me know if it needs any change ?
>
> listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
> advertised.listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://
> 172.1.1.157:9093
> listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
> inter.broker.listener.name=INTERNAL
>
> ssl.keystore.location=/opt/keystores/keystotr.jks
> ssl.keystore.password=ABCDEFGH
> ssl.key.password=ABCDEFGH
> ssl.truststore.location=/opt/keystores/truststore.jks
> ssl.truststore.password=ABCDEFGH
> ssl.keystore.type=JKS
> ssl.truststore.type=JKS
> security.protocol=SSL
> ssl.client.auth=required
> # allow.everyone.if.no.acl.found=false
> allow.everyone.if.no.acl.found=true
> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
> super.users=User:CN=KafkaBroker01
>
> Thanks.
>
> --Darshan
>

Running SSL and PLAINTEXT mode together (Kafka 10.2.1)

[Darshan]

Hi

I am wondering if there is a way to run the SSL and PLAINTEXT mode together
? I am running Kafka 10.2.1. We want our internal clients to use the
PLAINTEXT mode to write to certain topics, but any external clients should
use SSL to read messages on those topics. We also want to enforce ACLs.

To try this out, I modified my server.properties as follows, but without
any luck. Can someone please let me know if it needs any change ?

listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
advertised.listeners=INTERNAL://10.10.10.64:9092,EXTERNAL://172.1.1.157:9093
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
inter.broker.listener.name=INTERNAL

ssl.keystore.location=/opt/keystores/keystotr.jks
ssl.keystore.password=ABCDEFGH
ssl.key.password=ABCDEFGH
ssl.truststore.location=/opt/keystores/truststore.jks
ssl.truststore.password=ABCDEFGH
ssl.keystore.type=JKS
ssl.truststore.type=JKS
security.protocol=SSL
ssl.client.auth=required
# allow.everyone.if.no.acl.found=false
allow.everyone.if.no.acl.found=true
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:CN=KafkaBroker01

Thanks.

--Darshan