Not sure if this is the exact jaas.conf that you have, because this has
mismatched passwords:
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="someuser"
user_kafka="somePassword"
password="kafka-password";
};
To use username "some user", password "somePassword", the config should be:
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="someuser"
user_*someuser*="somePassword"
password="*somePassword*";
};
But I would have expected to see an error in the Kafka logs if the
inter-broker config was incorrect.
I am assuming all your hostnames can be found from the different machines
since PLAINTEXT was working earlier. But it would be worth checking that
mykafka01 can lookup mybalancer01. It would be worth running Kafka and a
console producer with debug logging turned on. Kafka uses
*config/log4j.properties* and console producer uses
*config/tools-log4j.properties.* Since you are testing with PLAINTEXT, it
should be easy to run a console producer with just standard arguments with
bootstrap server set to mybalancer01:9093.
On Mon, Nov 21, 2016 at 4:37 PM, Zac Harvey <zac.har...@welltok.com> wrote:
> Thanks again. So this might be very telling of the underlying problem:
>
>
> I did what you suggested:
>
>
> 1) I disabled (actually deleted) the first rule; then
>
> 2) I changed the load balancer's second (which is now its only) rule to
> accept TCP:9093 and to translate that to TCP:9093, making the conneciton
> PLAINTEXT all the way through to Kafka; then
>
> 3) I tried connecting a Scala consumer to the load balancer URL (
> mybalancer01.example.com) and I'm getting that ClosedChannelException
>
>
> For now there is only one Kafka broker sitting behind the load balancer.
> It's server.properties look like:
>
>
> listeners=PLAINTEXT://:9093,SASL_PLAINTEXT://:9092
>
> advertised.listeners=PLAINTEXT://mybalancer01.example.com:9093,SASL_
> PLAINTEXT://mykafka01.example.com:9092
>
> advertised.host.name=mykafka01.example.com
>
> security.inter.broker.protocol=SASL_PLAINTEXT
>
> sasl.enabled.mechanisms=PLAIN
>
> sasl.mechanism.inter.broker.protocol=PLAIN
>
> broker.id=1
>
> num.partitions=4
>
> zookeeper.connect=zkA:2181,zkB:2181,zkC:2181
>
> num.network.threads=3
>
> num.io.threads=8
>
> socket.send.buffer.bytes=102400
>
> socket.receive.buffer.bytes=102400
>
> log.dirs=/tmp/kafka-logs
>
> num.recovery.threads.per.data.dir=1
>
> log.retention.hours=168
>
> log.segment.bytes=1073741824
>
> log.retention.check.interval.ms=300000
>
> zookeeper.connection.timeout.ms=6000
>
> offset.metadata.max.bytes=4096
>
>
> Above, 'zkA', 'zkB' and 'zkC' are defined inside `/etc/hosts` and are
> valid server names.
>
>
> And then inside the kafka-run-class.sh script, instead of the default:
>
>
> if [ -z "$KAFKA_OPTS" ]; then
>
> KAFKA_OPTS=""
>
> fi
>
>
> I have:
>
>
> if [ -z "$KAFKA_OPTS" ]; then
>
> KAFKA_OPTS="-Djava.security.auth.login.config=/opt/kafka/
> config/jaas.conf"
>
> fi
>
>
> I also added the /opt/kafka/config/jaas.conf file like you suggested, and
> only changed the names of users and passwords:
>
>
> KafkaServer {
>
> org.apache.kafka.common.security.plain.PlainLoginModule required
>
> username="someuser"
>
> user_kafka="somePassword"
>
> password="kafka-password";
>
> };
>
>
> The fact that I can no longer even consume from a topic over PLAINTEXT
> (which is a regression of where I was before we started trying to add SSL)
> tells me there is something wrong in either server.properties or jaas.conf.
> I've checked the Kafka broker logs (server.log) each time I try connecting
> and this is the only line that gets printed:
>
>
> [2016-11-21 15:18:14,859] INFO [Group Metadata Manager on Broker 2]:
> Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.
> GroupMetadataManager)
>
>
> Not sure if that means anything. Any idea where I might be going wrong?
> Thanks again!
>
> ________________________________
> From: Rajini Sivaram <rajinisiva...@googlemail.com>
> Sent: Monday, November 21, 2016 11:03:14 AM
> To: users@kafka.apache.org
> Subject: Re: Can Kafka/SSL be terminated at a load balancer?
>
> Rule #1 and Rule #2 cannot co-exist. You are basically configuring your LB
> to point to a Kafka broker and you are pointing each Kafka broker to point
> to a LB. So you need a pair of ports with a security protocol for the
> connection to work. With two rules, Kafka picks up the wrong LB port for
> one of the security protocols.
>
> If you want to try without SSL first, the simplest way to try it out would
> be to disable Rule #1 and change Rule #2 to use port 9093 instead of 9095.
> Then you should be able to connect using PLAINTEXT (the test that is
> currently not working).
>
> I think you have the configuration:
>
> advertised.listeners=PLAINTEXT://mybalancer01.example.com:9093
> ,SASL_PLAINTEXT://mykafka01.example.com:9092
>
> And you have a client connecting with PLAINTEXT on mybalancer01:*9095*. The
> first connection would work, but subsequent connections would use the
> address provided by Kafka from advertised.listeners. The client will start
> connecting with PLAINTEXT on mybalancer01:*9093*, which is expecting SSL.
> If you disable Rule #1 and change Rule #2 to use port 9093, you should be
> able to test PLAINTEXT without changing Kafka config.
>
> On Mon, Nov 21, 2016 at 3:32 PM, Zac Harvey <zac.har...@welltok.com>
> wrote:
>
> > In the last email I should have mentioned: don't pay too much attention
> to
> > the code snippet, and after reviewing it, I can see it actually
> incomplete
> > (I forgot to include the section where I configure the topics and broker
> > configs to talk to Kafka!).
> >
> >
> > What I'm really concerned about is that before we added all these SSL
> > configs, I had plaintext (plaintext:9092 in/out of the load balancer
> > to/from Kafka) working fine. Now my consumer code can't even connect to
> the
> > load balancer/Kafka.
> >
> >
> > So I guess what I was really asking was: does that exception
> > (ClosedChannelException) indicate bad configs on the Kafka broker?
> >
> > ________________________________
> > From: Zac Harvey <zac.har...@welltok.com>
> > Sent: Thursday, November 17, 2016 4:44:06 PM
> > To: users@kafka.apache.org
> > Subject: Can Kafka/SSL be terminated at a load balancer?
> >
> > We have two Kafka nodes and for reasons outside of this question, would
> > like to set up a load balancer to terminate SSL with producers (clients).
> > The SSL cert hosted by the load balancer will be signed by trusted/root
> CA
> > that clients should natively trust.
> >
> >
> > Is this possible to do, or does Kafka somehow require SSL to be setup
> > directly on the Kafka servers themselves?
> >
> >
> > Thanks!
> >
>
>
>
> --
> Regards,
>
> Rajini
>
--
Regards,
Rajini
