Hi Vahid Thank you for sharing link to set it up. It is really a very useful document. When I ran describe command for group I see this error
bin/kafka-consumer-groups --bootstrap-server host:9097 --describe --group arun-group --command-config etc/kafka/producer.properties Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers). [2017-06-16 11:32:23,790] INFO Successfully logged in. (org.apache.kafka.common.security.authenticator.AbstractLogin) Error: Executing consumer group command failed due to The group coordinator is not available. I googled to figure out the issue and many say that it may be because of the port which I am not convinced. Any help is highly appreciated. Thanks Arunkumar Pichaimuthu, PMP -------------------------------------------- On Thu, 6/15/17, Vahid S Hashemian <vahidhashem...@us.ibm.com> wrote: Subject: Re: UNKNOWN_TOPIC_OR_PARTITION with SASL_PLAINTEXT ACL To: users@kafka.apache.org Date: Thursday, June 15, 2017, 6:49 PM Hi Arunkumar, Could you please take a look at this article: https://developer.ibm.com/opentech/2017/05/31/kafka-acls-in-practice/ The error message you posted earlier suggests there is some missing ACL (as indicated in the article). Let me know if that doesn't resolve the issue. Thanks. --Vahid From: Arunkumar <pm_arunku...@yahoo.com.INVALID> To: <users@kafka.apache.org> Date: 06/15/2017 04:37 PM Subject: Re: UNKNOWN_TOPIC_OR_PARTITION with SASL_PLAINTEXT ACL Hi Vahid Thank you for quick response. I set the ACL for topic and also created jaas permission as per the document for both producer and consumer. I have set what I have posted below. Do I need to set ACL like we set for Topics -- bin/kafka-acls --topic * --add -allow-host host:9097 --allow-principal User:arun --operation Write --authorizer-properties zookeeper.connect=host:2182 ? Please let me know. If you need all configuration for zookeeper, Broker, producer and consumer. I can share it as well. Thanks in advance KafkaServer { org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret" user_admin="admin-secret" user_arun="Arun123"; }; Client { org.apache.kafka.common.security.plain.PlainLoginModule required username="arun" password="Arun123"; }; KafkaClient { org.apache.kafka.common.security.plain.PlainLoginModule required username="arun" password="Arun123"; }; Thanks Arunkumar Pichaimuthu, PMP -------------------------------------------- On Thu, 6/15/17, Vahid S Hashemian <vahidhashem...@us.ibm.com> wrote: Subject: Re: UNKNOWN_TOPIC_OR_PARTITION with SASL_PLAINTEXT ACL To: users@kafka.apache.org Date: Thursday, June 15, 2017, 6:16 PM Hi Arunkumar, Have you given your Kafka consumer/producer necessary permissions to consume/produce messages? --Vahid From: Arunkumar <pm_arunku...@yahoo.com.INVALID> To: <users@kafka.apache.org> Date: 06/15/2017 04:07 PM Subject: UNKNOWN_TOPIC_OR_PARTITION with SASL_PLAINTEXT ACL Hi I am setting up ACL with SALS_PLAINTEXT. My zookeeper and broker starts without error. But when I try to start my consumer or if I send message through a producer it throws an exception (Both producer and consumer are kafka CLI) Stack trace for my consumer below. Any insight is highly appreciated. Thanks in advance bin/kafka-console-consumer --topic sample1 --from-beginning --consumer.config=etc/kafka/consumer.properties --bootstrap-server hostname:9097 [2017-06-15 17:21:45,286] INFO ConsumerConfig values: auto.commit.interval.ms = 5000 auto.offset.reset = earliest bootstrap.servers = [hostname:9097] check.crcs = true client.id = connections.max.idle.ms = 540000 enable.auto.commit = true exclude.internal.topics = true fetch.max.bytes = 52428800 fetch.max.wait.ms = 500 fetch.min.bytes = 1 group.id = test-consumer-group heartbeat.interval.ms = 1000 interceptor.classes = null key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer max.partition.fetch.bytes = 1048576 max.poll.interval.ms = 300000 max.poll.records = 500 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor] receive.buffer.bytes = 65536 reconnect.backoff.ms = 50 request.timeout.ms = 305000 retry.backoff.ms = 100 sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.mechanism = PLAIN security.protocol = SASL_PLAINTEXT send.buffer.bytes = 131072 session.timeout.ms = 10000 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = path.truststore ssl.truststore.password = [hidden] ssl.truststore.type = JKS value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer (org.apache.kafka.clients.consumer.ConsumerConfig) [2017-06-15 17:21:45,438] INFO Successfully logged in. (org.apache.kafka.common.security.authenticator.AbstractLogin) [2017-06-15 17:21:45,522] INFO Kafka version : 0.10.2.1-cp1 (org.apache.kafka.common.utils.AppInfoParser) [2017-06-15 17:21:45,523] INFO Kafka commitId : 078e7dc02a100018 (org.apache.kafka.common.utils.AppInfoParser) [2017-06-15 17:21:45,781] WARN Error while fetching metadata with correlation id 2 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:45,878] WARN Error while fetching metadata with correlation id 3 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:45,980] WARN Error while fetching metadata with correlation id 4 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,084] WARN Error while fetching metadata with correlation id 5 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,185] WARN Error while fetching metadata with correlation id 6 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,289] WARN Error while fetching metadata with correlation id 7 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,392] WARN Error while fetching metadata with correlation id 8 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,495] WARN Error while fetching metadata with correlation id 9 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,598] WARN Error while fetching metadata with correlation id 10 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,702] WARN Error while fetching metadata with correlation id 11 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,804] WARN Error while fetching metadata with correlation id 12 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:46,908] WARN Error while fetching metadata with correlation id 13 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:47,013] WARN Error while fetching metadata with correlation id 14 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:47,117] WARN Error while fetching metadata with correlation id 15 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:47,220] WARN Error while fetching metadata with correlation id 16 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:47,323] WARN Error while fetching metadata with correlation id 17 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:47,426] WARN Error while fetching metadata with correlation id 18 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:47,530] WARN Error while fetching metadata with correlation id 19 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-06-15 17:21:47,634] WARN Error while fetching metadata with correlation id 20 : {sample1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) Thanks Arunkumar Pichaimuthu, PMP
