kafka-acls.sh --list failed when zookeeper SASL/PLAIN authentication is enabled

[hui happy]

I have a kakfa/zookeeper(embedded zookeeper) cluster with SASL/PLAIN + ACL
enabled.
It worked fine with version kafka_2.12-1.0.0. But recently, I need to
upgrade to kafka_2.12-2.1.0. Unfortunately, the ACL function cannot work
normally.

kafka-acls.sh command failed, for example:

> # echo $KAFKA_OPTS
>
> KAFKA_OPTS=-Djava.security.auth.login.config=/work/sasl/kafka_server_jaas.conf
>
#

# /kafka_2.12-2.1.0/bin/kafka-acls.sh --authorizer
> kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> zookeeper.connect=zookeeper.example.com:2181 --list --topic test-topic



The error message from screen is:

> Error while executing ACL command: KeeperErrorCode = InvalidACL for
> /kafka-acl
> org.apache.zookeeper.KeeperException$InvalidACLException: KeeperErrorCode
> = InvalidACL for /kafka-acl
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:121)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at kafka.zookeeper.AsyncResponse.maybeThrow(ZooKeeperClient.scala:494)
> at kafka.zk.KafkaZkClient.createRecursive(KafkaZkClient.scala:1416)
> at kafka.zk.KafkaZkClient.createAclPaths(KafkaZkClient.scala:931)
> at
> kafka.security.auth.SimpleAclAuthorizer.configure(SimpleAclAuthorizer.scala:96)
> at kafka.admin.AclCommand$.withAuthorizer(AclCommand.scala:78)
> at kafka.admin.AclCommand$.listAcl(AclCommand.scala:119)
> at kafka.admin.AclCommand$.main(AclCommand.scala:56)
> at kafka.admin.AclCommand.main(AclCommand.scala)
>

The zookeeper log is:

> zookeeper.example.com| [2018-12-26 09:46:09,622] ERROR Missing
> AuthenticationProvider for sasl
> (org.apache.zookeeper.server.PrepRequestProcessor)
> zookeeper.example.com| [2018-12-26 09:46:09,622] INFO Got user-level
> KeeperException when processing sessionid:0x167e9e2c60c0003 type:create
> cxid:0x3 zxid:0x1008a txntype:-1 reqpath:n/a Error Path:/kafka-acl
> Error:KeeperErrorCode = InvalidACL for /kafka-acl
> (org.apache.zookeeper.server.PrepRequestProcessor)
> zookeeper.example.com| [2018-12-26 09:46:09,704] INFO Processed
> session termination for sessionid: 0x167e9e2c60c0003
> (org.apache.zookeeper.server.PrepRequestProcessor)
>

The  kafka SASL configure file /work/sasl/kafka_server_jaas.conf content is:

> # cat /work/sasl/kafka_server_jaas.conf
> KafkaServer {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="admin"
> password="adminpwd"
> user_admin="adminpwd"
> user_alice="alicepwd";
> };
>
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="alice"
> password="alicepwd";
> };
>
> Client {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="admin"
> password="adminpwd";
> };
>

And zookeeper SASL configure file zookeeper_jaas.conf content is:

> # cat /work/sasl/zookeeper_jaas.conf
> Server {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="admin"
> password="adminpwd"
> user_admin="adminpwd";
> };
>


Anybody can help this ? thanks.
Hui

Is kafka support dynamic ACL rule

[hui happy]

Hi

As I learned that kafka can use  '--resource-pattern-type prefixed'  to add
rule for prefixed topic.
For example an user 'kafkaclient', we could define a rule let the user can
access all topics start with that user name, i.e., 'kafkaclient--', such
as  'kafkaclient--topic1', 'kafkaclient--topic2', etc.

/opt/kafka/bin/kafka-acls.sh \

  --authorizer-properties zookeeper.connect=zookeeper:2181 \

  --add \

  --allow-principal User:"kafkaclient" \

  --operation All \

  --resource-pattern-type prefixed \

  --topic "kafkaclient--" \


But is it possible to define dynamic user name ?
In above case we know the username is 'kafkaclient', and if there are many
other users, we have to add rule for each user; these rules are similar,
except the user name.

So i want to know if it's possible to just define a single rule, using
dynamic user name, each user could access the topics start with itself
username. something likes:

/opt/kafka/bin/kafka-acls.sh \

  --authorizer-properties zookeeper.connect=zookeeper:2181 \

  --add \

  --allow-principal User:"**" \

  --operation All \

  --resource-pattern-type prefixed \

  --topic "**--" \


Then whatever to add user or add topic later, we don't need to add any
rules.

Thanks.
Hui