Well, I feel a bit stupid on this one . . . .
Guess what happens when I discover the following ACL rule that was in place?
Current ACLs for resource `Group:*`:
User:* has Allow permission for operations: Read from hosts: *
And then guess what happens when I delete that rule?
Yes, you guessed it . . . it works as you would expect and limits access to the
specifically defined group.
-----Original Message-----
From: Chapin, Ryan <ryan.cha...@hughes.com>
Sent: Monday, January 07, 2019 2:12 PM
To: users@kafka.apache.org
Subject: Unable to Restrict Access to Consumer Group ACLs
WARNING: The sender of this email could not be validated and may not match the
person in the "From" field.
CAUTION: This email originated from outside of the organization. Do not click
links or open attachments unless you recognize the sender and know the content
is safe.
I do not seem to be able to restrict access to specific users for specific
consumer groups.
It seems as though I am doing something wrong, or that there is a bug as I
imagine what I am trying to do is very straight forward.
We have an 11 node cluster running the 2.11-1.0.0 open source version of Kafka
in conjunction with an open source Zookeeper distro (zookeeper-3.4.6-1.el7)
from the bigtop repo. The cluster is secured via Kerberos (AD) authentication,
over TLS. All nodes communicate with each other over authenticated TLS
connections as well.
We are using the Zookeeper based authorization mechanism with the following
configs from Kafka's server.properties (the comments also indicate the only
Zookeeper config items specific to the ACL mechanism):
######################################################################
# This setting ensures that the acl settings for znodes that are # created by
kafka are not modifiable by other users that can connect # to ZooKeeper. This
has no effect on the data that can be accessed # in kafka, but ensures that
outside actors cannot modify the znodes # that enable kafka to maintain state
on the topics.
#
zookeeper.set.acl=true
######################################################################
# Given that we have figured out that the two following ZooKeeper # configs:
#
# kerberos.removeHostFromPrincipal=true
# kerberos.removeRealmFromPrincipal=true
#
# will take a principal in the form of
#
# kafka/kafka01.prod.quasar.nadops....@gtn.nadops.net
#
# and 'convert' it to:
#
# kafka
#
# This means that all of the kafka brokers now 'share' a username and # we can
easily configure the super.users to grant all access to all # znodes to all of
the brokers without having to continually update # this config and/or set it
conditionally based on the environment # and list of hosts.
#
super.users=User:kafka
######################################################################
# Authorization Configs:
#
# The following is a stepping stone using the Zookeeper based # authorization
mechanism
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
# Since we were able to sort out the ability to define the kafka # brokers as
super.users, we can, by default ensure that only those # users that have
specifically configured acls can get access to # kafka resources.
#
allow.everyone.if.no.acl.found=false
I would like to be able to provide self-service access to certain topics such
that certain topics allow read access to any authenticated user from any host.
Further, I want to ensure that only certain users can use specifically defined
consumer groups when reading from those topics.
For example:
There is a topic "rchapin_topic" configured with three partitions. There is a
production application that is reading from this topic and driving a customer
facing dashboard. The production application is authenticating as the
"rchapin" user with the "rchapin_group" consumer id.
Any other authenticated user should be able to read from that topic, but SHOULD
NOT be able to do so with the "rchapin_group" consumer group. If any other
consumer starts reading from the topic with that consumer group, it will start
siphoning off data from the production application.
Following are the ACLs that we have attempted to put in place that we figure
would allow us to configure such a use case.
Current ACLs for resource `Topic:rchapin_topic`:
User:rchapin has Allow permission for operations: Read from hosts: *
User:rchapin has Allow permission for operations: Write from hosts: *
User:rchapin has Allow permission for operations: Describe from hosts: *
User:* has Allow permission for operations: Describe from hosts: *
User:* has Allow permission for operations: Read from hosts: *
Current ACLs for resource `Group:rchapin_group`:
User:rchapin has Allow permission for operations: Read from hosts: *
With the aforementioned configurations I can connect as any other authenticated
use to read data from the rchapin topic using the rchapin_group consumer group
and begin siphoning off data from the rchapin production application.
If I add an ACL that specifically adds "deny-principal" for all users for the
group as follows:
# kafka-acls.sh --authorizer-properties zookeeper.connect=zk2-01 --add
--deny-principal User:* --group rchapin_group Adding ACLs for resource
`Group:rchapin_group`:
User:* has Deny permission for operations: All from hosts: *
Current ACLs for resource `Group:rchapin_group`:
User:rchapin has Allow permission for operations: Read from
hosts: *
User:* has Deny permission for operations: All from hosts: *
Then no one is able to read from that topic with the rchapin_group consumer id,
but any authenticated user can connect with another group id to consume from
the topic.
Is it possible to restrict access to consumer group to a specific user? If so,
what am I doing wrong? Or is there a bug in this version of Kafka?
Any help is greatly appreciated.
-- Ryan Chapin
