Hi Sonke, IP address based security doesn't really work, though. Users can spoof IP addresses. They can poison the ARP cache on a local network, or impersonate a DNS server.
For users who want some access controls, but don't care about security, maybe we should make it easier to use and create users without enabling kerberos or similar? best, Colin On Wed, Jan 24, 2018, at 12:59, Sönke Liebau wrote: > Hi everyone, > > the current ACL functionality in Kafka is a bit limited concerning > host based rules when specifying multiple hosts. A common scenario for > this would be that if have a YARN cluster running Spark jobs that > access Kafka and want to create ACLs based on the ip addresses of the > cluster nodes. > Currently kafka-acls only allows to specify individual ips, so this > would look like > > ./kafka-acls --add --producer \ > --topic test --authorizer-properties zookeeper.connect=localhost:2181 \ > --allow-principal User:spark \ > --allow-host 10.0.0.10 \ > --allow-host 10.0.0.11 \ > --allow-host ... > > which can get unwieldy if you have a 200 node cluster. Internally this > command would not create a single ACL with multiple host entries, but > rather one ACL per host that is specified on the command line, which > makes the ACL listing a bit confusing. > > There are currently a few jiras in various states around this topic: > KAFKA-3531 [1], KAFKA-4759 [2], KAFKA-4985 [3] & KAFKA-5713 [4] > > KAFKA-4759 has a patch available, but would currently only add > interpretation of CIDR notation, no specific ranges, which I think > could easily be added. > > Colin McCabe commented in KAFKA-4985 that so far this was not > implemented as no standard for expressing ip ranges with a fast > implementation had been found so far, the available patch uses the > ipmath [5] package for parsing expressions and range checking - which > seems fairly small and focused. > > This would allow for expressions of the following type: > 10.0.0.1 > 10.0.0.1-10.0.0.10 > 10.0.0.0/24 > > I'd suggest extending this a little to allow a semicolon separated > list of values: > 10.0.0.1;10.0.0.1-10.0.0.10;10.0.0.0/24 > > Performance considerations > Internally the ipmath package represents ip addresses as longs, so if > we stick with the example of a 200 node cluster from above, with the > current implementation that would be 200 string comparisons for every > request, whereas with a range it could potentially come down to two > long comparisons. This is of course a back-of-the-envelope calculation > at best, but there at least seems to be a case for investigating this > a bit further I think. > > > These changes would probably necessitate a KIP - though with some > consideration they could be made in a way that no existing public > facing functionality is changed, but for transparency and proper > documentation I'd say a KIP would be preferable. > > I'd be happy to draft one if people think this is worthwhile. > > Let me know what you think. > > best regards, > Sönke > > [1] https://issues.apache.org/jira/browse/KAFKA-3531 > [2] https://issues.apache.org/jira/browse/KAFKA-4759 > [3] https://issues.apache.org/jira/browse/KAFKA-4985 > [4] https://issues.apache.org/jira/browse/KAFKA-5713 > [5] https://github.com/jgonian/commons-ip-math
