Viktor Somogyi-Vass created KAFKA-9814:
------------------------------------------
Summary: Allow AdminClient.listTopics to list based on expression
Key: KAFKA-9814
URL: https://issues.apache.org/jira/browse/KAFKA-9814
Project: Kafka
Issue Type: Improvement
Affects Versions: 2.5.0
Reporter: Viktor Somogyi-Vass
Assignee: Viktor Somogyi-Vass
Currently listTopics would list all the topics in the cluster which often isn't
desired due to the large number of topics and because the authorizer would
check access for every topic which could result in various bloated access/audit
logs that would make them hard to read and also in case of many topics it isn't
really effective to return all of them due to the response's size.
To take an example let's have a user who issues a naive command such as:
{noformat}
kafka-topics --list --topic topic6 --bootstrap-server mycompany.com:9092
--command-config client.properties
{noformat}
In this case the admin client would issue a metadata request for all the
topics. When this request gets to the broker then it will try to check
permissions to every topic in the cluster and then return those which are
allowed for "describe" to the client. Then the command when it gets the answer
will simply filter out {{topic6}} from the response.
Looking at this problem we can see that if we allow the broker to do the
filtering then we can:
* do less permission checks
* return less data to the clients
Regarding the solution I see two path but both of them of course subject
community discussion:
# modify the MetadataRequest and put some kind of regex in it
# create a new list API that would contain this filter pattern
Perhaps the list API would make more sense as the MetadataRequest is currently
used because it just fits but there is no real reason to extend it with this as
the functionality won't be used elsewhere.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
