Hello Maxim, Thanks for sharing this.
I had a look and it seems like the behaviour on the wire changed with KAFKA-9435. I believe this change [0] in ReplicaManager causes all topics in online log dirs to be a part of the response inadvertently. We do however set partition information only for the queried topic. I'd suggest creating a JIRA for this. As for an option to specify ALL_PARTITIONS, I reckon that would require a KIP since it would be a change to the public interface. [0]: https://github.com/apache/kafka/pull/7972/files#diff-78812e247ffeae6f8c49b1b22506434701b1e1bafe7f92ef8f8708059e292bf0R674 Regards, Gaurav On Mon, May 20, 2024 at 10:35 PM Maxim Senin <mse...@cogility.com.invalid> wrote: > Hello. > > I’m having a problem with Kafka protocol API. > > Requests: > DescribeLogDirs Request (Version: 0) => [topics] > topics => topic [partitions] > topic => STRING > partitions => INT32 > > My request contains `[{topic: “blah”, partitions: > [0,1,2,3,4,5,6,7,8,9]}]`, but the result > > Responses: > DescribeLogDirs Response (Version: 0) => throttle_time_ms [results] > throttle_time_ms => INT32 > results => error_code log_dir [topics] > error_code => INT16 > log_dir => STRING > topics => name [partitions] > name => STRING > partitions => partition_index partition_size offset_lag is_future_key > partition_index => INT32 > partition_size => INT64 > offset_lag => INT64 > is_future_key => BOOLEAN > > > > contains entries for *all* topics. My workaround had been to filter the > returned list by topic name to find the one I was requesting the data for, > but I don’t understand why it’s not limiting the results to just the topic > I requested in the first place. > > Also, I think there should be an option to just specify ALL_PARTITIONS > because that would save me from having to retrieve topic metadata from the > broker to count the number of partitions. Kafka server would probably have > means to do that more efficiently. > > Is this a bug or am I doing something wrong? > > Thanks, > Maxim > > ________________________________ > > COGILITY SOFTWARE CORPORATION LEGAL DISCLAIMER: The information in this > email is confidential and is intended solely for the addressee. Access to > this email by anyone else is unauthorized. If you are not the intended > recipient, any disclosure, copying, distribution or any action taken or > omitted to be taken in reliance on it, is prohibited and may be unlawful. >
