Re: Kafka Protocol : Compact Array or Array ?
Thanks Luke. That helps. Where can I find information about schemas of all requests ? Regards, Neeraj Sent from my iPhone > On 2 Oct 2023, at 6:03 pm, Luke Chen wrote: > > Hi Neeraj, > > Yes, for MetadataRequest, version 0 ~ 8, the topic is ARRAY type. After > version 9, it'll be COMPACT_ARRAY. > It's because of this definition: "flexibleVersions": "9+". > You can check KIP-482 for more information: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-482 > > Thanks. > Luke > >> On Mon, Oct 2, 2023 at 5:09 AM Neeraj Vaidya >> wrote: >> >> Hi All, >> I've raised this on StackOverflow as well : >> https://stackoverflow.com/questions/77208870/kafka-binary-protocol-array-or-compact-array >> In case someone can help in answering that question. >> >> Regards, >> Neeraj >> On Sunday, 1 October, 2023 at 11:32:49 am GMT+11, Neeraj Vaidya < >> neeraj.vai...@yahoo.co.in> wrote: >> >> Hi All, >> There are 2 types of arrays specified in the Kafka protocol documentation >> : ARRAY and COMPACT_ARRAY. >> But in the protocol details for the different messages, it does not >> explicitly specify if the array type is which one of the above. >> >> For example, the BNF grammar for the section for MetadataRequest API is as >> below : >> >> Metadata Request (Version: 0) => [topics] >> topics => name >>name => STRING >> >> What is the type of [topics] ? Is it ARRAY or COMPACT_ARRAY ? >> >> After playing around with the protocol using some tests, I think for >> Version:0 of this API request, the broker expects this to be of type ARRAY. >> >> But for higher versions, say v9, COMPACT_ARRAY is expected. >> >> I think the protocol really needs to be explicit and is lacking in this >> respect. >> >> Regards, >> Neeraj
Re: Kafka Protocol : Compact Array or Array ?
Hi Neeraj, Yes, for MetadataRequest, version 0 ~ 8, the topic is ARRAY type. After version 9, it'll be COMPACT_ARRAY. It's because of this definition: "flexibleVersions": "9+". You can check KIP-482 for more information: https://cwiki.apache.org/confluence/display/KAFKA/KIP-482 Thanks. Luke On Mon, Oct 2, 2023 at 5:09 AM Neeraj Vaidya wrote: > Hi All, > I've raised this on StackOverflow as well : > https://stackoverflow.com/questions/77208870/kafka-binary-protocol-array-or-compact-array > In case someone can help in answering that question. > > Regards, > Neeraj > On Sunday, 1 October, 2023 at 11:32:49 am GMT+11, Neeraj Vaidya < > neeraj.vai...@yahoo.co.in> wrote: > > Hi All, > There are 2 types of arrays specified in the Kafka protocol documentation > : ARRAY and COMPACT_ARRAY. > But in the protocol details for the different messages, it does not > explicitly specify if the array type is which one of the above. > > For example, the BNF grammar for the section for MetadataRequest API is as > below : > > Metadata Request (Version: 0) => [topics] > topics => name > name => STRING > > What is the type of [topics] ? Is it ARRAY or COMPACT_ARRAY ? > > After playing around with the protocol using some tests, I think for > Version:0 of this API request, the broker expects this to be of type ARRAY. > > But for higher versions, say v9, COMPACT_ARRAY is expected. > > I think the protocol really needs to be explicit and is lacking in this > respect. > > Regards, > Neeraj
Re: Kafka Protocol : Compact Array or Array ?
Hi All, I've raised this on StackOverflow as well : https://stackoverflow.com/questions/77208870/kafka-binary-protocol-array-or-compact-array In case someone can help in answering that question. Regards, Neeraj On Sunday, 1 October, 2023 at 11:32:49 am GMT+11, Neeraj Vaidya wrote: Hi All, There are 2 types of arrays specified in the Kafka protocol documentation : ARRAY and COMPACT_ARRAY. But in the protocol details for the different messages, it does not explicitly specify if the array type is which one of the above. For example, the BNF grammar for the section for MetadataRequest API is as below : Metadata Request (Version: 0) => [topics] topics => name name => STRING What is the type of [topics] ? Is it ARRAY or COMPACT_ARRAY ? After playing around with the protocol using some tests, I think for Version:0 of this API request, the broker expects this to be of type ARRAY. But for higher versions, say v9, COMPACT_ARRAY is expected. I think the protocol really needs to be explicit and is lacking in this respect. Regards, Neeraj
Kafka Protocol : Compact Array or Array ?
Hi All, There are 2 types of arrays specified in the Kafka protocol documentation : ARRAY and COMPACT_ARRAY. But in the protocol details for the different messages, it does not explicitly specify if the array type is which one of the above. For example, the BNF grammar for the section for MetadataRequest API is as below : Metadata Request (Version: 0) => [topics] topics => name name => STRING What is the type of [topics] ? Is it ARRAY or COMPACT_ARRAY ? After playing around with the protocol using some tests, I think for Version:0 of this API request, the broker expects this to be of type ARRAY. But for higher versions, say v9, COMPACT_ARRAY is expected. I think the protocol really needs to be explicit and is lacking in this respect. Regards, Neeraj
