Hi Divij, As this issue manifests after the Kafka controller restart, is there a way to detect this corrupted Kafka cluster before restarting the controller?
Regards, Atul ________________________________ From: Atul Kumar (atkumar3) <atul.ku...@appdynamics.com> Sent: 16 December 2022 18:24 To: atul.ku...@appdynamics.com.invalid <atul.ku...@appdynamics.com.INVALID>; users@kafka.apache.org <users@kafka.apache.org> Subject: Re: [SUSPECTED SPAM] Re: Critical bug in Kafka 2.8.1 | topic Id in memory: <> does not match the topic Id for partition <> provided in the request: <>. Hi Divij, Thanks for the reply! We were using the Kafka-manager tool (with an older Kafka client) which was the root cause of this issue. Regards, Atul ________________________________ From: Divij Vaidya <divijvaidy...@gmail.com> Sent: 08 December 2022 17:55 To: atul.ku...@appdynamics.com.invalid <atul.ku...@appdynamics.com.invalid> Cc: users@kafka.apache.org <users@kafka.apache.org> Subject: [SUSPECTED SPAM] Re: Critical bug in Kafka 2.8.1 | topic Id in memory: <> does not match the topic Id for partition <> provided in the request: <>. Hi Atul There is a known bug which has similar symptoms that you observed. See: https://issues.apache.org/jira/browse/KAFKA-14190 The bug manifests when you use a client < 2.8.0 with `--zookeeper` flag. It could be avoided by either upgrading the client or by using `--bootstrap-server` instead of `--zookeeper`. Does the above explanation match your Kafka setup? -- Divij Vaidya On Thu, Dec 8, 2022 at 1:16 PM Atul Kumar (atkumar3) <atul.ku...@appdynamics.com.invalid> wrote: > Hello everyone, > > We are using Kafka 2.8.1, and recently while rolling restart of one of our > clusters, we encountered the following issue on the Kafka cluster > > topic Id in memory: <> does not match the topic Id for partition <> > provided in the request: <>. > > All the produce requests to the impacted topic were failing. We were able > to fix the issue by deleting partition.metadata from each broker for the > impacted topic and doing a rolling restart of the Kafka cluster. We are > using the Kafka stream application. > > The same issue was reported in Jira: > https://issues.apache.org/jira/browse/KAFKA-12835 and is marked fixed > with version 2.8.1 but we can still see this issue. > > Any help would be highly appreciated on any fix for this issue. > > Thanks, > Atul Kumar > > > > > > > > > > > > > > > > > > > > > > Regards, > Atul >
