Hi Evan, I haven't seen this before. Can you share your Kafka write configuration, and any other stack traces that could be relevant?
John On Tue, Sep 13, 2022 at 10:23 AM Evan Galpin <egal...@apache.org> wrote: > Hi all, > > I've recently started using the KafkaIO connector as a sink, and am new to > Kafka in general. My kafka clusters are hosted by Confluent Cloud. I'm > using Beam SDK 2.41.0. At least daily, the producers in my Beam pipeline > are getting stuck in a loop frantically logging this message: > > Node n disconnected. > > Resetting the last seen epoch of partition <topic-partition> to x since > the associated topicId changed from null to <topic_id> > > Updating the running pipeline "resolves" the issue I believe as a result > of recreating the Kafka producer clients, but it seems that as-is the > KafkaIO producer clients are not resilient to node disconnects. Might I be > missing a configuration option, or are there any known issues like this? > > Thanks, > Evan >
