Hello, Check if your topic replication factor is not below min.isr setting of Kafka. I had the same problem and that was it for me.
Frank Op za 9 apr. 2022 04:01 schreef Praneeth Ramesh <sr.prane...@gmail.com>: > Hi All > > I have a job which reads from kafka and applies some transactions and > writes the data back to kafka topics. > When I use the exactly once semantics I see that the kafka producer is not > initialized and the operator task is hanging in INITIALIZING state and > fails eventually. > > It works fine when I change the mode to AT_LEAST_ONCE. I see that in this > mode the transactionId is null in the producer config and > enable.idempotence is false. > > But when I enable EXACTLY_ONCE I see that the producer configs having > transactionId which I set and enable.idempotence to true. From trace logs > what I can see is the transaction manager is initializing and > reinitializing the transactionId again and again and does not make any > progress. > > I tried to analyze the thread dump and see that there are threads blocked > at > at > org.apache.kafka.clients.producer.internals.TransactionalRequestResult.await(TransactionalRequestResult.java:68) > at > org.apache.kafka.clients.producer.KafkaProducer.initTransactions(KafkaProducer.java:632) > at > org.apache.flink.connector.kafka.sink.FlinkKafkaInternalProducer.initTransactionId(FlinkKafkaInternalProducer.java:156) > > > I made sure that there are no ACLs problems with brokers. Any idea what > could be the problem? > > Attaching the TM logs where the application is reinitializing the > transactions. > > > Thanks for any help or pointers. > > > > > -- > Regards > Praneeth Ramesh >
