Re: [Issue] Repeatedly receiving same message from Kafka

2023-08-16 Thread Dennis Jung
Hello, Thank you. I think it is a problem caused by Kafka configuration, not Flink. I'll take a look and let you know if there's an issue in Flink. BR, Jung 2023년 8월 15일 (화) 오후 9:40, Hector Rios 님이 작성: > Hi there > > It would be helpful if you could include the code for your pipeline. One >

Re: [Issue] Repeatedly receiving same message from Kafka

2023-08-15 Thread Hector Rios
Hi there It would be helpful if you could include the code for your pipeline. One suggestion, can you disable the "EXACTLY_ONCE" semantics on the producer. Using EXACTLY_ONCE will leverage Kafka transactions and thus add overhead. I would disable it to see if you still get the same situation.

[Issue] Repeatedly receiving same message from Kafka

2023-08-15 Thread Dennis Jung
Sorry, I've forgot putting title, so sending again. 2023년 8월 15일 (화) 오후 6:27, Dennis Jung 님이 작성: > (this is issue from Flink 1.14) > > Hello, > > I've set up following logic to consume messages from kafka, and produce > them to another kafka broker. For producer, I've configured >