Re: Flink Kafka EXACTLY_ONCE causing KafkaException ByteArraySerializer is not an instance of Serializer

2020-08-02 Thread Qingsheng Ren
Hi Vikash, Sorry for the late reply. Is your version of Flink kafka *connector* 1.10.1 too? Actually it's a bug in the connector, so I think you need to upgrade the connector to 1.10.1 too, not just Flink itself. I tried Flink 1.10.0/1.10.1 + flink-kafka-connector 1.10.0 and indeed reproduced

Re: Flink Kafka EXACTLY_ONCE causing KafkaException ByteArraySerializer is not an instance of Serializer

2020-07-31 Thread Vikash Dat
Thanks for the reply. I am currently using 1.10 but also saw it happens in 1.10.1 when experimenting. I have not tried 1.11 since EMR only has up to 1.10 at the moment. Are there any known work arounds? On Fri, Jul 31, 2020 at 02:42 Qingsheng Ren wrote: > Hi Vikash, > > It's a bug about

Re: Flink Kafka EXACTLY_ONCE causing KafkaException ByteArraySerializer is not an instance of Serializer

2020-07-31 Thread Qingsheng Ren
Hi Vikash, It's a bug about classloader used in `abortTransaction()` method in `FlinkKafkaProducer`, Flink version 1.10.0. I think it has been fixed in 1.10.1 and 1.11 according to FLINK-16262. Are you using Flink version 1.10.0? Vikash Dat 于2020年7月30日周四 下午9:26写道: > Has anyone had success

Re: Flink Kafka EXACTLY_ONCE causing KafkaException ByteArraySerializer is not an instance of Serializer

2020-07-30 Thread Vikash Dat
Has anyone had success with using exactly_once in a kafka producer in flink? As of right now I don't think the code shown in the docs (https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-producer) actually works. -- Sent from:

Flink Kafka EXACTLY_ONCE causing KafkaException ByteArraySerializer is not an instance of Serializer

2020-07-29 Thread Vikash Dat
I'm using Flink 1.10 and Kafka (AWS MSK) 2.2 and trying to do a simple app that consumes from one kafka topic and produces events into another topic. I would like to utilize the exactly_once semantic, however, I am experiencing the following error: org.apache.kafka.common.KafkaException: