If you follow the best practices, then topics should never have different schemas as you can't enforce schema compatibility. You also have very limited processing capabilities and clumsy workflows attached to it. If you want to encode different kinds of events, then the common approach is to use some kind of envelope schema where different event types are encoded as optional fields.
If you want to stick with your custom approach, then you probably want to implement your own AvroDeserializationSchema that reuses the existing CachedSchemaCoderProvider. If you check the code of RegistryAvroDeserializationSchema, you will notice that the actual implementation is rather slim. @Override public T deserialize(byte[] message) throws IOException { checkAvroInitialized(); getInputStream().setBuffer(message); Schema writerSchema = schemaCoder.readSchema(getInputStream()); GenericDatumReader<T> datumReader = getDatumReader(); datumReader.setSchema(writerSchema); datumReader.setExpected(writerSchema); // <-- the difference return datumReader.read(null, getDecoder()); } On Thu, Nov 12, 2020 at 1:42 PM ashwinkonale <ashwin.kon...@gmail.com> wrote: > Hi Arvid, > Thanks a lot for your reply. And yes, we do use confluent schema registry > extensively. But the `ConfluentRegistryAvroDeserializationSchema` expects > reader schema to be provided. That means it reads the message using writer > schema and converts to reader schema. But this is not what I want always. > If > I have messages of different schema in the same topic, I cannot apply > `ConfluentRegistryAvroDeserializationSchema` correct ? I also came across > this question > < > https://stackoverflow.com/questions/58849635/is-it-possible-to-deserialize-avro-messageconsuming-message-from-kafka-without> > > . I am also doing the same thing in my pipeline by providing custom > deserialiser using confluentSchemaRegistryClient. So as far as I > understood, > in this usecase there is no way to tell flink about > `GenericRecordAvroTypeInfo` of the genericRecord which comes out of source > function. Please tell me if my understanding is correct. > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng