Hi, We are trying to evaluate the "Schema evolution" with Apache Pulsar and AVRO. We tried the below use-case -
Created a "TestMessage" with id,string1 attributes. Published/consumed 100k messages on a topic. This essentially uploaded first schema version for the topic. We now produced another 100k messages for the topic. It is not consumed yet. Now, we changed "TestMessage" and added another attribute "string2". Produced another 100k messages on same topic. If we now start consumer on the topic, we get the below error - org.apache.pulsar.client.api.SchemaSerializationException: java.io.EOFException at org.apache.pulsar.client.impl.schema.AvroSchema.decode( AvroSchema.java:115) at org.apache.pulsar.client.impl.MessageImpl.getValue(MessageImpl.java:247) at com.x.pulsarclient.demo.PulsarProducer$PulsarConsumer.run( PulsarProducer.java:94) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: java.io.EOFException at org.apache.pulsar.shade.org.apache.avro.io.BinaryDecoder.ensureBounds( BinaryDecoder.java:473) On some search on open issues, came across *https://issues.apache.org/jira/browse/AVRO-1440 <https://issues.apache.org/jira/browse/AVRO-1440>* which mentions that reader must have access to writers schema. Given, we wanted to leverage the "Reflection" way of runtime AVRO schema generation and usage, how can we do this? Both producer and consumer are on same JVM. Even, if they were on different, the topic will have messages generated by two different schemas - how will consumer consume both old and new messages? Can anyone shed some light on this topic? What am I missing? Any help is much appreciated. Thanks, Prakash
