Hello, I am trying to understand the effect of schema registry on our pipeline's performance. In order to do sowe created a very simple pipeline that reads from kafka, runs a simple transformation of adding new field and writes of kafka. the messages are in avro format
I ran this pipeline with 3 different options on same configuration : 1 kafka partition, 1 task manager, 1 slot, 1 parallelism: * when i used apicurio as the schema registry i was able to process only 2000 messages per second * when i used confluent schema registry i was able to process 7000 messages per second * when I did not use any schema registry and used plain avro deserializer/serializer i was able to process *30K* messages per second. I understand that using a schema registry may cause a reduction in performance but in my opinion the difference is too high. Any comments or suggestions about these results? Thanks in advance Sigalit
