Hi Umesh, Sorry for the late reply. I find that the pull #2070 was not active for a long time. I will try to continue to work on it. Maybe we can support it in the next release (maybe within a month).
Best, Zelin Yu On Wed, Jan 10, 2024 at 1:21 AM umesh dangat <[email protected]> wrote: > Hello, > > I am a software engineer at Yelp Inc and lead the data infrastructure > group at Yelp. We have a complex real time streaming ecosystem comprising > flink, kafka and our custom schema registry service. I am trying to > evaluate Apache Paimon as a potential replacement for a lot of our data > pipelines, involving streaming reads, joins and aggregations to help > minimize our growing operational complexity and cost. Also paimon seems to > solve the schema evolution problem better than flink sqlclient? (which we > use currently) > > One issue with integrating paimon in our ecosystem seems to be that it does > not support debezium avro format. Although Jingsong Li pointed me to this > <https://github.com/apache/incubator-paimon/pull/2070> branch that does > seem to add support for debezium avro format using confluent schema > registry. This would allow us to ingest our data from kafka into paimon and > then evaluate it. > > I wanted to know if we have plans to push this branch to master soonish. I > can help with reviewing, since I plan to consume data written using this > format for some of our production workflows. > > Thanks, > Umesh >
