Re: [DISCUSS] Flink Avro Cloudera Registry (FLINK-14577)

Thanks Dawid, Gyula we will create a pull request then with the proposed changes were we can further elaborate on the dependencies. Regards, Matyas On Thu, Nov 14, 2019 at 11:10 AM Dawid Wysakowicz wrote: > Hi Gyula, > > First of all sorry for the delayed response. > > I see the argument for

Re: [DISCUSS] Flink Avro Cloudera Registry (FLINK-14577)

Hi Gyula, First of all sorry for the delayed response. I see the argument for handling metadata from kafka headers. I haven't noticed the schema you are proposing is actually KafkaDeserializationSchema, which means it works only with Kafka. I still believe it would be really beneficial for the

Re: [DISCUSS] Flink Avro Cloudera Registry (FLINK-14577)

Hi Dawid, In general I agree if we can provide a completely unified way of handling this registries that would be great but I wonder if that makes sense in the long term. While the cloudera schema registry only supports Avro at the moment, it aims to support other formats in the future, and

Re: [DISCUSS] Flink Avro Cloudera Registry (FLINK-14577)

Hi Gyula, I did not want to discourage this contribution. I do agree we should treat this connector equally to the confluent's schema registry. I just wanted to express my uncertainty about general approach to new connectors contributions. By no means I wanted to discourage this contribution. As

Re: [DISCUSS] Flink Avro Cloudera Registry (FLINK-14577)

Thanks Matyas for starting the discussion! I think this would be a very valuable addition to Flink as many companies are already using the Hortonworks/Cloudera registry and it would enable them to connect to Flink easily. @Dawid: Regarding the implementation this a much more lightweight connector

Re: [DISCUSS] Flink Avro Cloudera Registry (FLINK-14577)

Hi Matyas, I think this would be a valuable addition. You may reuse some of the already available abstractions for writing avro deserialization schema based on a schema registry (have a look at RegistryDeserializationSchema and SchemaCoderProvider). There is also an opened PR for adding a similar

[DISCUSS] Flink Avro Cloudera Registry (FLINK-14577)

Dear Flink Community! We have noticed a recent request for Hortonworks schema registry support ( FLINK-14577 ). We have an implementation for it already, and we would be happy to contribute it to Apache Flink. You can find the documentation