Hi Ryan, thanks for the reply. Verifying the component with the schemas you have would be super helpful.
I think enum is actually a type that is generally useful. Although it is not a part of ANSI SQL, MySQL and some other databases have this type. BTW, ENUM_STRING proposed in this FLIP is actually not a type by itself. The ENUM_STRING is just a syntax sugar which actually creates a "new AtomicDataType(new VarCharType(Integer.MAX_VALUE), ENUM_CLASS)". So, we are not really introducing a new type here. However, in order to make the VARCHAR to ENUM conversion work, the ENUM class has to be considered as a ConversionClass of the VARCHAR type, and a StringToEnum converter is required. And yes, AvroSchemaUtils should be annotated as @PublicEvolving. Thanks, Jiangjie (Becket) Qin On Thu, Aug 31, 2023 at 5:22 PM Ryan Skraba <ryan.skr...@aiven.io.invalid> wrote: > Hey -- I have a certain knowledge of Avro, and I'd be willing to help > out with some of these enhancements, writing tests and reviewing. I > have a *lot* of Avro schemas available for validation! > > The FLIP looks pretty good and covers the possible cases pretty > rigorously. I wasn't aware of some of the gaps you've pointed out > here! > > How useful do you think the new ENUM_STRING DataType would be outside > of the Avro use case? It seems like a good enough addition that would > solve the problem here. > > A small note: I assume the AvroSchemaUtils is meant to be annotated > @PublicEvolving as well. > > All my best, Ryan > > > On Tue, Aug 29, 2023 at 4:35 AM Becket Qin <becket....@gmail.com> wrote: > > > > Hi folks, > > > > I would like to start the discussion about FLIP-158[1] which proposes to > > clean up and enhance the Avro support in Flink. More specifically, it > > proposes to: > > > > 1. Make it clear what are the public APIs in flink-avro components. > > 2. Fix a few buggy cases in flink-avro > > 3. Add more supported Avro use cases out of the box. > > > > Feedbacks are welcome! > > > > Thanks > > > > Jiangjie (Becket) Qin > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-358%3A+flink-avro+enhancement+and+cleanup >
