kbuci commented on PR #18539: URL: https://github.com/apache/hudi/pull/18539#issuecomment-4410375746
> hi, @kbuci I have made a PR to add a adapter class for variant type access: #18702, so that we can get rid of those Java reflections, it majorly covers the schema converters, I actually copied some logic from your PR, hope it helps. @danny0405 Thanks for sharing! I guess then it would be better if I wait for that to land and rebase this PR off of that right? @cshuo @danny0405 After taking a step back, I think we should first get consensus on how we want Flink in HUDI to always correctly infer the correct Flink type to use for Variant and Vector and Blob. Can we discuss on https://github.com/apache/hudi/issues/18711 and I can work on and land initial PR(s) that does the needed "wiring" so that when we start tackling blob and vector we can use a similar approach as variant. Since although with Variant we are a bit "lucky" in the sense that parquet has an official variant annotation, it would be good to have a common solution and agree on what kind of "backwards compatibility" we want (like HUDI 1.3 Flink being able to read data written by Spark 4.0 HUDI 1.2 builds). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
