GitHub user danny0405 edited a comment on the discussion: RFC-99: Hudi Type System
@rahil-c , These engines will evolve their data types too, like the new introduced `Varient` type in Spark, the evolving may be slow especially for SQL types(the discussion for standard takes quite a long time), let's define the types with clear semantics(logicla type) and it's physical data structure probably, we can always migrate to new mappings upon the introducing of new engine types. @the-other-tim-brown not sure if we really needs to introduce the notion, Hudi just needs the logical type IMO because each engine or file format would after all has their own type systems too, and they should decide what to store with proper phisical presentation. GitHub link: https://github.com/apache/hudi/discussions/14253#discussioncomment-14965124 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
