On Thu, Jan 27, 2022 at 10:26 PM Walaa Eldin Moustafa <wa.moust...@gmail.com> wrote:
> *2. Iceberg schema lower casing:* Before Iceberg, when users read Hive > tables from Spark, the returned schema is lowercase since Hive stores all > metadata in lowercase mode. If users move to Iceberg, such readers could > break once Iceberg returns proper case schema. This feature is to add > lowercasing for backward compatibility with existing scripts. This feature > is added as an option and is not enabled by default. > This isn't quite correct. Hive lowercases top-level columns. It does not lowercase field names inside structs. > *3. Hive table proper casing:* conversely, we leverage the Avro schema to > supplement the lower case Hive schema when reading Hive tables. This is > useful if someone wants to still get proper cased schemas while still in > the Hive mode (to be forward-compatible with Iceberg). The same flag used > in (2) is used here. > Are there users of Avro schemas in Hive outside of LinkedIn? I've never seen it used. I don't think you should tie #2 and #3 together. Supporting default values and union types are useful extensions. .. Owen
