voonhous commented on issue #16639: URL: https://github.com/apache/hudi/issues/16639#issuecomment-5538250972
### Avro round-trips left in place Accepted for now. Each of these is a `HoodieSchema`-typed API whose body still converts to Avro (or, for `asNullable`, to the field-id `InternalSchema`) and back. They are correct and covered; what is missing is a native implementation. Line numbers are as of #19810 (`c803cdca6c03`). | Helper | Location | Round trip today | Blocked on | | --- | --- | --- | --- | | `HoodieSchemaUtils#projectSchema` | `common/schema/HoodieSchemaUtils.java:788` | `HoodieSchema.fromAvroSchema(HoodieAvroUtils.projectSchema(fileSchema.toAvroSchema(), fields))` | a nested-projection benchmark, so a native rewrite can be shown not to regress the nested path | | `HoodieSchemaUtils#createNewSchemaField`, 5-arg with `HoodieFieldOrder` | `common/schema/HoodieSchemaUtils.java:425` | validated alias of `HoodieSchemaField.of` that routes through `HoodieAvroUtils#createNewSchemaField` | `HoodieSchemaField.of` taking a `HoodieFieldOrder`, then moving the call sites off the alias | | `HoodieAvroUtils#recordNeedsRewriteForExtendedAvroTypePromotion` | `common/avro/HoodieAvroUtils.java:1476` | signature and walk are `(Schema, Schema)`; callers unwrap a `HoodieSchema` to reach it | a logical-type accessor on `HoodieSchema` | | `HoodieSchemaUtils#asNullable` | `common/schema/HoodieSchemaUtils.java:273` | no Avro since #19810, but still `HoodieSchema -> InternalSchema (ColumnUpdateChange) -> HoodieSchema` | a field-wise `HoodieSchema.createNullable` wrap, which is a behavior change: it would keep the non-null defaults, ENUM and union order the `InternalSchema` round trip drops (pinned by `testAsNullablePinsTheInternalSchemaRoundTripLosses`) | Two javadocs point at this issue for exactly this: the `HoodieSchemaUtils` class javadoc and the 5-arg `createNewSchemaField` javadoc both say the delegation is "being retired under #16639". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
