voonhous opened a new pull request, #18661:
URL: https://github.com/apache/hudi/pull/18661

   ### Describe the issue this Pull Request addresses
   
   Hudi today supports two parallel schema-evolution paths, schema-on-read 
(gated by `hoodie.schema.on.read.enable=true`, backed by `InternalSchema` under 
`.hoodie/.schema/`) and the default schema-on-write path that relies on 
Spark/Avro to reconcile schemas implicitly. 
   
   The dual paths are confusing for users (the default silently fails to track 
renames and drops) and force every write entry point in the code to branch on 
`schemaOnReadEnabled`. With the broader push to break Hudi's hard dependency on 
Avro (RFC-46, RFC-87, RFC-99), the schema-evolution layer is the natural next 
consolidation.
   
   This PR proposes RFC-104 to unify on schema-on-read.
   
   ### Summary and Changelog
   
   Adds RFC-104, "Unify schema evolution on schema-on-read." No production code 
changes.
   
   - `rfc/rfc-104/rfc-104.md`: new RFC. Covers the staged migration: close 
engine gaps (Spark, Flink writer via RFC-87, Hive MR reader, Trino plugin), 
resolve metadata-table interactions, lazy-bootstrap an `InternalSchema` for 
legacy tables on first write under the new default, then delete the 
schema-on-write branches in 2.0. Lists four open questions for community 
discussion (Trino projection ownership, HFile schema-evolution decision, 
post-2.0 lifetime of `hoodie.schema.on.read.enable`, deprecation-warning 
timing).
   
   ### Impact
   
   None at this stage: RFC only, no behavior change. The eventual 
implementation will be a 2.0 breaking change (schema-on-write path removed, 
default flipped). 
   
   The RFC explicitly notes that pre-bootstrap rename/drop history is 
unrecoverable for legacy tables, since schema-on-write never tracked it.
   
   ### Risk Level
   
   None
   
   ### Documentation Update
   
   None, the RFC itself is the documentation. User-facing docs and the Hudi 
website will be updated when the implementation lands.
   
   ### Contributor's checklist
   
   - [X] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [X] Enough context is provided in the sections above
   - [X] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to