Davis-Zhang-Onehouse opened a new pull request, #20701: URL: https://github.com/apache/hudi/pull/20701
### Describe the issue this Pull Request addresses `HoodieSchemaUtils.generateProjectionSchema(HoodieSchema, List<String>)` builds the projected record with a bare `HoodieSchema.createRecord(name, namespace, doc, fields)`, so any schema-level (record-level) properties on the source schema are lost in the projection. The other helpers in `HoodieSchemaUtils` that rebuild a record schema keep those properties: `removeFields` copies `getObjectProps()`, `pruneDataSchema` calls `copyProperties`, and `addMetadataFields`, `appendFieldsToSchema` and `mergeSchemas` go through `createNewSchemaFromFieldsWithReference`. `generateProjectionSchema` is the one helper that does not. Its callers include the bootstrap record-key schema, the Hive file group reader and Avro Parquet reader schemas, and the MERGE INTO partial-update schema. ### Summary and Changelog A projection now keeps the source schema's record-level properties, as the other record-rebuilding helpers in `HoodieSchemaUtils` already do. - `HoodieSchemaUtils.generateProjectionSchema` builds its result with `createNewSchemaFromFieldsWithReference(originalSchema, projectedFields)`, which keeps the name, namespace, doc and schema-level properties of the source schema. Field matching and field copying are unchanged. - Adds `TestHoodieSchemaUtils#testGenerateProjectionSchemaPreservesSchemaLevelProps`. The test gives the source schema a namespace, a doc and a record-level property. It then asserts that the projection keeps all three, and that the property also survives a `toString`/`parse` round trip. Before this change it fails with `expected: <value> but was: <null>`, and after it the test passes. - Not changed here: the Hive realtime reader has its own `HoodieRealtimeRecordReaderUtils.generateProjectionSchema`, and `HoodieSchemaUtils.projectSchema` (nested-path projection) also rebuilds records without properties. Both can follow separately. ### Impact No public API change. A projected schema now carries the record-level properties of the schema it was projected from. This change leaves field handling as it was. For MERGE INTO partial updates on MOR tables, the projected schema is serialized as the partial-update schema written with the update. Any record-level properties of the table schema are therefore now kept there as well. ### Risk Level low. The change affects only record-level properties of the projected schema, and only for source schemas that carry them. The `hudi-common` schema-handler tests and the `hudi-hadoop-mr` reader tests that build projection schemas still pass. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
