yihua opened a new pull request, #19871: URL: https://github.com/apache/hudi/pull/19871
### Describe the issue this Pull Request addresses closes #19870 `AvroSchemaEvolutionUtils.reconcileSchema` dereferences the incoming schema before the Avro NULL check its own comment promises, and `reconcileSchemaRequirements` hands a null source schema back untouched whenever the table schema has no fields. A HoodieStreamer row source whose schema provider reports no target schema, writing to a table whose latest commit carries a zero-column record schema, therefore fails every empty batch with a `NullPointerException` and never advances its checkpoint. ### Summary and Changelog An absent incoming schema now falls back to the table schema everywhere the writer schema is deduced, so an empty batch lands an empty commit and the checkpoint keeps moving. - `AvroSchemaEvolutionUtils.reconcileSchema`: treat a Java `null` like Avro NULL. - `AvroSchemaEvolutionUtils.reconcileSchemaRequirements`: check the source schema before the empty-target early return, so a null source is never returned. - `HoodieSchemaUtils.deduceWriterSchemaWithReconcile`: tolerate an absent incoming schema on the schema-on-read branch as well, including the legacy-reconciliation error message. - Tests: `TestAvroSchemaEvolutionUtils` (both utilities), `TestHoodieSchemaUtils` (full deduction, reconcile on and off), and `TestHoodieIncrSourceE2E` (a HoodieStreamer run over the S3 incremental source with a schema provider returning null and a zero-column table schema: the empty batch commits with the advanced checkpoint, an unchanged checkpoint commits only with `allowCommitOnNoCheckpointChange`, and the non-empty batch that follows evolves the zero-column schema and writes its rows). The utility and deduction tests fail with the reported NPE without the fix. The streamer test pins the end-to-end behavior; on this line `StreamSync.getDeducedSchemaProvider` already maps a null provider schema to a NULL schema before deduction, so it passes either way, and the NPE is reachable only by callers that hand `deduceWriterSchema` or `reconcileSchema` a null directly. ### Impact None to public APIs. An empty batch against a table with no usable schema resolves to the table schema instead of throwing. The reconcileSchemaRequirements reorder also flips the result when both schemas are degenerate: a fieldless or NULL source against a fieldless or NULL target now yields the target where it previously yielded the source. ### Risk Level low ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
