nanjeshramesh opened a new pull request, #20556: URL: https://github.com/apache/hudi/pull/20556
### Describe the issue this Pull Request addresses Closes #20504 A `MERGE INTO` whose source has a `tinyint` or `smallint` column fails in `ExpressionPayload` with `ClassCastException: MutableInt cannot be cast to MutableByte` when the matched action is `UPDATE SET *`, even though the target column is `int`. ### Summary and Changelog The conditions and assignments of a `MERGE INTO` are evaluated against a row that `ExpressionPayload` deserializes from Avro. Hudi's schema has no 8 or 16 bit integer type, so `tinyint` and `smallint` columns arrive in that row as `int`. `MergeIntoHoodieTableCommand` bound the expressions with the column's Spark type instead, so the generated code called `getByte` or `getShort` on an int slot. `bindReferences` now binds references of type `ByteType` or `ShortType` as `IntegerType` and casts them back to the original type, so the rest of the expression keeps its types. The change is made in both the Spark 3 and Spark 4 copies of `MergeIntoHoodieTableCommand`. Added `Test MergeInto with tinyint and smallint source columns assigned to int columns` to `TestMergeIntoTable`. It uses `tinyint` and `smallint` source columns in a matched condition, a matched `UPDATE SET *` and a not matched `INSERT *`. It fails on master with the exception from the issue and passes with this change. The rest of `TestMergeIntoTable` and `TestMergeIntoTable2` still passes locally on Spark 3.5. ### Impact `MERGE INTO` statements with `tinyint` or `smallint` source columns no longer fail. Other types are bound as before. ### Risk Level low. Only references typed `ByteType` or `ShortType` are rewritten, and the cast back to the original type keeps the expression types unchanged. The Spark 4 copy of the file has the same change and was not compiled locally. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
