danny0405 opened a new pull request, #20566:
URL: https://github.com/apache/hudi/pull/20566

   ### Describe the issue this Pull Request addresses
   
   Flink schema evolution can silently change integral and string values when 
converting them to DECIMAL because TypeConverters routes them through double. 
For example:
   
   - BIGINT `9007199254740993L` becomes `9007199254740992` in DECIMAL(19, 0).
   - STRING `"9007199254740993.01"` becomes `9007199254740994.00` in 
DECIMAL(22, 2).
   
   Both values fit the target decimal type and should retain their exact value.
   
   ### Summary and Changelog
   
   Preserve decimal precision during Flink schema evolution.
   
   - Use `BigDecimal.valueOf(long)` for Integer and Long inputs.
   - Parse string inputs directly with `new BigDecimal(string)`.
   - Add TestTypeConverters regression coverage for positive and negative 
values above 2^53, long limits, high-precision strings, scale adjustment, 
rounding, floating-point conversions, and overflow.
   - No code was copied from external sources.
   
   ### Impact
   
   Flink readers using these schema-evolution conversions receive exact decimal 
values before the existing target precision and scale rules are applied. Float 
and double conversion paths remain unchanged. No public API, configuration, or 
storage-format changes. No performance benchmark was run.
   
   ### Risk Level
   
   low
   
   The change is limited to integral and string conversion to DECIMAL. Flink's 
existing DecimalData conversion still handles target scale, rounding, and 
overflow.
   
   #### Testing
   
   - Compiled TypeConverters and TestTypeConverters directly against the 
Maven-resolved Flink 1.20 test classpath with JDK 17 and executed the tests 
through the JUnit Platform launcher: all 10 tests passed.
   - Ran the same tests against the original TypeConverters implementation: 8 
failed, including both examples above.
   - `mvn -o -pl hudi-client/hudi-flink-client -Pflink1.20 checkstyle:check`: 
passed with zero violations.
   - `git diff --check`: passed.
   - The normal module test command, `mvn -o -pl hudi-client/hudi-flink-client 
-Pflink1.20 -Dtest=TestTypeConverters -DfailIfNoTests=false test 
-Drat.skip=true`, was blocked during main-source compilation by an unrelated 
cached adapter mismatch: ParquetSchemaConverter references the missing 
DataTypeAdapter.variantParquetAnnotation() method. The full module suite has 
not passed locally.
   
   ### Documentation Update
   
   none — this corrects precision loss in existing supported schema-evolution 
conversions and introduces no feature, configuration, or API requiring 
documentation changes.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to