danny0405 opened a new pull request, #20570:
URL: https://github.com/apache/hudi/pull/20570

   ### Describe the issue this Pull Request addresses
   
   Flink COW mini-batches rewrite a file group and delete the preceding 
intermediate base file. `WriteStatMerger` retains the latest file path but sums 
every intermediate file's `fileSizeInBytes`. For example, a 100 MB intermediate 
file followed by a 110 MB replacement produces a merged status reporting 210 MB 
for the surviving 110 MB file. This inflated size propagates into files-index 
metadata and can distort size-based planning.
   
   ### Summary and Changelog
   
   - Retain the latest `fileSizeInBytes` for COW replacement files using a 
single conditional assignment.
   - Preserve existing `HoodieDeltaWriteStat` size aggregation and cumulative 
`totalWriteBytes` accounting.
   - Add regression coverage for growing, shrinking, and zero-size 
replacements, repeated merges, and unchanged delta aggregation.
   - Extend `TestWriteStatusMerger` to check the surviving file's size.
   
   The existing event merger appends new statuses after buffered statuses and 
reduces each file group sequentially, so the second stat supplies both the 
latest path and its size. No code was copied from external sources.
   
   ### Impact
   
   Corrects reported surviving base-file sizes for Flink COW file groups 
written through multiple mini-batches. No public API, configuration, 
storage-format, or compatibility changes. Existing delta-write behavior is 
preserved; this change does not repair metadata from earlier commits.
   
   ### Risk Level
   
   low
   
   The change is confined to file-size selection during write-stat merging. 
Incorrect ordering could select a stale size, but the existing caller already 
guarantees newer statuses are the second operand and uses that operand for the 
retained path.
   
   Validation with JDK 17:
   
   ```bash
   mvn -o -pl hudi-client/hudi-flink-client 
-Dtest=TestWriteStatMerger,TestWriteStatusMerger test
   ```
   
   All 9 focused tests passed, with zero failures or errors. Module Checkstyle 
and Apache RAT license checks passed. The full Flink integration suite was not 
run.
   
   ### Documentation Update
   
   none — this restores expected file-size accounting without adding features 
or changing configuration.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to