danny0405 opened a new pull request, #20676:
URL: https://github.com/apache/hudi/pull/20676

   ### Describe the issue this Pull Request addresses
   
   Issue: #20408; parent #20401. Existing keyed SDP acceptance covers COW only 
and leaves MOR log merging unverified. This completes that test subset without 
closing the broader issue.
   
   ### Summary and Changelog
   
   - Parameterize the real CLI/Connect keyed-flow scenario with --table-type 
cow|mor, retaining cow as the default.
   - Run the existing ordering, late update/delete, deletion, reinsertion and 
invalid-configuration matrix against both new and pre-existing MOR targets.
   - Disable inline/streaming compaction. After an update, require log files 
and verify read-optimized base rows differ from the exact snapshot, proving log 
merging is exercised.
   - Compare completed commit filenames before and after the unchanged 
checkpoint restart for both table types.
   - Document how to run each matrix. No external code was copied.
   
   ### Impact
   
   Test and developer-documentation changes only; no runtime, API, 
configuration default or storage-format changes. The new command-line option 
selects the test target type.
   
   ### Risk Level
   
   low. Both COW and MOR real Spark Connect 4.1.1 CLI matrices passed under 
python -O with JDK 17 and the current-master Spark 4.1 bundle. Each checks 
exact rows after six input stages, unchanged restart and five rejected 
configurations. Both MOR targets demonstrated old base rows and updated 
snapshot rows with log files present. Python compilation and git diff --check 
passed. Metadata indexing remains disabled; this is delete input, not upstream 
hard-delete CDC or full-refresh acceptance.
   
   ### Documentation Update
   
   Updated hudi-integ-test/spark-pipelines/KEYED_FLOWS.md with the MOR command, 
compaction settings, log-merge evidence and unchanged-restart assertion. No 
website change is needed for test infrastructure.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to