prathik-onehouse opened a new pull request, #20571:
URL: https://github.com/apache/hudi/pull/20571

   ### Describe the issue this Pull Request addresses
   
   closes #19634
   
   `show_logfile_records` and `show_logfile_metadata` crash on common MOR log 
shapes: a table with only deltacommits (`merge => true`), a delete or command 
block, a key updated in more than one log block (`merge => true`), and a log 
file with no data block.
   
   ### Summary and Changelog
   
   Both procedures now handle these shapes instead of failing.
   
   - The merged scan is bounded by the latest completed instant of any write 
action, so a deltacommit-only table works. Log blocks written after the last 
compaction are now included too; the previous commit-and-replace bound skipped 
them (the CLI twin still uses that bound).
   - Merged records are converted with `toIndexedRecord` instead of being cast 
to a payload, keys the scanner treats as deleted are not shown, and rows are 
rendered with `GenericData` so a lazily decoded record prints its values rather 
than its raw bytes.
   - Non-data blocks are skipped by `show_logfile_records` and listed with a 
zero record count by `show_logfile_metadata`, as the CLI does.
   - The merged scan reads its schema from the last globbed log file that has a 
data block, and the unmerged scan accepts files without one (the HUDI-6694 
approach from the CLI).
   
   `TestHoodieLogFileProcedure` replaces the tests that pinned the crashes with 
tests of the fixed behavior, and adds cases for a key updated twice on an 
uncompacted table, a last log file without a data block, and log blocks written 
after a compaction. Six of its eight tests fail without the fix.
   
   ### Impact
   
   The two procedures return records or metadata for these log shapes instead 
of throwing. With `merge => true`, log blocks written after the last compaction 
are now part of the merged view.
   
   ### Risk Level
   
   low. Only the two diagnostic procedures change.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [PR] fix(spark): stop show_logfi... prathik-onehouse (via GitHub) via commits

Reply via email to