kjs1095 opened a new pull request, #12090:
URL: https://github.com/apache/ozone/pull/12090

   ## What changes were proposed in this pull request?
   
   suggested by @adoroszlai in #12054
   
   ### Context from JIRA description
   
   Python helpers in `dev-support/ci` have unit tests, but CI runs them through 
`.bats` wrappers.
   
   This introduces some problems
   - Domain leak
     - Every new Python helper needs its own `.bats` file, and a contributor 
has to know that
   - Swallowed test errors
     - When a Python test fails, bats run captures the output, so the job 
summary shows only `not ok 1 coverage_summary.py unit tests`. The failing test 
name and the assertion diff are not visible anywhere in CI
   
   What value it brings
   - Reduce cognitive load
     - A new Python helper only needs its `test_*.py` file
     - The bats check is back to testing shell scripts only, which is what its 
name says
   - Make test errors transparent to reduce debug effort
     - A failing Python test shows the test name and assertion diff
   
   ### Changes
   1. Introduce new `hadoop-ozone/dev-support/checks/pyunit.sh`
     a. Contains `#checks:basic`
     b. Runs `python3 -m unittest discover -s dev-support/ci -p 'test_*.py'`
   2. Add `check_needs_pyunit` to `selective_ci_checks.sh` and test in 
`selective_ci_checks.bats`
   3. Remove `junit_summary.bats` and `coverage_summary.bats`
   4. Run `pyunit` on ubuntu-slim like `author` and `bats`
   
   ### Design decisions 
   - Set `ERROR_PATTERN` to empty as in `bats.sh` and `author.sh`
     - This variable is fallback pattern (`_post_process.sh`) when the content 
of `REPORT_FILE` is empty, and it's ideal to put related knowledge in the same 
place
       - How we ran a Python unittest and,
       - How we produced report from unittest results
     - `_post_process.sh` only catches up to 25 lines
   - Trigger on Python under `dev-support/ci` only 
   - Commit id for `selective_ci_checks.bats` test case
     - The commit change scope is simple (just `xml_to_md.py`)
   - `ci.yml` basic job wiring for `pyunit`. It's raised by AI review
     - I checked the issue HDDS-17602, and it is reasonable to execute `pyunit` 
on a lightweight setup. And it seems fair to include in this PR. But if 
reviewers think this item is out of scope, I can revert the commit and handle 
it in a separate PR.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-17840
   
   ## How was this patch tested?
   
   (Please explain how this patch was tested. Ex: unit tests, manual tests, 
workflow run on the fork git repo.)
   
   - Local ran `pyunit.sh`
     - Passed all tests, `failure` = 0
   
   ```bash
   [ozone-root] $ ./hadoop-ozone/dev-support/checks/pyunit.sh
   
   test_invalid_report (test_coverage_summary.TestMain) ... ok
   test_missing_report (test_coverage_summary.TestMain) ... ok
   test_prints_summary (test_coverage_summary.TestMain) ... ok
   test_combined_uses_sonar_formula (test_coverage_summary.TestParseReport) ... 
ok
   test_overall_counters (test_coverage_summary.TestParseReport) ... ok
   test_packages_counters (test_coverage_summary.TestParseReport) ... ok
   test_percent (test_coverage_summary.TestPercent) ... ok
   test_no_packages (test_coverage_summary.TestRenderSummary) ... ok
   test_overall_section (test_coverage_summary.TestRenderSummary) ... ok
   test_package_table_sorted_by_missed 
(test_coverage_summary.TestRenderSummary) ... ok
   test_report_url (test_coverage_summary.TestRenderSummary) ... ok
   test_find_reports_dedupes_both_trees (test_junit_summary.TestMain) ... ok
   test_find_reports_skips_iteration_dirs (test_junit_summary.TestMain) ... ok
   test_main_bad_args_still_exit_zero (test_junit_summary.TestMain) ... ok
   test_main_no_reports_is_silent_noop (test_junit_summary.TestMain) ... ok
   test_main_prints_summary (test_junit_summary.TestMain) ... ok
   test_main_survives_malformed_xml (test_junit_summary.TestMain) ... ok
   test_main_survives_non_parse_errors (test_junit_summary.TestMain) ... ok
   test_classification (test_junit_summary.TestParseReport) ... ok
   test_message_is_collapsed_raw_text (test_junit_summary.TestParseReport) ... 
ok
   test_message_junit5_tail_edge_cases (test_junit_summary.TestParseReport) ... 
ok
   test_message_keeps_junit5_tail_when_custom_message_floods 
(test_junit_summary.TestParseReport) ... ok
   test_module_name_from_path (test_junit_summary.TestParseReport) ... ok
   test_format_time (test_junit_summary.TestRender) ... ok
   test_render_counts_list (test_junit_summary.TestRender) ... ok
   test_render_counts_omit_zero (test_junit_summary.TestRender) ... ok
   test_render_escapes_every_cell (test_junit_summary.TestRender) ... ok
   test_render_report_url (test_junit_summary.TestRender) ... ok
   test_render_tables (test_junit_summary.TestRender) ... ok
   
   ----------------------------------------------------------------------
   Ran 29 tests in 0.027s
   
   OK
   ```
     - Verified with injected failure and exception. Output separate lines in 
`summary.txt` and `failures` = 2
   
   - Local run `bats.sh`
     - no bats wrapper for `test_junit_summary.py` and 
`test_coverage_summary.py`
   
   ```bash
   [ozone-root] $ ./hadoop-ozone/dev-support/checks/bats.sh
   
   # new `python helper change` is introduced
   
   1..41
   ...
   ok 41 python helper change
   ...
   
   ```
   
   - Fork CI ran `bats`
   
   | Before | After |
   | --- | --- |
   | [CI job before this 
PR](https://github.com/kjs1095/ozone/actions/runs/43954482760) | [CI job after 
this PR](https://github.com/kjs1095/ozone/actions/runs/44396299404) |
   | bats wrappers for 
<ul><li>test_junit_summary.py</li><li>test_coverage_summary.py</li></ul> | New 
`pyunit` step to run test automatically |
   | <img width="420" height="200" alt="HDDS-17840-before-ci" 
src="https://github.com/user-attachments/assets/e369f5ae-9ddf-4f1a-b046-b3daeb34330d";
 /> | <img width="420" height="200" alt="HDDS-17840-after-ci-pyunit" 
src="https://github.com/user-attachments/assets/ecba27da-bef7-4dbc-8b2c-28afaed684f4";
 /> |
   
   - Fork CI ran, `ubuntu-slim` for `pyunit`
   
   <img width="740" height="389" alt="HDDS-17840-pyunit-ubuntu-slim" 
src="https://github.com/user-attachments/assets/3f4a5b96-75cf-4e8c-a96b-8846b4dfa9eb";
 />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to