VanditGupta opened a new pull request, #404:
URL: https://github.com/apache/airflow-steward/pull/404
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
## Summary
- Add `--list-tags` to `skill_evals.runner` so users can discover valid
`--tag` values without grepping every `case-meta.json`.
- The flag walks the same case tree as the existing runner, prints each
distinct tag alphabetically with its case count (e.g. `llama 12`), and exits
without running prompts or requiring `--cli`.
- Includes regression tests for tag listing, empty-tag handling, and the
`collect_tag_counts` helper.
## Type of change
- [ ] Skill change (`.claude/skills/<name>/`) — eval fixtures updated below
- [ ] Tool / bridge contract (`tools/<system>/*.md`)
- [x] Python package (`tools/*/` with `pyproject.toml`)
- [ ] Groovy reference impl
- [ ] Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
- [ ] Documentation (`docs/`, `README.md`, `CONTRIBUTING.md`)
- [ ] Project template (`projects/_template/`)
- [ ] CI / dev loop (`prek`, workflows, validators)
- [ ] Other:
## Test plan
- [ ] `prek run --all-files` passes
- [x] For Python packages touched: `uv run pytest` / `ruff check` / `mypy`
passes
- [ ] For Groovy bridges touched: command-line invocation tested end-to-end
- [ ] For skill changes: eval suite passes for the affected skill
(`PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner
tools/skill-evals/evals/<skill>/`)
- [x] For skill *behaviour* changes: a new or updated eval fixture is
included in this PR
(a regression test for the bug fixed / the behaviour added — see
CONTRIBUTING.md)
- [x] Other:
- `uv run --directory tools/skill-evals --group dev pytest
tests/test_runner.py -k "list_tags or collect_tag_counts or tag_filter"` — 4
passed
- Manual smoke test: `python3 -m skill_evals.runner --list-tags
<fixtures-dir>` prints sorted tags with counts
## RFC-AI-0004 compliance
- [ ] **HITL** — any new mutation is gated on explicit user confirmation
- [ ] **Sandbox** — no new unrestricted host access; network reach declared
in the adapter
- [ ] **Vendor neutrality** — placeholders (`<PROJECT>`, `<tracker>`,
`<upstream>`, `<security-list>`) used in all skill / tool prose (the
`check-placeholders` prek hook is the mechanical gate)
- [ ] **Conversational + correctable** — agentic-override path documented if
behaviour is adopter-tunable
- [ ] **Write-access discipline** — no autonomous outbound messages; drafts
only, sent on confirmation
- [ ] **Privacy LLM** — private content does not reach a non-approved LLM;
redactor invoked where needed
## Linked issues
Closes #378
## Notes for reviewers (optional)
- `--list-tags` reuses `find_cases` and `load_case_tags`; existing `--tag`
filtering is unchanged.
- When no tags exist under the path, the runner prints `no tags found` and
exits 0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
