justinmclean opened a new pull request, #493: URL: https://github.com/apache/airflow-steward/pull/493
## Summary ## What Adds a new SOFT advisory lint to `skill-and-tool-validator` that flags ASF-coupled tokens in skill bodies, tokens a non-ASF adopter cannot satisfy without editing the skill (e.g. `svn` commands, `[email protected]`, Vulnogram URLs, bare PMC/ICLA/incubator mentions). ## Why Skills are meant to be project-agnostic, but ASF-specific assumptions leak into prose over time. This surfaces them so maintainers can generalise the coupling (or consciously keep the ASF default) before it becomes a hidden adoption blocker. ## How `validate_asf_coupling()` scans skill bodies against a tiered pattern list: - **high** confidence: almost never legitimate outside an ASF workflow (svn release commands, hardcoded announce list, dist/dev paths). - **low** confidence: common ASF prose that may be intentional (bare PMC, ICLA, incubator). Each hit is tagged with a confidence level and a remedy class (placeholder / adapter / capability-flag) so maintainers know how to generalise it. The rule is advisory only: it surfaces on stderr and never fails the run. It reuses the existing allowlist-path and inline-allow-marker machinery, plus extra markers for lines that already name a generalisation mechanism (`release-dist-backend`, `<announce-list>`, `ASF default:`, etc.). ## Testing New `TestValidateAsfCoupling` class, 17 cases covering high/low patterns, remedy-class reporting, allowlisted paths, and inline-allow-marker suppression. All pass: uv run --directory tools/skill-and-tool-validator --with pytest \ pytest tests/test_validator.py::TestValidateAsfCoupling -v Ran against the repo's 42 SKILL.md files as a sanity check: 77 advisory hits (1 high, 76 low), all behaving as designed. The single high hit is a hardcoded `[email protected]` in `security-issue-sync` (left as a follow-up, not in scope here). ## Type of change - [ ] Skill change (`.claude/skills/<name>/`) — eval fixtures updated below - [ ] Tool / bridge contract (`tools/<system>/*.md`) - [ ] Python package (`tools/*/` with `pyproject.toml`) - [ ] Groovy reference impl - [ ] Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM) - [ ] Documentation (`docs/`, `README.md`, `CONTRIBUTING.md`) - [ ] Project template (`projects/_template/`) - [X] CI / dev loop (`prek`, workflows, validators) - [ ] Other: ## Test plan - [X] `prek run --all-files` passes - [X] For Python packages touched: `uv run pytest` / `ruff check` / `mypy` passes - [ ] For Groovy bridges touched: command-line invocation tested end-to-end - [ ] For skill changes: eval suite passes for the affected skill (`PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/`) - [ ] For skill *behaviour* changes: a new or updated eval fixture is included in this PR (a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md) - [ ] Other: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
