potiuk opened a new pull request, #418: URL: https://github.com/apache/airflow-steward/pull/418
## Summary The bulk-mode pre-flight classifier (`security-issue-sync` → `bulk-mode.md`) has a rule table that evolves as we learn how real adopter trackers behave. Each rule change needs a before / after measurement to know whether the change helped (more skips) or hurt (false-positive skips). The recent classifier-tuning PR (#416) was driven by a one-off `/tmp/` script — easy to lose, hard to reproduce, not CI-friendly. `tools/preflight-audit/` promotes that pattern into a permanent tool. ``` preflight-audit classify --repo <r> --issues 1,2,3 [--now ISO] preflight-audit classify --load resp.json --now ISO [--json] ``` - **Live mode** shells out to `gh api graphql` (same aliased multi-field query the skill builds inline). - **Replay mode** reads a pre-fetched response — deterministic, network-free, suitable for CI eval fixtures. - `--bot-logins` extends bot-equivalent detection for adopters with personal-account bots. - Default output is a grouped table; `--json` for machine-readable. ## How the rules stay in sync The classifier in `src/preflight_audit/classifier.py` is the **executable spec** of the rule table in `bulk-mode.md`. Both must be edited in lock-step — a PR that changes one should change the other. 37 unit tests cover each rule with a focused positive + negative case, the skill-marker detection (including parametrized edge cases), and `classify_response` end-to-end. ## Intended workflow when changing a rule 1. Run `preflight-audit` against your tracker → capture **before**. 2. Edit the rule table in `bulk-mode.md` AND the matching condition in `classifier.py`. 3. Re-run → capture **after**. 4. Cite both numbers in the PR body. ## Test plan - [x] 37 unit tests — green - [x] `ruff check` / `ruff format` / `mypy` — green - [x] `prek` (markdownlint, typos, format, validators) — green - [x] `lychee` on the new README — clean - [x] `skill-and-tool-validate` — no new violations - [ ] CI lychee + tests-ok on this PR 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
