shivaam opened a new pull request, #64516: URL: https://github.com/apache/airflow/pull/64516
Add a deterministic check to `breeze pr auto-triage` that flags `area:UI` PRs from non-collaborator contributors when their PR description lacks screenshots or demo videos. So owners, members, and committers / collaborators are excluded from this check. The rationale: having one PR merged doesn't mean they know the project's conventions around UI demos. There are some PR - #62885 that do not need a UI mock but it is not simple to exclude them. ## What this does When triaging PRs, the tool now checks if a PR: 1. Has the `area:UI` label 2. Is authored by a non-collaborator (not COLLABORATOR/MEMBER/OWNER) 3. Lacks demo evidence in the PR body (screenshots, videos, or image links) If all three conditions are met, the PR is flagged with a `[WARNING] Missing UI demo` violation. The suggested action is **comment** (not draft conversion) — this is a soft violation handled by a new branch in `_compute_default_action`. ## How it works - **`assess_pr_ui_demo()`** in `utils/github.py` — new deterministic check function following the same pattern as `assess_pr_checks`, `assess_pr_conflicts`, and `assess_pr_unresolved_comments` - **Detection patterns** cover: GitHub drag-and-drop `<img>` tags, `user-attachments/assets/` URLs (images & videos), markdown `` syntax, and direct media file URLs (`.png`, `.jpg`, `.gif`, `.mp4`, `.mov`, `.webm`, etc.) - **Integrated into `_assess_pr_deterministic()`** so the check runs everywhere: main triage flow, TUI, pagination batches, and single-PR mode - **`_compute_default_action`** soft-violation branch ensures UI-demo-only flags get `COMMENT` instead of falling through to `DRAFT` ## Dry-run validation against all open `area:UI` PRs Ran `breeze pr auto-triage --label "area:UI" --llm-use api --dry-run --answer-triage s --authors all`: | Metric | Count | |--------|-------| | Total `area:UI` PRs assessed | 35 | | Flagged for Missing UI demo | 32 | | Flagged for other issues only (CI, conflicts, comments) | 3 | | Passed all checks (had video demo) | 1 | | False positives | **0** | | False negatives | **0** | ### Breakdown of flagged PRs | Category | Count | Action | |----------|-------|--------| | UI demo only (no other issues) | 7 | `comment` | | UI demo + CI failures | 14 | `draft` | | UI demo + merge conflicts | 5 | `draft` | | UI demo + unresolved comments | 6 | `comment` | ### True negatives (correctly NOT flagged) - PR #60718 — has video demo via `user-attachments/assets/` URL → passed all checks - PR #62256, #62882, #64271 — have `<img>` tags in body → not flagged ### True positives (correctly flagged) - PR #64010 — Feat: Dag Runs UI now has multi-select bulk actions - PR #64179 — Fix Gantt view still visible when time range is outside dagrun window - PR #63733 — Add (UI feature, contributor: henrywoo) - PR #64267 — UI: (code change, contributor: vamsikrishna) ### False negative (marked as Area:UI, but dont need a UI mock) - PR #62885 — Complete Portuguese (pt) UI translation to 90%+ coverage ## Known limitations - **Grace period interaction:** When CI failures are within the grace window and are the only hard issue, the grace-period skip logic does not check for UI demo violations. A PR with grace-period CI + missing UI demo could be silently skipped. This is a pre-existing structural pattern in the grace period logic, not introduced by this change. ## Tests Tests covering: - `_has_demo_evidence` helper (empty/None body, text-only, all media patterns, false positive resistance) - `assess_pr_ui_demo` (every exit path: no label, collaborator associations, demo present/absent) - `_compute_default_action` soft-violation branch (comment for soft-only, draft when combined with hard issues) --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Opus 4.6 Generated-by: Claude Opus 4.6 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
