namanjain24-sudo opened a new pull request, #75623: URL: https://github.com/apache/airflow/pull/75623
`dev/skill-evals/cases/newsfragment.yaml` has not changed since the harness landed in #69308, so it still encodes the six-type newsfragment policy. #74589 then narrowed `airflow-core` to `significant` only: > In `airflow-core/` use only `significant` newsfragments, and only for a change users must read before > upgrading: a behavior change, a removal or a new default, or migration steps. [...] Never add one for an > ordinary bug fix, feature, improvement or doc change That change updated `AGENTS.md`, `contributing-docs/18_contribution_workflow.rst` and the PR-drafting skill, but not these cases. Two of them describe additive features — a new endpoint plus a new `[connection_test]` config section, and a new coordinator layer plus a config option — and assert `should_create === true`. Neither is a behavior change, a removal, a new default or a migration, so an agent following the current guidance answers `false` and the case fails it for being right. The harness that exists to catch guidance drift had drifted from the guidance. Flipping both would leave all six cases expecting "no", which an agent that always declines passes perfectly, so the suite would stop detecting guidance that is too restrictive. Two cases that do warrant a `significant` note keep the other direction covered, both taken from newsfragments already on `main` rather than invented: the asset-triggered run default (#68517) and the async Postgres driver default (#68496). That keeps the balance the file had before — six "no" to two "yes". I may have the policy call wrong on the two flipped cases; if maintainers read "a new config section plus a new endpoint" as something users must read before upgrading, say so and I will restore them and adjust the reasoning instead. ### The eval hash `compute_guidance_hash` covers `cases/*.yaml`, so this trips `check-eval-hash` until the eval is re-run and `last-eval-hash.txt` is updated. I do not have Claude auth available to run it, so this follows the README's documented fallback and #75599's approach: the hook is skipped, the hash is left stale, and the PR stays a draft until a maintainer can run the eval and push the updated hash to the branch. Everything else in `prek run --files dev/skill-evals/cases/newsfragment.yaml` passes, and the file parses as valid YAML with eight cases. related: #62500 ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
