meel-ayush opened a new pull request, #76955:
URL: https://github.com/apache/airflow/pull/76955
### Description
Part of **AIP-120**:
When a commit on a personal fork touches environment or workflow files,
`full_tests_needed` triggers the entire 145-job CI matrix. On GitHub-hosted
public runners (2 vCPUs, 7 GB RAM), this frequently fails due to 503 artifact
egress errors, JVM out-of-memory crashes when spinning up Kafka, Cassandra, and
Trino simultaneously, and Kind cluster startup timeouts.
Furthermore, two edge cases cause unnecessary runs or late failures on forks:
1. GitHub's compare API (`/repos/{fork}/airflow/compare/{base}...{sha}`)
caps file lists at 300. Large doc or typing refactors hit this limit and are
marked truncated, triggering the full test matrix.
2. Fork pushes with merge conflicts against upstream run through the full
test suite before anyone notices the branch cannot be merged.
This change:
* Caps heavy combinatorial matrices on fork pushes (Kind K8s clusters,
Kustomize overlays, 24 lowest-dependency provider slices, multi-container JVM
integrations, and PROD image builds) so they only run when matching files or
explicit labels (`area:kubernetes`, `area:e2e-tests`) require them.
* Keeps core unit and DB tests running across Postgres, MySQL, and SQLite,
as well as Task SDK tests and Helm unit tests.
* Adds a local git diff fallback in `fork_compare.py` when GitHub's compare
API hits the 300-file cap or fails, using the merge base SHA returned by the
API to resolve uncapped file changes without deep fetches.
* Adds an early conflict check in `ci-amd.yml` and `ci-arm.yml` using `git
merge-tree --write-tree` against the upstream target branch
(`branch_defaults.py`), failing fast with an actionable rebase message if
conflicts exist.
* Merges CLI/workflow labels with API-fetched labels in `ci_commands.py`
rather than overwriting them with an empty tuple when no PR exists yet.
### Tests
- Unit tests added in `dev/breeze/tests/test_fork_compare.py`,
`dev/breeze/tests/test_ci_commands.py`, and
`dev/breeze/tests/test_selective_checks.py` covering local git fallback with
merge base reuse, conflict detection, label merging, and fork push test
selection.
---
##### Was generative AI tooling used to co-author this PR?
- [ ] Yes (please specify the tool below)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
