alamb commented on issue #25088:
URL: https://github.com/apache/datafusion/issues/25088#issuecomment-6297640524
This has been happening again a lot recently (taking hours to start running)
I have been exploring what apache projects are consuming jobs and Claude
came up with this one
```shell
gh api --paginate
"repos/apache/airflow/actions/runs?status=in_progress&per_page=100" \
--jq '.workflow_runs[] | select(.head_repository.full_name ==
"apache/airflow") | "\(.id) \(.head_sha) \(.html_url)"' \
| while read -r id sha runurl; do
link=$(gh api "repos/apache/airflow/commits/$sha/pulls" --jq
'.[0].html_url // empty' 2>/dev/null)
gh api --paginate
"repos/apache/airflow/actions/runs/$id/jobs?per_page=100" \
--jq '.jobs[] | select(.status=="in_progress") |
select((.labels|join(","))|test("self-hosted")|not) |
"\(.labels|join(","))\t\(.started_at)\t\(.workflow_name) / \(.name)"' \
| sed "s|^|${link:-$runurl}\t|"
done | sort -t$'\t' -k1,1 -k3,3
```
The current output is
```shell
done | sort -t$'\t' -k1,1 -k3,3
https://github.com/apache/airflow/pull/74789 ubuntu-22.04
2026-10-20T17:32:54Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:API...CLI
https://github.com/apache/airflow/pull/74789 ubuntu-22.04
2026-10-20T17:32:54Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74789 ubuntu-22.04
2026-10-20T17:33:09Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74789 ubuntu-22.04
2026-10-20T17:33:09Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74789 ubuntu-22.04
2026-10-20T17:33:10Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74789 ubuntu-22.04
2026-10-20T17:33:42Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:32:51Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:API...CLI
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:32:57Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:32:58Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:32:58Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:06Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:06Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:06Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:API...CLI
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:06Z Tests (AMD) / Non-DB tests: providers /
Non-DB-prov::3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:07Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:27Z Tests (AMD) / Provider distributions tests / Compat
2.11.1:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:27Z Tests (AMD) / Provider distributions tests / Compat
3.0.6:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:27Z Tests (AMD) / Provider distributions tests / Compat
3.2.2:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:27Z Tests (AMD) / Provider distributions tests / Compat
3.3.2:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:33:42Z Tests (AMD) / Provider distributions tests / Compat
3.1.8:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74790 ubuntu-22.04
2026-10-20T17:47:29Z Tests (AMD) / Build PROD images / Build PROD Regular
image 3.11
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:12Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:12Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:18Z Tests (AMD) / CI image checks / Static checks
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:21Z Tests (AMD) / Provider distributions tests / Compat
3.2.2:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:39Z Tests (AMD) / Provider distributions tests / Compat
3.0.6:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:39Z Tests (AMD) / Provider distributions tests / Compat
3.3.2:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:47Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:48Z Tests (AMD) / Provider distributions tests / Compat
2.11.1:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:48Z Tests (AMD) / Provider distributions tests / Compat
3.1.8:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:50Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:33:53Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:API...CLI
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:34:00Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:34:00Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:34:03Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:API...CLI
https://github.com/apache/airflow/pull/74791 ubuntu-22.04
2026-10-20T17:48:54Z Tests (AMD) / Build PROD images / Build PROD Regular
image 3.11
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:16Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:26Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:26Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:API...CLI
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:26Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:32Z Tests (AMD) / Provider distributions tests / Compat
3.2.2:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:39Z Tests (AMD) / Provider distributions tests / Compat
3.0.6:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:39Z Tests (AMD) / Provider distributions tests / Compat
3.1.8:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:39Z Tests (AMD) / Provider distributions tests / Providers
wheel tests
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:40Z Tests (AMD) / Provider distributions tests / Compat
3.3.2:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:42:59Z Tests (AMD) / CI image checks / Static checks
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:13Z Tests (AMD) / Provider distributions tests / Compat
2.11.1:P3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:16Z Tests (AMD) / Postgres tests: providers /
DB-prov:Postgres:14:3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:17Z Tests (AMD) / CI image checks / Build documentation
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:19Z Tests (AMD) / MySQL tests: providers /
DB-prov:MySQL:8.0:3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:19Z Tests (AMD) / Sqlite tests: providers /
DB-prov:Sqlite:3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:26Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:API...CLI
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:32Z Tests (AMD) / Low dep tests: providers /
prov:LowestDeps:14:3.11:samba...zendesk
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:32Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:35Z Tests (AMD) / Low dep tests: providers /
prov:LowestDeps:14:3.11:apprise...duckdb
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:39Z Tests (AMD) / Generate constraints / Generate
constraints for 3.11 on linux/amd64
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:52Z Tests (AMD) / Low dep tests: providers /
prov:LowestDeps:14:3.11:airbyte...apache.tinker
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:43:52Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:14Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:31Z Tests (AMD) / Low dep tests: providers /
prov:LowestDeps:14:3.11:microsoft.psr...salesforce
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:43Z Tests (AMD) / Low dep tests: providers /
prov:LowestDeps:14:3.11:edge3...microsoft.mss
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:43Z Tests (AMD) / Special tests / Latest SQLAlchemy test:
core / DB-core:LatestSQLAlchemy-Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:43Z Tests (AMD) / Special tests / Min SQLAlchemy test:
providers / DB-prov:MinSQLAlchemy-Postgres:14:3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:44Z Tests (AMD) / Special tests / Latest SQLAlchemy test:
core / DB-core:LatestSQLAlchemy-Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:50Z Tests (AMD) / Special tests / Min SQLAlchemy test: core
/ DB-core:MinSQLAlchemy-Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:53Z Tests (AMD) / Special tests / Latest Boto test:
providers / prov:LatestBoto-Postgres:14:3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:44:53Z Tests (AMD) / Special tests / Pendulum2 test: core /
core:Pendulum2-Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:45:02Z Tests (AMD) / Special tests / Latest Boto test: core /
core:LatestBoto-Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:45:02Z Tests (AMD) / Special tests / Latest Boto test: core /
core:LatestBoto-Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:45:02Z Tests (AMD) / Special tests / Pendulum2 test: core /
core:Pendulum2-Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:45:18Z Tests (AMD) / Special tests / Latest SQLAlchemy test:
providers / DB-prov:LatestSQLAlchemy-Postgres:14:3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:45:18Z Tests (AMD) / Special tests / Min SQLAlchemy test: core
/ DB-core:MinSQLAlchemy-Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:45:18Z Tests (AMD) / Special tests / Pendulum2 test: providers
/ prov:Pendulum2-Postgres:14:3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:45:18Z Tests (AMD) / Special tests / Quarantined test:
providers / Qrnt-prov:Postgres:14:3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74793 ubuntu-22.04
2026-10-20T17:45:23Z Tests (AMD) / Non-DB tests: providers /
Non-DB-prov::3.11:-amazon,celer...standard
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:41:10Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:API...CLI
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:41:10Z Tests (AMD) / MySQL tests: core /
DB-core:MySQL:8.0:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:41:12Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:41:27Z Tests (AMD) / Low dep tests:core /
core:LowestDeps:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:41:37Z Tests (AMD) / Provider distributions tests / Providers
wheel tests
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:42:22Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:Core...Serialization
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:42:23Z Tests (AMD) / Postgres tests: core /
DB-core:Postgres:14:3.11:API...CLI
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:42:23Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:API...CLI
https://github.com/apache/airflow/pull/74794 ubuntu-22.04
2026-10-20T17:42:23Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:Core...Serialization
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T16:59:23Z Tests (AMD) / Sqlite tests: core /
DB-core:Sqlite:3.11:Core...Serialization
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T17:41:30Z Tests (AMD) / Kubernetes tests / K8S
System:LocalExecutor-3.12-v1.32.11-true
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T17:41:40Z Tests (AMD) / Kubernetes tests / K8S
System:LocalExecutor-3.14-v1.34.11-true
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T17:41:43Z Tests (AMD) / Kubernetes tests / K8S
System:LocalExecutor-3.12-v1.32.11-false
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T17:41:48Z Tests (AMD) / Kubernetes tests / K8S
System:LocalExecutor-3.11-v1.31.12-false
https://github.com/apache/airflow/pull/75231 ubuntu-22.04-arm
2026-10-20T17:42:52Z Tests (ARM) / Kubernetes tests / K8S
System:LocalExecutor-3.13-v1.37.0-false
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T17:43:54Z Tests (AMD) / Kubernetes tests / K8S
System:LocalExecutor-3.14-v1.34.11-false
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T17:44:14Z Tests (AMD) / Kubernetes tests / K8S
System:LocalExecutor-3.11-v1.35.8-false
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T17:50:59Z Tests (AMD) / Kubernetes tests / K8S
System:LocalExecutor-3.12-v1.36.4-false
https://github.com/apache/airflow/pull/75231 ubuntu-22.04
2026-10-20T17:50:59Z Tests (AMD) / Kubernetes tests / K8S
System:LocalExecutor-3.13-v1.37.0-false
```
It looks like a few PRs consumed a lot of the runner time (each launches 14
jobs via some matrix)
- https://github.com/apache/airflow/pull/75231
- https://github.com/apache/airflow/pull/74793
- https://github.com/apache/airflow/pull/75231
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
