[
https://issues.apache.org/jira/browse/FLINK-40718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Purushottam Sinha updated FLINK-40718:
--------------------------------------
Component/s: Connectors / Kafka
(was: Connectors / MongoDB)
Description:
This was already done for the Flink main repo and for
flink-connector-opensearch (FLINK-40711 / PR #77). ASF Infra has flagged the
whole Flink project as a top-5 consumer of the shared GitHub Actions runner
pool foundation-wide and requested usage be brought down.
{{flink-connector-kafka}}'s {{weekly.yml}} runs 5 matrix entries x 2 jobs
({{compile_and_test}} + {{python_test}}, up to 10 jobs/week) across {{main}}
plus 4 release branches ({{v5.0}}x2, {{v4.0}}, {{v3.4}}). Only one entry
({{2.2-SNAPSHOT}}/{{main}}) is a snapshot version -- the other 4 are
fixed/released Flink versions on release branches that rarely get commits,
rerun every week regardless. The file already has a comment noting release
lines run a single JDK "to stay under the ASF limit of 20 concurrent jobs",
showing this repo is already resource-constrained.
Port the same {{weekly-trigger.yml}} pattern: split both {{compile_and_test}}
and {{python_test}} into a snapshot job (always runs) and a fixed-version job
(only runs on new commits to a watched branch or a non-green previous run).
was:
This was already done for the Flink main repo, and for
flink-connector-opensearch (FLINK-40711 / PR #77). We need the same for
flink-connector-mongodb: Apache Infra has complained about connector CI
resource usage, and this connector's weekly.yml re-runs its full
fixed-Flink-version matrix every week even when nothing changed and the last
run was green.
Port the same {{weekly-trigger.yml}} pattern: split {{weekly.yml}} into a
snapshot job (always runs, since snapshot Flink versions change upstream
regardless of connector commits) and a fixed-version job (only runs on new
commits to a watched branch or a non-green previous run).
> Do not run weekly if there is no new commits since the last green run
> ---------------------------------------------------------------------
>
> Key: FLINK-40718
> URL: https://issues.apache.org/jira/browse/FLINK-40718
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
> Reporter: Purushottam Sinha
> Priority: Major
>
> This was already done for the Flink main repo and for
> flink-connector-opensearch (FLINK-40711 / PR #77). ASF Infra has flagged the
> whole Flink project as a top-5 consumer of the shared GitHub Actions runner
> pool foundation-wide and requested usage be brought down.
> {{flink-connector-kafka}}'s {{weekly.yml}} runs 5 matrix entries x 2 jobs
> ({{compile_and_test}} + {{python_test}}, up to 10 jobs/week) across {{main}}
> plus 4 release branches ({{v5.0}}x2, {{v4.0}}, {{v3.4}}). Only one entry
> ({{2.2-SNAPSHOT}}/{{main}}) is a snapshot version -- the other 4 are
> fixed/released Flink versions on release branches that rarely get commits,
> rerun every week regardless. The file already has a comment noting release
> lines run a single JDK "to stay under the ASF limit of 20 concurrent jobs",
> showing this repo is already resource-constrained.
> Port the same {{weekly-trigger.yml}} pattern: split both {{compile_and_test}}
> and {{python_test}} into a snapshot job (always runs) and a fixed-version job
> (only runs on new commits to a watched branch or a non-green previous run).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
