[
https://issues.apache.org/jira/browse/SPARK-60834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng updated SPARK-60834:
----------------------------------
Description:
Reduce Apache Spark's GitHub Actions consumption by applying branch-4.x's
existing one-in-ten deterministic commit-count sampling (PR #57841) to master.
For pushes to apache/spark master, run the expensive build_main.yml reusable
build only when git rev-list --count HEAD is divisible by ten. A lightweight
job records the commit count and sampling decision. For ordinary single-commit
pushes this skips nine out of ten post-merge builds.
Preserve builds for fork PR branches, the existing skip for fork master syncs,
and the behavior of other upstream branches. Scheduled workflows remain
unchanged.
The retained build continues to use the existing affected-module selection for
the current push. It does not run a full suite or accumulate changes from
skipped pushes, so coverage for omitted changes can be delayed until scheduled
testing exercises them.
The proportional estimate for the measured calendar week is 233,048
runner-minutes against ASF's 250,000 limit; the projected highest rolling
five-day total is 180,643 against 216,000. These estimates exclude the added
gate cost and require monitoring as activity changes. Workflow concurrency
above 20 jobs remains a separate policy gap.
Implementation and detailed analysis: https://github.com/apache/spark/pull/60045
was:
Reduce Apache Spark's GitHub Actions consumption by running master post-merge
CI once every five commits. This adapts the deterministic commit-count gate
introduced for branch-4.x in PR #57841, where the frequency is once every ten
commits.
For pushes to apache/spark master, run the expensive build_main.yml reusable
build only when git rev-list --count HEAD is divisible by five. A lightweight
job records the commit count and sampling decision. For ordinary single-commit
pushes this skips four out of five post-merge builds.
Preserve builds for fork PR branches, the existing skip for fork master syncs,
and the behavior of other upstream branches. Scheduled workflows remain
unchanged.
The retained build continues to use the existing affected-module selection for
the current push. It does not run a full suite or accumulate changes from
skipped pushes, so coverage for omitted changes can be delayed until scheduled
testing exercises them.
The proportional estimate for the measured calendar week is 242,509
runner-minutes against ASF's 250,000 limit; the projected highest rolling
five-day total is 187,480 against 216,000. These estimates exclude the added
gate cost and require monitoring as activity changes. Workflow concurrency
above 20 jobs remains a separate policy gap.
Implementation and detailed analysis: https://github.com/apache/spark/pull/60045
Summary: Run master post-merge builds every ten commits (was: Run
master post-merge builds every five commits)
> Run master post-merge builds every ten commits
> ----------------------------------------------
>
> Key: SPARK-60834
> URL: https://issues.apache.org/jira/browse/SPARK-60834
> Project: Spark
> Issue Type: Improvement
> Components: Project Infra
> Affects Versions: 5.0.0
> Reporter: Ruifeng Zheng
> Priority: Major
> Labels: pull-request-available
>
> Reduce Apache Spark's GitHub Actions consumption by applying branch-4.x's
> existing one-in-ten deterministic commit-count sampling (PR #57841) to master.
> For pushes to apache/spark master, run the expensive build_main.yml reusable
> build only when git rev-list --count HEAD is divisible by ten. A lightweight
> job records the commit count and sampling decision. For ordinary
> single-commit pushes this skips nine out of ten post-merge builds.
> Preserve builds for fork PR branches, the existing skip for fork master
> syncs, and the behavior of other upstream branches. Scheduled workflows
> remain unchanged.
> The retained build continues to use the existing affected-module selection
> for the current push. It does not run a full suite or accumulate changes from
> skipped pushes, so coverage for omitted changes can be delayed until
> scheduled testing exercises them.
> The proportional estimate for the measured calendar week is 233,048
> runner-minutes against ASF's 250,000 limit; the projected highest rolling
> five-day total is 180,643 against 216,000. These estimates exclude the added
> gate cost and require monitoring as activity changes. Workflow concurrency
> above 20 jobs remains a separate policy gap.
> Implementation and detailed analysis:
> https://github.com/apache/spark/pull/60045
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
