This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4c6884291e8b [SPARK-48149][INFRA] Serialize `build_python.yml` to run
a single Python version per cron schedule
4c6884291e8b is described below
commit 4c6884291e8b97a7d64dd13530f7ecabe2839d16
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Mon May 6 16:06:57 2024 -0700
[SPARK-48149][INFRA] Serialize `build_python.yml` to run a single Python
version per cron schedule
### What changes were proposed in this pull request?
This PR aims to serialize `build_python.yml` to run a single Python version
per cron schedule to reduce the maximum concurrency per single GitHub Action
job.
### Why are the changes needed?
Currently, `build_python.yml` triggers 60 jobs. `30` of `60` jobs are
running concurrently because 10 test pipelines are required per Python version.
- https://github.com/apache/spark/actions/workflows/build_python.yml
<img width="731" alt="Screenshot 2024-05-06 at 14 28 08"
src="https://github.com/apache/spark/assets/9700541/e4f4e9d2-2b2e-43b9-a760-6b9943c7b5b7";>
According to https://infra.apache.org/github-actions-policy.html,
> All workflows SHOULD have a job concurrency level less than or equal to
15.
After this PR, the maximum concurrently level will be 10.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual review because this is a daily CI.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #46407 from dongjoon-hyun/SPARK-48149.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.github/workflows/build_python.yml | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/build_python.yml
b/.github/workflows/build_python.yml
index 3354fb726368..9195dc4af518 100644
--- a/.github/workflows/build_python.yml
+++ b/.github/workflows/build_python.yml
@@ -17,18 +17,26 @@
# under the License.
#
+# According to https://infra.apache.org/github-actions-policy.html,
+# all workflows SHOULD have a job concurrency level less than or equal to 15.
+# To do that, we run one python version per cron schedule
name: "Build / Python-only (master, PyPy 3.9/Python 3.10/Python 3.12)"
on:
schedule:
- cron: '0 15 * * *'
+ - cron: '0 17 * * *'
+ - cron: '0 19 * * *'
jobs:
run-build:
strategy:
fail-fast: false
matrix:
- pyversion: ["pypy3", "python3.10", "python3.12"]
+ include:
+ - pyversion: ${{ github.event.schedule == '0 15 * * *' && "pypy3" }}
+ - pyversion: ${{ github.event.schedule == '0 17 * * *' &&
"python3.10" }}
+ - pyversion: ${{ github.event.schedule == '0 19 * * *' &&
"python3.12" }}
permissions:
packages: write
name: Run
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
