This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8b2251734519 [SPARK-48135][INFRA] Run `buf` and `ui` only in PR
builders and Java 21 Daily CI
8b2251734519 is described below
commit 8b22517345190e007ca87c7491116ad590ad46f2
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Sun May 5 16:40:11 2024 -0700
[SPARK-48135][INFRA] Run `buf` and `ui` only in PR builders and Java 21
Daily CI
### What changes were proposed in this pull request?
This PR aims to run `buf` and `ui` tests only in PR builders and Java 21
Daily CI.
### Why are the changes needed?
Currently, Apache Spark CI is running `buf` and `ui` tests always because
they finish quickly.
https://github.com/apache/spark/blob/32ba5c1db62caaaa2674e8acced56f89ed840bf9/.github/workflows/build_and_test.yml#L102-L103
- `buf` job
https://github.com/apache/spark/blob/32ba5c1db62caaaa2674e8acced56f89ed840bf9/.github/workflows/build_and_test.yml#L571-L574
- `ui` job
https://github.com/apache/spark/blob/32ba5c1db62caaaa2674e8acced56f89ed840bf9/.github/workflows/build_and_test.yml#L1049-L1052
However, ASF Infra team's guideline recommends to maintain the job
concurrency level under or equal to `15`. We had better offload `buf` and `ui`
from per-commit CI.
- https://infra.apache.org/github-actions-policy.html
> All workflows SHOULD have a job concurrency level less than or equal to
15.
### Does this PR introduce _any_ user-facing change?
No because this is an infra update.
### How was this patch tested?
Pass the CIs and manual review because PR builders will not be affected by
this.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #46392 from dongjoon-hyun/SPARK-48135.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.github/workflows/build_and_test.yml | 8 ++++++--
.github/workflows/build_java21.yml | 4 +++-
2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index f626cd72be15..8a85d26c0eca 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -82,10 +82,14 @@ jobs:
pandas=$pyspark
kubernetes=`./dev/is-changed.py -m kubernetes`
sparkr=`./dev/is-changed.py -m sparkr`
+ buf=true
+ ui=true
else
pandas=false
kubernetes=false
sparkr=false
+ buf=false
+ ui=false
fi
# 'build' is always true for now.
# It does not save significant time and most of PRs trigger the
build.
@@ -99,8 +103,8 @@ jobs:
\"docker-integration-tests\": \"false\",
\"lint\" : \"true\",
\"k8s-integration-tests\" : \"$kubernetes\",
- \"buf\" : \"true\",
- \"ui\" : \"true\",
+ \"buf\" : \"$buf\",
+ \"ui\" : \"$ui\",
}"
echo $precondition # For debugging
# Remove `\n` to avoid "Invalid format" error
diff --git a/.github/workflows/build_java21.yml
b/.github/workflows/build_java21.yml
index bfeedd4174cf..a2fb0e6e2c1d 100644
--- a/.github/workflows/build_java21.yml
+++ b/.github/workflows/build_java21.yml
@@ -47,5 +47,7 @@ jobs:
"sparkr": "true",
"tpcds-1g": "true",
"docker-integration-tests": "true",
- "k8s-integration-tests": "true"
+ "k8s-integration-tests": "true",
+ "buf": "true",
+ "ui": "true"
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
