This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new fbfcd402851e [SPARK-48203][INFRA] Spin off `pyspark` tests from `build_branch34.yml` Daily CI fbfcd402851e is described below commit fbfcd402851ee604789b8ba72a1ee0e67ef5ebe4 Author: Dongjoon Hyun <dh...@apple.com> AuthorDate: Wed May 8 12:30:12 2024 -0700 [SPARK-48203][INFRA] Spin off `pyspark` tests from `build_branch34.yml` Daily CI ### What changes were proposed in this pull request? This PR aims to create `build_branch34_python.yml` in order to spin off `pyspark` tests from `build_branch34.yml` Daily CI. ### Why are the changes needed? Currently, `build_branch34.yml` creates more than 15 test pipelines concurrently which is beyond of ASF Infra policy. - https://github.com/apache/spark/actions/workflows/build_branch35.yml We had better offload this to `Python only Daily CI` like `master` branch's `Python Only` Daily CI. - https://github.com/apache/spark/actions/workflows/build_python_3.10.yml ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manual review. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #46480 from dongjoon-hyun/SPARK-48203. Authored-by: Dongjoon Hyun <dh...@apple.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- .github/workflows/build_branch34.yml | 1 - .../{build_branch34.yml => build_branch34_python.yml} | 13 +++---------- 2 files changed, 3 insertions(+), 11 deletions(-) diff --git a/.github/workflows/build_branch34.yml b/.github/workflows/build_branch34.yml index 68887970d4d8..deb6c4240797 100644 --- a/.github/workflows/build_branch34.yml +++ b/.github/workflows/build_branch34.yml @@ -43,7 +43,6 @@ jobs: jobs: >- { "build": "true", - "pyspark": "true", "sparkr": "true", "tpcds-1g": "true", "docker-integration-tests": "true", diff --git a/.github/workflows/build_branch34.yml b/.github/workflows/build_branch34_python.yml similarity index 74% copy from .github/workflows/build_branch34.yml copy to .github/workflows/build_branch34_python.yml index 68887970d4d8..c109ba2dc792 100644 --- a/.github/workflows/build_branch34.yml +++ b/.github/workflows/build_branch34_python.yml @@ -17,7 +17,7 @@ # under the License. # -name: "Build (branch-3.4, Scala 2.13, Hadoop 3, JDK 8)" +name: "Build / Python-only (branch-3.4)" on: schedule: @@ -36,17 +36,10 @@ jobs: hadoop: hadoop3 envs: >- { - "SCALA_PROFILE": "scala2.13", - "PYTHON_TO_TEST": "", - "ORACLE_DOCKER_IMAGE_NAME": "gvenzl/oracle-xe:21.3.0" + "PYTHON_TO_TEST": "" } jobs: >- { - "build": "true", "pyspark": "true", - "sparkr": "true", - "tpcds-1g": "true", - "docker-integration-tests": "true", - "k8s-integration-tests": "true", - "lint" : "true" + "pyspark-pandas": "true" } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org