This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ddc1f6b2a466 [SPARK-48106][INFRA] Use `Python 3.11` in `pyspark` tests
of `build_and_test.yml`
ddc1f6b2a466 is described below
commit ddc1f6b2a466892110ea0010c36f83847b9dc36e
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu May 2 23:34:47 2024 -0700
[SPARK-48106][INFRA] Use `Python 3.11` in `pyspark` tests of
`build_and_test.yml`
### What changes were proposed in this pull request?
This PR aims to use Python `3.11` instead of `3.9` in `pyspark` tests of
`build_and_test.yml`.
### Why are the changes needed?
To reduce GitHub Action usage to meet ASF INFRA policy.
- https://infra.apache.org/github-actions-policy.html
> The average number of minutes a project uses in any consecutive
five-day period MUST NOT exceed the equivalent of 30 full-time runners (216,000
minutes, or 3,600 hours).
`Python 3.11` is faster in general.
- https://docs.python.org/3/whatsnew/3.11.html#summary-release-highlights
> Python 3.11 is between 10-60% faster than Python 3.10. On average, we
measured a 1.25x speedup on the standard benchmark suite.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #46353 from dongjoon-hyun/SPARK-48106.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/build_and_test.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 56516c95dcb8..3bb37e74805f 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -366,7 +366,7 @@ jobs:
pyspark-pandas-connect-part3
env:
MODULES_TO_TEST: ${{ matrix.modules }}
- PYTHON_TO_TEST: 'python3.9'
+ PYTHON_TO_TEST: 'python3.11'
HADOOP_PROFILE: ${{ inputs.hadoop }}
HIVE_PROFILE: hive2.3
GITHUB_PREV_SHA: ${{ github.event.before }}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
