This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 6dda3258533f [SPARK-47096][INFRA] Upgrade Python version in Maven builds 6dda3258533f is described below commit 6dda3258533fed149788cbf0d58e902ea78d51ee Author: Hyukjin Kwon <gurwls...@apache.org> AuthorDate: Mon Feb 19 16:58:01 2024 -0800 [SPARK-47096][INFRA] Upgrade Python version in Maven builds ### What changes were proposed in this pull request? This PR upgrades Python version to make the build compatible with macos-14 ### Why are the changes needed? macos-14 build fails during Python 3.9 installation, see https://github.com/apache/spark/actions/runs/7964626045/job/21742574260 ``` Error: dyld[4738]: Library not loaded: /usr/local/opt/gettext/lib/libintl.8.dylib Referenced from: <E2FD7085-038F Error: -3B8E-94C6-6649527BFDBE> /Users/runner/hostedtoolcache/Python/3.9.18/x64/bin/python3.9 Reason: tried: '/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/opt/gettext/lib/libintl.8.dylib' (no such file), '/usr/local/lib/libintl.8.dylib' (no such file), '/usr/lib/libintl.8.dylib' (no such file, not in dyld cache) ./setup.sh: line 52: 4738 Abort trap: 6 ./python -m ensurepip ``` In fact, marcos-14 already has Python 3.11 (https://github.com/actions/runner-images/blob/main/images/macos/macos-14-Readme.md). ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? Will monitor the scheduled build. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #45172 from HyukjinKwon/SPARK-47096. Lead-authored-by: Hyukjin Kwon <gurwls...@apache.org> Co-authored-by: Hyukjin Kwon <gurwls...@gmail.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- .github/workflows/maven_test.yml | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/.github/workflows/maven_test.yml b/.github/workflows/maven_test.yml index ed2af672d6e0..80898b3a507a 100644 --- a/.github/workflows/maven_test.yml +++ b/.github/workflows/maven_test.yml @@ -159,20 +159,22 @@ jobs: with: distribution: zulu java-version: ${{ matrix.java }} - - name: Install Python 3.9 + - name: Install Python 3.11 uses: actions/setup-python@v5 # We should install one Python that is higher than 3+ for SQL and Yarn because: # - SQL component also has Python related tests, for example, IntegratedUDFTestUtils. # - Yarn has a Python specific test too, for example, YarnClusterSuite. - if: contains(matrix.modules, 'resource-managers#yarn') || (contains(matrix.modules, 'sql#core')) || contains(matrix.modules, 'connect') + # macos (14) already has its Python installed, see also SPARK-47096 and + # https://github.com/actions/runner-images/blob/main/images/macos/macos-14-Readme.md + if: contains(inputs.os, 'ubuntu') && (contains(matrix.modules, 'resource-managers#yarn') || (contains(matrix.modules, 'sql#core')) || contains(matrix.modules, 'connect')) with: - python-version: '3.9' + python-version: '3.11' architecture: x64 - - name: Install Python packages (Python 3.9) + - name: Install Python packages (Python 3.11) if: (contains(matrix.modules, 'sql#core')) || contains(matrix.modules, 'connect') run: | - python3.9 -m pip install 'numpy>=1.20.0' pyarrow pandas scipy unittest-xml-reporting 'grpcio==1.59.3' 'grpcio-status==1.59.3' 'protobuf==4.25.1' - python3.9 -m pip list + python3.11 -m pip install 'numpy>=1.20.0' pyarrow pandas scipy unittest-xml-reporting 'grpcio==1.59.3' 'grpcio-status==1.59.3' 'protobuf==4.25.1' + python3.11 -m pip list # Run the tests. - name: Run tests env: ${{ fromJSON(inputs.envs) }} --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org