This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 86f15e4a779e [SPARK-47054][PYTHON][TESTS] Remove pinned version of torch for Python 3.12 support 86f15e4a779e is described below commit 86f15e4a779ec746373c78c189830cb339b07492 Author: Hyukjin Kwon <gurwls...@apache.org> AuthorDate: Thu Feb 15 22:28:37 2024 -0800 [SPARK-47054][PYTHON][TESTS] Remove pinned version of torch for Python 3.12 support ### What changes were proposed in this pull request? This PR unpins the version for torch in our CI. This PR is dependent on https://github.com/apache/spark/pull/45115 ### Why are the changes needed? Testing latest version. This also blocks SPARK-46078. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? Manually tested via `./dev/lint-python`. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #45113 from HyukjinKwon/SPARK-47054. Authored-by: Hyukjin Kwon <gurwls...@apache.org> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- dev/infra/Dockerfile | 6 +++--- dev/requirements.txt | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile index 53810756d30c..fa663bc6e419 100644 --- a/dev/infra/Dockerfile +++ b/dev/infra/Dockerfile @@ -100,7 +100,7 @@ ARG CONNECT_PIP_PKGS="grpcio==1.59.3 grpcio-status==1.59.3 protobuf==4.25.1 goog # Add torch as a testing dependency for TorchDistributor and DeepspeedTorchDistributor RUN python3.9 -m pip install $BASIC_PIP_PKGS unittest-xml-reporting $CONNECT_PIP_PKGS && \ - python3.9 -m pip install 'torch<=2.0.1' torchvision --index-url https://download.pytorch.org/whl/cpu && \ + python3.9 -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu && \ python3.9 -m pip install deepspeed torcheval && \ python3.9 -m pip cache purge @@ -111,7 +111,7 @@ RUN apt-get update && apt-get install -y \ && rm -rf /var/lib/apt/lists/* RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10 RUN python3.10 -m pip install $BASIC_PIP_PKGS unittest-xml-reporting $CONNECT_PIP_PKGS && \ - python3.10 -m pip install 'torch<=2.0.1' torchvision --index-url https://download.pytorch.org/whl/cpu && \ + python3.10 -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu && \ python3.10 -m pip install deepspeed torcheval && \ python3.10 -m pip cache purge @@ -122,7 +122,7 @@ RUN apt-get update && apt-get install -y \ && rm -rf /var/lib/apt/lists/* RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11 RUN python3.11 -m pip install $BASIC_PIP_PKGS unittest-xml-reporting $CONNECT_PIP_PKGS && \ - python3.11 -m pip install 'torch<=2.0.1' torchvision --index-url https://download.pytorch.org/whl/cpu && \ + python3.11 -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu && \ python3.11 -m pip install deepspeed torcheval && \ python3.11 -m pip cache purge diff --git a/dev/requirements.txt b/dev/requirements.txt index 46a02450d375..6fcd04b6d44a 100644 --- a/dev/requirements.txt +++ b/dev/requirements.txt @@ -61,7 +61,7 @@ googleapis-common-protos-stubs==2.2.0 grpc-stubs==1.24.11 # TorchDistributor dependencies -torch<=2.0.1 +torch torchvision torcheval --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org