This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 61435b42fdc [SPARK-45142][INFRA] Specify the range for Spark Connect
dependencies in pyspark base image
61435b42fdc is described below
commit 61435b42fdc4071f35aba6af9248ff9ad8fc8514
Author: Hyukjin Kwon <gurwls...@apache.org>
AuthorDate: Wed Sep 13 21:20:19 2023 +0900
[SPARK-45142][INFRA] Specify the range for Spark Connect dependencies in
pyspark base image
### What changes were proposed in this pull request?
This PR proposes to pin the dependencies related to Spark Connect in its
base image according to the range we support.
See also
https://github.com/apache/spark/blob/master/python/docs/source/getting_started/install.rst#dependencies
### Why are the changes needed?
To properly test the dependency versions we support.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
In this PR, it will be tested.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #42898 from HyukjinKwon/SPARK-45142.
Authored-by: Hyukjin Kwon <gurwls...@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
dev/infra/Dockerfile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
index feee7415004..99423ce072c 100644
--- a/dev/infra/Dockerfile
+++ b/dev/infra/Dockerfile
@@ -88,7 +88,7 @@ RUN pypy3 -m pip install numpy 'pandas<=2.0.3' scipy coverage
matplotlib
RUN python3.9 -m pip install numpy 'pyarrow==12.0.1' 'pandas<=2.0.3' scipy
unittest-xml-reporting plotly>=4.8 'mlflow>=2.3.1' coverage matplotlib openpyxl
'memory-profiler==0.60.0' 'scikit-learn==1.1.*'
# Add Python deps for Spark Connect.
-RUN python3.9 -m pip install grpcio protobuf googleapis-common-protos
grpcio-status
+RUN python3.9 -m pip install 'grpcio>=1.48,<1.57' 'grpcio-status>=1.48,<1.57'
'protobuf==3.20.3' 'googleapis-common-protos==1.56.4'
# Add torch as a testing dependency for TorchDistributor
RUN python3.9 -m pip install torch torchvision --index-url
https://download.pytorch.org/whl/cpu
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
