This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new ff4ca0b1bca [SPARK-45812][BUILD][PYTHON][PS] Upgrade Pandas to 2.1.2 ff4ca0b1bca is described below commit ff4ca0b1bcab7a5f8f14f845a168b12664e16e51 Author: Haejoon Lee <haejoon....@databricks.com> AuthorDate: Mon Nov 6 20:10:10 2023 -0800 [SPARK-45812][BUILD][PYTHON][PS] Upgrade Pandas to 2.1.2 ### What changes were proposed in this pull request? This PR proposes to upgrade Pandas to 2.1.2. See https://pandas.pydata.org/docs/dev/whatsnew/v2.1.2.html for detail ### Why are the changes needed? Pandas 2.1.2 is released, and we should support the latest Pandas. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? The existing CI should pass ### Was this patch authored or co-authored using generative AI tooling? No. Closes #43689 from itholic/SPARK-45812. Authored-by: Haejoon Lee <haejoon....@databricks.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- dev/infra/Dockerfile | 4 ++-- python/pyspark/pandas/supported_api_gen.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile index 001de613a92..e6a58cc3fc7 100644 --- a/dev/infra/Dockerfile +++ b/dev/infra/Dockerfile @@ -84,8 +84,8 @@ RUN Rscript -e "devtools::install_version('roxygen2', version='7.2.0', repos='ht # See more in SPARK-39735 ENV R_LIBS_SITE "/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library" -RUN pypy3 -m pip install numpy 'pandas<=2.1.1' scipy coverage matplotlib -RUN python3.9 -m pip install numpy pyarrow 'pandas<=2.1.1' scipy unittest-xml-reporting plotly>=4.8 'mlflow>=2.3.1' coverage matplotlib openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*' +RUN pypy3 -m pip install numpy 'pandas<=2.1.2' scipy coverage matplotlib +RUN python3.9 -m pip install numpy pyarrow 'pandas<=2.1.2' scipy unittest-xml-reporting plotly>=4.8 'mlflow>=2.3.1' coverage matplotlib openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*' # Add Python deps for Spark Connect. RUN python3.9 -m pip install 'grpcio>=1.48,<1.57' 'grpcio-status>=1.48,<1.57' 'protobuf==3.20.3' 'googleapis-common-protos==1.56.4' diff --git a/python/pyspark/pandas/supported_api_gen.py b/python/pyspark/pandas/supported_api_gen.py index c4471a0af36..8d49fef2799 100644 --- a/python/pyspark/pandas/supported_api_gen.py +++ b/python/pyspark/pandas/supported_api_gen.py @@ -98,7 +98,7 @@ def generate_supported_api(output_rst_file_path: str) -> None: Write supported APIs documentation. """ - pandas_latest_version = "2.1.1" + pandas_latest_version = "2.1.2" if LooseVersion(pd.__version__) != LooseVersion(pandas_latest_version): msg = ( "Warning: Latest version of pandas (%s) is required to generate the documentation; " --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org