This is an automated email from the ASF dual-hosted git repository.
gaogaotiantian pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 0fea5ee6d6f4 [SPARK-57911][DOC] Upgrade sphinx version to 8.2.3
0fea5ee6d6f4 is described below
commit 0fea5ee6d6f43dfe252561d429cb1dd9e45935f7
Author: Tian Gao <[email protected]>
AuthorDate: Mon Jul 6 18:10:50 2026 -0700
[SPARK-57911][DOC] Upgrade sphinx version to 8.2.3
### What changes were proposed in this pull request?
Upgrade sphinx version to 8.2.3
### Why are the changes needed?
The current 4.5.0 is too old (2022). We had to pin some extension versions
to make it work. We should upgrade util tools from time to time.
### Does this PR introduce _any_ user-facing change?
No. Sphinx is only used to generate our docs. Our users do not use it at
all.
### How was this patch tested?
CI.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #56980 from gaogaotiantian/upgrade-sphinx.
Authored-by: Tian Gao <[email protected]>
Signed-off-by: Tian Gao <[email protected]>
---
.github/workflows/pages.yml | 5 ++---
dev/create-release/spark-rm/Dockerfile | 12 ++++--------
dev/requirements.txt | 11 ++---------
dev/spark-test-image/docs/Dockerfile | 6 ++----
4 files changed, 10 insertions(+), 24 deletions(-)
diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml
index fdf15337cf67..49b342f1dad3 100644
--- a/.github/workflows/pages.yml
+++ b/.github/workflows/pages.yml
@@ -60,11 +60,10 @@ jobs:
cache: 'pip'
- name: Install Python dependencies
run: |
- pip install 'sphinx==4.5.0' mkdocs 'pydata_sphinx_theme>=0.13'
sphinx-copybutton nbsphinx numpydoc jinja2 markupsafe 'pyzmq<24.0.0' \
- ipython ipython_genutils sphinx_plotly_directive 'numpy>=1.23.2'
pyarrow 'pandas==2.3.3' 'plotly>=4.8' 'docutils<0.18.0' \
+ pip install 'sphinx==8.2.3' mkdocs 'pydata_sphinx_theme>=0.13'
sphinx-copybutton nbsphinx numpydoc jinja2 markupsafe 'pyzmq<24.0.0' \
+ ipython ipython_genutils sphinx_plotly_directive 'numpy>=1.23.2'
pyarrow 'pandas==2.3.3' 'plotly>=4.8' 'docutils' \
'flake8==3.9.0' 'mypy==1.8.0' 'pytest==7.1.3'
'pytest-mypy-plugins==1.9.3' 'ruff==0.14.8' \
'pandas-stubs==1.2.0.53' 'grpcio==1.76.0' 'grpcio-status==1.76.0'
'protobuf==6.33.5' 'grpc-stubs==1.24.11'
'googleapis-common-protos-stubs==2.2.0' \
- 'sphinxcontrib-applehelp==1.0.4' 'sphinxcontrib-devhelp==1.0.2'
'sphinxcontrib-htmlhelp==2.0.1' 'sphinxcontrib-qthelp==1.0.3'
'sphinxcontrib-serializinghtml==1.1.5'
- name: Install Ruby for documentation generation
uses: ruby/setup-ruby@4dc28cf14d77b0afa6832d9765ac422dbf0dfedd # v1
with:
diff --git a/dev/create-release/spark-rm/Dockerfile
b/dev/create-release/spark-rm/Dockerfile
index f25da7a969ec..e3b1504fba1d 100644
--- a/dev/create-release/spark-rm/Dockerfile
+++ b/dev/create-release/spark-rm/Dockerfile
@@ -55,17 +55,13 @@ RUN python3.11 -m pip install --ignore-installed
'blinker>=1.6.2' && \
python3.11 -m pip cache purge
# Sphinx and documentation packages
-# Should unpin 'sphinxcontrib-*' after upgrading sphinx>5
-# See 'ipython_genutils' in SPARK-38517, 'docutils<0.18.0' in SPARK-39421
-RUN python3.11 -m pip install 'sphinx==4.5.0' mkdocs
'pydata_sphinx_theme>=0.13' \
+# See 'ipython_genutils' in SPARK-38517, 'docutils' in SPARK-39421
+RUN python3.11 -m pip install 'sphinx==8.2.3' mkdocs
'pydata_sphinx_theme>=0.13' \
sphinx-copybutton nbsphinx numpydoc jinja2 markupsafe 'pyzmq<24.0.0' \
ipython ipython_genutils sphinx_plotly_directive 'numpy>=1.23.2' pyarrow
pandas \
- 'plotly>=4.8' 'docutils<0.18.0' 'flake8==3.9.0' 'mypy==1.19.1'
'pytest==7.1.3' \
+ 'plotly>=4.8' 'docutils' 'flake8==3.9.0' 'mypy==1.19.1' 'pytest==7.1.3' \
'pytest-mypy-plugins==1.9.3' 'ruff==0.14.8' 'pandas-stubs==1.2.0.53' \
- 'grpcio==1.76.0' 'grpc-stubs==1.24.11'
'googleapis-common-protos-stubs==2.2.0' \
- 'sphinxcontrib-applehelp==1.0.4' 'sphinxcontrib-devhelp==1.0.2' \
- 'sphinxcontrib-htmlhelp==2.0.1' 'sphinxcontrib-qthelp==1.0.3' \
- 'sphinxcontrib-serializinghtml==1.1.5'
+ 'grpcio==1.76.0' 'grpc-stubs==1.24.11'
'googleapis-common-protos-stubs==2.2.0'
# Set Python 3.11 as the default
RUN ln -sf "$(which python3.11)" "/usr/local/bin/python" && \
diff --git a/dev/requirements.txt b/dev/requirements.txt
index 759045fd4ac9..5e99abf1e34c 100644
--- a/dev/requirements.txt
+++ b/dev/requirements.txt
@@ -45,17 +45,10 @@ ipython
nbsphinx
numpydoc
jinja2
-sphinx==4.5.0
-# With sphinx 4.5.0, we need to set the upperbound version of sphinxcontrib*,
it should be removed after upgrading sphinx>=5
-sphinxcontrib-applehelp<=1.0.4
-sphinxcontrib-devhelp<=1.0.2
-sphinxcontrib-htmlhelp<=2.0.1
-sphinxcontrib-jsmath<=1.0.1
-sphinxcontrib-qthelp<=1.0.3
-sphinxcontrib-serializinghtml<=1.1.5
+sphinx==8.2.3
sphinx-plotly-directive
sphinx-copybutton
-docutils<0.18.0
+docutils
markupsafe
# Development scripts
diff --git a/dev/spark-test-image/docs/Dockerfile
b/dev/spark-test-image/docs/Dockerfile
index 8b13d9534670..fd4bc7f1e6ec 100644
--- a/dev/spark-test-image/docs/Dockerfile
+++ b/dev/spark-test-image/docs/Dockerfile
@@ -89,12 +89,10 @@ ENV VIRTUAL_ENV=/opt/spark-venv
RUN python3.12 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
-# Should unpin 'sphinxcontrib-*' after upgrading sphinx>5
# See 'ipython_genutils' in SPARK-38517
# See 'docutils<0.18.0' in SPARK-39421
-RUN python3.12 -m pip install 'sphinx==4.5.0' mkdocs
'pydata_sphinx_theme>=0.13' sphinx-copybutton nbsphinx numpydoc jinja2
markupsafe \
- ipython ipython_genutils sphinx_plotly_directive 'numpy>=1.23.2'
'pyarrow>=23.0.0' 'pandas==2.3.3' 'plotly>=4.8' 'docutils<0.18.0' \
+RUN python3.12 -m pip install 'sphinx==8.2.3' mkdocs
'pydata_sphinx_theme>=0.13' sphinx-copybutton nbsphinx numpydoc jinja2
markupsafe \
+ ipython ipython_genutils sphinx_plotly_directive 'numpy>=1.23.2'
'pyarrow>=23.0.0' 'pandas==2.3.3' 'plotly>=4.8' 'docutils' \
'flake8==3.9.0' 'mypy==1.19.1' 'pytest==7.1.3' 'pytest-mypy-plugins==1.9.3'
'ruff==0.14.8' \
'pandas-stubs==1.2.0.53' 'grpcio==1.76.0' 'grpcio-status==1.76.0'
'protobuf==6.33.5' 'grpc-stubs==1.24.11'
'googleapis-common-protos-stubs==2.2.0' \
- 'sphinxcontrib-applehelp==1.0.4' 'sphinxcontrib-devhelp==1.0.2'
'sphinxcontrib-htmlhelp==2.0.1' 'sphinxcontrib-qthelp==1.0.3'
'sphinxcontrib-serializinghtml==1.1.5' \
&& python3.12 -m pip cache purge
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
