This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new de66d8f2ec7 [SPARK-45706][PYTHON][DOCS] Makes entire Binder build
fails fast during setting up
de66d8f2ec7 is described below
commit de66d8f2ec78525ea73eba8a0d8fa6e4d4839ead
Author: Hyukjin Kwon <gurwls...@apache.org>
AuthorDate: Fri Oct 27 21:20:40 2023 +0900
[SPARK-45706][PYTHON][DOCS] Makes entire Binder build fails fast during
setting up
### What changes were proposed in this pull request?
This PR proposes to make entire Binder build fails fast during setting up
to prevent the Binder image to be successfully built which it cannot be rebuilt
later on the same commit.
### Why are the changes needed?
Binder build is currently broken for Spark 3.5.0:
https://mybinder.org/v2/gh/apache/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb
Seems like we uploaded PySpark late into PyPI, and the installation steps
just slightly ignored the failure (a user triggered the first docker image for
Binder, and that's being reused at that time PySpark wasn't uploaded to PyPI).
### Does this PR introduce _any_ user-facing change?
Yes, it fixes the user-facing live notebooks (at
https://spark.apache.org/docs/latest/api/python/index.html).
### How was this patch tested?
Manually tested in my fork:
-
https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_df.ipynb
-
https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_connect.ipynb
-
https://mybinder.org/v2/gh/HyukjinKwon/spark/ce5ddad9903?filepath=python%2Fdocs%2Fsource%2Fgetting_started%2Fquickstart_ps.ipynb
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #43553 from HyukjinKwon/SPARK-45706.
Authored-by: Hyukjin Kwon <gurwls...@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
(cherry picked from commit f1d1dc1f87a7e4accd1c3c2d824f39df05465906)
Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
binder/postBuild | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/binder/postBuild b/binder/postBuild
index 70ae23b3937..fec233f8c8c 100644
--- a/binder/postBuild
+++ b/binder/postBuild
@@ -20,6 +20,11 @@
# This file is used for Binder integration to install PySpark available in
# Jupyter notebook.
+# SPARK-45706: Should fail fast. Otherwise, the Binder image is successfully
+# built, and it cannot be rebuilt.
+set -o pipefail
+set -e
+
VERSION=$(python -c "exec(open('python/pyspark/version.py').read());
print(__version__)")
TAG=$(git describe --tags --exact-match 2>/dev/null)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
