This is an automated email from the ASF dual-hosted git repository. yikun pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark-docker.git
The following commit(s) were added to refs/heads/master by this push: new 243ce20 [SPARK-40969] Replace spark TGZ url with apache archive url 243ce20 is described below commit 243ce201296c20ae48b32a87d254800e8ad197ef Author: Qian.Sun <qian.sun2...@gmail.com> AuthorDate: Tue Nov 1 11:14:04 2022 +0800 [SPARK-40969] Replace spark TGZ url with apache archive url ### What changes were proposed in this pull request? This PR aims to replace spark TGZ url with apache archive url. ### Why are the changes needed? ``` #13 [linux/amd64 4/9] RUN set -ex; export SPARK_TMP="$(mktemp -d)"; cd $SPARK_TMP; wget -nv -O spark.tgz "https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz"; wget -nv -O spark.tgz.asc "https://downloads.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc"; export GNUPGHOME="$(mktemp -d)"; gpg --keyserver hkps://keys.openpgp.org --recv-key "80FB8EBE8EBA68504989703491B5DC815DBF10D3" || gpg --keyserver hkps://keyserver.ubuntu.com [...] #0 0.132 ++ mktemp -d #0 0.133 + export SPARK_TMP=/tmp/tmp.oEdW8CyP9h #0 0.133 + SPARK_TMP=/tmp/tmp.oEdW8CyP9h #0 0.133 + cd /tmp/tmp.oEdW8CyP9h #0 0.133 + wget -nv -O spark.tgz https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz #0 0.152 https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz: #0 0.152 2022-10-31 04:06:44 ERROR 404: Not Found. #13 ERROR: process "/bin/sh -c set -ex; export SPARK_TMP=\"$(mktemp -d)\"; cd $SPARK_TMP; wget -nv -O spark.tgz \"$SPARK_TGZ_URL\"; wget -nv -O spark.tgz.asc \"$SPARK_TGZ_ASC_URL\"; export GNUPGHOME=\"$(mktemp -d)\"; gpg --keyserver hkps://keys.openpgp.org --recv-key \"$GPG_KEY\" || gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys \"$GPG_KEY\"; gpg --batch --verify spark.tgz.asc spark.tgz; gpgconf --kill all; rm -rf \"$GNUPGHOME\" spark.t [...] ``` Old url `https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz` is not found. Better to use unity apache archive url. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? No need to add new tests. Closes #22 from dcoliversun/SPARK-40969. Authored-by: Qian.Sun <qian.sun2...@gmail.com> Signed-off-by: Yikun Jiang <yikunk...@gmail.com> --- 3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile | 4 ++-- 3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile | 4 ++-- 3.3.0/scala2.12-java11-r-ubuntu/Dockerfile | 4 ++-- 3.3.0/scala2.12-java11-ubuntu/Dockerfile | 4 ++-- 3.3.1/scala2.12-java11-python3-r-ubuntu/Dockerfile | 4 ++-- 3.3.1/scala2.12-java11-python3-ubuntu/Dockerfile | 4 ++-- 3.3.1/scala2.12-java11-r-ubuntu/Dockerfile | 4 ++-- 3.3.1/scala2.12-java11-ubuntu/Dockerfile | 4 ++-- Dockerfile.template | 4 ++-- 9 files changed, 18 insertions(+), 18 deletions(-) diff --git a/3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile b/3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile index 8c2761e..fb48b80 100644 --- a/3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile +++ b/3.3.0/scala2.12-java11-python3-r-ubuntu/Dockerfile @@ -42,8 +42,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc \ GPG_KEY=80FB8EBE8EBA68504989703491B5DC815DBF10D3 RUN set -ex; \ diff --git a/3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile b/3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile index 6a0017a..1b6a02c 100644 --- a/3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile +++ b/3.3.0/scala2.12-java11-python3-ubuntu/Dockerfile @@ -41,8 +41,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc \ GPG_KEY=80FB8EBE8EBA68504989703491B5DC815DBF10D3 RUN set -ex; \ diff --git a/3.3.0/scala2.12-java11-r-ubuntu/Dockerfile b/3.3.0/scala2.12-java11-r-ubuntu/Dockerfile index 975e444..0b49aef 100644 --- a/3.3.0/scala2.12-java11-r-ubuntu/Dockerfile +++ b/3.3.0/scala2.12-java11-r-ubuntu/Dockerfile @@ -40,8 +40,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc \ GPG_KEY=80FB8EBE8EBA68504989703491B5DC815DBF10D3 RUN set -ex; \ diff --git a/3.3.0/scala2.12-java11-ubuntu/Dockerfile b/3.3.0/scala2.12-java11-ubuntu/Dockerfile index c38e5dc..cbdf6c5 100644 --- a/3.3.0/scala2.12-java11-ubuntu/Dockerfile +++ b/3.3.0/scala2.12-java11-ubuntu/Dockerfile @@ -39,8 +39,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz.asc \ GPG_KEY=80FB8EBE8EBA68504989703491B5DC815DBF10D3 RUN set -ex; \ diff --git a/3.3.1/scala2.12-java11-python3-r-ubuntu/Dockerfile b/3.3.1/scala2.12-java11-python3-r-ubuntu/Dockerfile index b126f1f..def11ff 100644 --- a/3.3.1/scala2.12-java11-python3-r-ubuntu/Dockerfile +++ b/3.3.1/scala2.12-java11-python3-r-ubuntu/Dockerfile @@ -42,8 +42,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz.asc \ GPG_KEY=86727D43E73A415F67A0B1A14E68B3E6CD473653 RUN set -ex; \ diff --git a/3.3.1/scala2.12-java11-python3-ubuntu/Dockerfile b/3.3.1/scala2.12-java11-python3-ubuntu/Dockerfile index af02e65..3da8521 100644 --- a/3.3.1/scala2.12-java11-python3-ubuntu/Dockerfile +++ b/3.3.1/scala2.12-java11-python3-ubuntu/Dockerfile @@ -41,8 +41,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz.asc \ GPG_KEY=86727D43E73A415F67A0B1A14E68B3E6CD473653 RUN set -ex; \ diff --git a/3.3.1/scala2.12-java11-r-ubuntu/Dockerfile b/3.3.1/scala2.12-java11-r-ubuntu/Dockerfile index bc2dcf7..04ab3ea 100644 --- a/3.3.1/scala2.12-java11-r-ubuntu/Dockerfile +++ b/3.3.1/scala2.12-java11-r-ubuntu/Dockerfile @@ -40,8 +40,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz.asc \ GPG_KEY=86727D43E73A415F67A0B1A14E68B3E6CD473653 RUN set -ex; \ diff --git a/3.3.1/scala2.12-java11-ubuntu/Dockerfile b/3.3.1/scala2.12-java11-ubuntu/Dockerfile index c732406..8b342c4 100644 --- a/3.3.1/scala2.12-java11-ubuntu/Dockerfile +++ b/3.3.1/scala2.12-java11-ubuntu/Dockerfile @@ -39,8 +39,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz.asc \ GPG_KEY=86727D43E73A415F67A0B1A14E68B3E6CD473653 RUN set -ex; \ diff --git a/Dockerfile.template b/Dockerfile.template index a220247..4819cb2 100644 --- a/Dockerfile.template +++ b/Dockerfile.template @@ -48,8 +48,8 @@ RUN set -ex && \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://dlcdn.apache.org/spark/spark-{{ SPARK_VERSION }}/spark-{{ SPARK_VERSION }}-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://downloads.apache.org/spark/spark-{{ SPARK_VERSION }}/spark-{{ SPARK_VERSION }}-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-{{ SPARK_VERSION }}/spark-{{ SPARK_VERSION }}-bin-hadoop3.tgz \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-{{ SPARK_VERSION }}/spark-{{ SPARK_VERSION }}-bin-hadoop3.tgz.asc \ GPG_KEY={{ SPARK_GPG_KEY }} RUN set -ex; \ --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org