Prashant Sharma created SPARK-31200:
---------------------------------------
Summary: Docker image build fails with Mirror sync in progress?
errors.
Key: SPARK-31200
URL: https://issues.apache.org/jira/browse/SPARK-31200
Project: Spark
Issue Type: Improvement
Components: Kubernetes
Affects Versions: 3.1.0
Reporter: Prashant Sharma
Somehow I was having trouble with Debian mirrors while building the spark
image. Following errors would appear, in spite of trying to switch between
various mirrors.
{code}
bash-3.2$ bin/docker-image-tool.sh -r scrapcodes -t v3.1.0-f1cc86 build
Sending build context to Docker daemon 203.4MB
Step 1/18 : ARG java_image_tag=8-jre-slim
Step 2/18 : FROM openjdk:${java_image_tag}
---> 381b20190cf7
Step 3/18 : ARG spark_uid=185
---> Using cache
---> 65c06f86753c
Step 4/18 : RUN set -ex && apt-get update && ln -s /lib /lib64 &&
apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps &&
mkdir -p /opt/spark && mkdir -p /opt/spark/examples && mkdir -p
/opt/spark/work-dir && touch /opt/spark/RELEASE && rm /bin/sh && ln
-sv /bin/bash /bin/sh && echo "auth required pam_wheel.so use_uid" >>
/etc/pam.d/su && chgrp root /etc/passwd && chmod ug+rw /etc/passwd &&
rm -rf /var/cache/apt/*
---> Running in 96bcbe927d35
+ apt-get update
Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
Get:2 http://deb.debian.org/debian buster-updates InRelease [49.3 kB]
Get:3 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
Err:3 http://deb.debian.org/debian buster/main amd64 Packages
File has unexpected size (13217 != 7906744). Mirror sync in progress? [IP:
151.101.10.133 80]
Hashes of expected file:
- Filesize:7906744 [weak]
- SHA256:80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc
- MD5Sum:80b6d9c1b6630b2234161e42f4040ab3 [weak]
Release file created at: Sat, 08 Feb 2020 10:57:10 +0000
Get:5 http://deb.debian.org/debian buster-updates/main amd64 Packages [7380 B]
Err:5 http://deb.debian.org/debian buster-updates/main amd64 Packages
File has unexpected size (13233 != 7380). Mirror sync in progress? [IP:
151.101.10.133 80]
Hashes of expected file:
- Filesize:7380 [weak]
- SHA256:6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d
Release file created at: Fri, 20 Mar 2020 02:28:11 +0000
Get:4 http://security-cdn.debian.org/debian-security buster/updates InRelease
[65.4 kB]
Get:6 http://security-cdn.debian.org/debian-security buster/updates/main amd64
Packages [183 kB]
Fetched 419 kB in 1s (327 kB/s)
Reading package lists...
E: Failed to fetch
http://deb.debian.org/debian/dists/buster/main/binary-amd64/by-hash/SHA256/80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc
File has unexpected size (13217 != 7906744). Mirror sync in progress? [IP:
151.101.10.133 80]
Hashes of expected file:
- Filesize:7906744 [weak]
- SHA256:80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc
- MD5Sum:80b6d9c1b6630b2234161e42f4040ab3 [weak]
Release file created at: Sat, 08 Feb 2020 10:57:10 +0000
E: Failed to fetch
http://deb.debian.org/debian/dists/buster-updates/main/binary-amd64/by-hash/SHA256/6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d
File has unexpected size (13233 != 7380). Mirror sync in progress? [IP:
151.101.10.133 80]
Hashes of expected file:
- Filesize:7380 [weak]
- SHA256:6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d
Release file created at: Fri, 20 Mar 2020 02:28:11 +0000
E: Some index files failed to download. They have been ignored, or old ones
used instead.
The command '/bin/sh -c set -ex && apt-get update && ln -s /lib /lib64
&& apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps
&& mkdir -p /opt/spark && mkdir -p /opt/spark/examples && mkdir -p
/opt/spark/work-dir && touch /opt/spark/RELEASE && rm /bin/sh && ln
-sv /bin/bash /bin/sh && echo "auth required pam_wheel.so use_uid" >>
/etc/pam.d/su && chgrp root /etc/passwd && chmod ug+rw /etc/passwd &&
rm -rf /var/cache/apt/*' returned a non-zero code: 100
Failed to build Spark JVM Docker image, please refer to Docker build output for
details.
{code}
Soon, I found that, it was due to my ISP (Since we are working from home now !)
trying to intercept http traffic. So it may not happen to others. But, just in
case, if some is hit by such errors. Changing mirrors to https from http helped
resolve it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
