[jira] [Created] (SPARK-34349) No python3 in docker images
Tim Hughes created SPARK-34349: -- Summary: No python3 in docker images Key: SPARK-34349 URL: https://issues.apache.org/jira/browse/SPARK-34349 Project: Spark Issue Type: Bug Components: Kubernetes Affects Versions: 3.0.1 Reporter: Tim Hughes The spark-py container image doesn't receive the instruction to use python3 and defaults to python 2.7 The worker container was build using the following commands {code:java} mkdir ./tmp wget -qO- https://www.mirrorservice.org/sites/ftp.apache.org/spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz | tar -C ./tmp/ -xzf - cd ../spark-3.0.1-bin-hadoop3.2/ ./bin/docker-image-tool.sh -r docker.io/timhughes -t spark-3.0.1-bin-hadoop3.2 -p kubernetes/dockerfiles/spark/bindings/python/Dockerfile build docker push docker.io/timhughes/spark-py:spark-3.0.1-bin-hadoop3.2{code} This is the code I am using to initialize the workers {code:java} import os from pyspark import SparkContext, SparkConf from pyspark.sql import SparkSession# Create Spark config for our Kubernetes based cluster manager sparkConf = SparkConf() sparkConf.setMaster("k8s://https://kubernetes.default.svc.cluster.local:443;) sparkConf.setAppName("spark") sparkConf.set("spark.kubernetes.container.image", "docker.io/timhughes/spark-py:spark-3.0.1-bin-hadoop3.2") sparkConf.set("spark.kubernetes.namespace", "spark") sparkConf.set("spark.executor.instances", "2") sparkConf.set("spark.executor.cores", "1") sparkConf.set("spark.driver.memory", "1024m") sparkConf.set("spark.executor.memory", "1024m") sparkConf.set("spark.kubernetes.pyspark.pythonVersion", "3") sparkConf.set("spark.kubernetes.authenticate.driver.serviceAccountName", "spark") sparkConf.set("spark.kubernetes.authenticate.serviceAccountName", "spark") sparkConf.set("spark.driver.port", "29413") sparkConf.set("spark.driver.host", "my-notebook-deployment.spark.svc.cluster.local") # Initialize our Spark cluster, this will actually # generate the worker nodes. spark = SparkSession.builder.config(conf=sparkConf).getOrCreate() sc = spark.sparkContext {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-34315) docker-image-tool.sh debconf trying to configure kerberos
[ https://issues.apache.org/jira/browse/SPARK-34315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17276681#comment-17276681 ] Tim Hughes commented on SPARK-34315: Created a pull request https://github.com/apache/spark/pull/31425 > docker-image-tool.sh debconf trying to configure kerberos > - > > Key: SPARK-34315 > URL: https://issues.apache.org/jira/browse/SPARK-34315 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.0.1 >Reporter: Tim Hughes >Priority: Critical > Attachments: full-logs.txt > > > When building the docker containers using the docker-image-tool.sh there is > RUN `apt install -y bash tini libc6 libpam-modules krb5-user libnss3` Which > leads to `debconf` trying to configure kerberos. I have tried putting > nothing, EXAMPLE.COM, my corporate kerberos realm and none of them work. it > just hangs after enter is pressed > > {{Setting up krb5-config (2.6) ...}} > {{debconf: unable to initialize frontend: Dialog}} > {{debconf: (TERM is not set, so the dialog frontend is not usable.)}} > {{debconf: falling back to frontend: Readline}} > {{debconf: unable to initialize frontend: Readline}} > {{debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install > the Term::ReadLine module) (@INC contains: /etc/perl > /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 > /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 > /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at > /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)}} > {{debconf: falling back to frontend: Teletype}} > {{Configuring Kerberos Authentication}} > {{---}} > {{When users attempt to use Kerberos and specify a principal or user name > without}} > {{specifying what administrative Kerberos realm that principal belongs to, > the}} > {{system appends the default realm. The default realm may also be used as > the}} > {{realm of a Kerberos service running on the local machine. Often, the > default}} > {{realm is the uppercase version of the local DNS domain.}} > {{Default Kerberos version 5 realm: EXAMPLE.ORG}} > {{^CFailed to build Spark JVM Docker image, please refer to Docker build > output for details.}} > > > {{## Steps to reproduce}} > {{```}} > {{wget -qO- > https://www.mirrorservice.org/sites/ftp.apache.org/spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz > | tar -xzf -}} > {{cd spark-3.0.1-bin-hadoop3.2/}} > {{./bin/docker-image-tool.sh build}} > {{```}} > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-34315) docker-image-tool.sh debconf trying to configure kerberos
[ https://issues.apache.org/jira/browse/SPARK-34315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17276680#comment-17276680 ] Tim Hughes commented on SPARK-34315: [https://github.com/apache/spark/blob/master/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile#L34] Prefixing the line that installs krb5-user with `DEBIAN_FRONTEND=noninteractive` allows the container to be built. > docker-image-tool.sh debconf trying to configure kerberos > - > > Key: SPARK-34315 > URL: https://issues.apache.org/jira/browse/SPARK-34315 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.0.1 >Reporter: Tim Hughes >Priority: Critical > Attachments: full-logs.txt > > > When building the docker containers using the docker-image-tool.sh there is > RUN `apt install -y bash tini libc6 libpam-modules krb5-user libnss3` Which > leads to `debconf` trying to configure kerberos. I have tried putting > nothing, EXAMPLE.COM, my corporate kerberos realm and none of them work. it > just hangs after enter is pressed > > {{Setting up krb5-config (2.6) ...}} > {{debconf: unable to initialize frontend: Dialog}} > {{debconf: (TERM is not set, so the dialog frontend is not usable.)}} > {{debconf: falling back to frontend: Readline}} > {{debconf: unable to initialize frontend: Readline}} > {{debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install > the Term::ReadLine module) (@INC contains: /etc/perl > /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 > /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 > /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at > /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)}} > {{debconf: falling back to frontend: Teletype}} > {{Configuring Kerberos Authentication}} > {{---}} > {{When users attempt to use Kerberos and specify a principal or user name > without}} > {{specifying what administrative Kerberos realm that principal belongs to, > the}} > {{system appends the default realm. The default realm may also be used as > the}} > {{realm of a Kerberos service running on the local machine. Often, the > default}} > {{realm is the uppercase version of the local DNS domain.}} > {{Default Kerberos version 5 realm: EXAMPLE.ORG}} > {{^CFailed to build Spark JVM Docker image, please refer to Docker build > output for details.}} > > > {{## Steps to reproduce}} > {{```}} > {{wget -qO- > https://www.mirrorservice.org/sites/ftp.apache.org/spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz > | tar -xzf -}} > {{cd spark-3.0.1-bin-hadoop3.2/}} > {{./bin/docker-image-tool.sh build}} > {{```}} > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-34315) docker-image-tool.sh debconf trying to configure kerberos
[ https://issues.apache.org/jira/browse/SPARK-34315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Hughes updated SPARK-34315: --- Environment: (was: # ## Full logs {{}}{{}}) > docker-image-tool.sh debconf trying to configure kerberos > - > > Key: SPARK-34315 > URL: https://issues.apache.org/jira/browse/SPARK-34315 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.0.1 >Reporter: Tim Hughes >Priority: Critical > Attachments: full-logs.txt > > > When building the docker containers using the docker-image-tool.sh there is > RUN `apt install -y bash tini libc6 libpam-modules krb5-user libnss3` Which > leads to `debconf` trying to configure kerberos. I have tried putting > nothing, EXAMPLE.COM, my corporate kerberos realm and none of them work. it > just hangs after enter is pressed > > {{Setting up krb5-config (2.6) ...}} > {{debconf: unable to initialize frontend: Dialog}} > {{debconf: (TERM is not set, so the dialog frontend is not usable.)}} > {{debconf: falling back to frontend: Readline}} > {{debconf: unable to initialize frontend: Readline}} > {{debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install > the Term::ReadLine module) (@INC contains: /etc/perl > /usr/local/lib/x86_64-linux-gnu/perl/5.28.1 /usr/local/share/perl/5.28.1 > /usr/lib/x86_64-linux-gnu/perl5/5.28 /usr/share/perl5 > /usr/lib/x86_64-linux-gnu/perl/5.28 /usr/share/perl/5.28 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at > /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)}} > {{debconf: falling back to frontend: Teletype}} > {{Configuring Kerberos Authentication}} > {{---}} > {{When users attempt to use Kerberos and specify a principal or user name > without}} > {{specifying what administrative Kerberos realm that principal belongs to, > the}} > {{system appends the default realm. The default realm may also be used as > the}} > {{realm of a Kerberos service running on the local machine. Often, the > default}} > {{realm is the uppercase version of the local DNS domain.}} > {{Default Kerberos version 5 realm: EXAMPLE.ORG}} > {{^CFailed to build Spark JVM Docker image, please refer to Docker build > output for details.}} > > > {{## Steps to reproduce}} > {{```}} > {{wget -qO- > https://www.mirrorservice.org/sites/ftp.apache.org/spark/spark-3.0.1/spark-3.0.1-bin-hadoop3.2.tgz > | tar -xzf -}} > {{cd spark-3.0.1-bin-hadoop3.2/}} > {{./bin/docker-image-tool.sh build}} > {{```}} > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-34315) docker-image-tool.sh debconf trying to configure kerberos
[ https://issues.apache.org/jira/browse/SPARK-34315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Hughes updated SPARK-34315: --- Attachment: full-logs.txt > docker-image-tool.sh debconf trying to configure kerberos > - > > Key: SPARK-34315 > URL: https://issues.apache.org/jira/browse/SPARK-34315 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.0.1 > Environment: ## Full logs > > {{}}{{$ bin/docker-image-tool.sh build}} > {{Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet > msg.}} > {{STEP 1: FROM openjdk:8-jre-slim}} > {{STEP 2: ARG spark_uid=185}} > {{--> Using cache > d24913e4f80a167a2682380bc0565b0eefac2e7e5b94f1491b99712e1154dd3b}} > {{--> d24913e4f80}} > {{STEP 3: RUN set -ex && sed -i 's/http:\/\/deb.\(.*\)/https:\/\/deb.\1/g' > /etc/apt/sources.list && apt-get update && ln -s /lib /lib64 && apt install > -y bash tini libc6 libpam-modules krb5-user libnss3 && mkdir -p /opt/spark && > mkdir -p /opt/spark/examples && mkdir -p /opt/spark/work-dir && touch > /opt/spark/RELEASE && rm /bin/sh && ln -sv /bin/bash /bin/sh && echo "auth > required pam_wheel.so use_uid" >> /etc/pam.d/su && chgrp root /etc/passwd && > chmod ug+rw /etc/passwd && rm -rf /var/cache/apt/*}} > {{+ sed -i s/http:\/\/deb.\(.*\)/https:\/\/deb.\1/g /etc/apt/sources.list}} > {{+ apt-get update}} > {{Get:1 http://security.debian.org/debian-security buster/updates InRelease > [65.4 kB]}} > {{Get:2 https://deb.debian.org/debian buster InRelease [121 kB] }} > {{Get:3 https://deb.debian.org/debian buster-updates InRelease [51.9 kB]}} > {{Get:4 http://security.debian.org/debian-security buster/updates/main amd64 > Packages [271 kB]}} > {{Get:5 https://deb.debian.org/debian buster/main amd64 Packages [7907 kB]}} > {{Get:6 https://deb.debian.org/debian buster-updates/main amd64 Packages > [7848 B]}} > {{Fetched 8426 kB in 4s (1995 kB/s) }} > {{Reading package lists... Done}} > {{+ ln -s /lib /lib64}} > {{+ apt install -y bash tini libc6 libpam-modules krb5-user libnss3}} > {{Reading package lists... Done}} > {{Building dependency tree }} > {{Reading state information... Done}} > {{bash is already the newest version (5.0-4).}} > {{bash set to manually installed.}} > {{libc6 is already the newest version (2.28-10).}} > {{libc6 set to manually installed.}} > {{libpam-modules is already the newest version (1.3.1-5).}} > {{libpam-modules set to manually installed.}} > {{The following package was automatically installed and is no longer > required:}} > {{ lsb-base}} > {{Use 'apt autoremove' to remove it.}} > {{The following additional packages will be installed:}} > {{ bind9-host geoip-database krb5-config krb5-locales libbind9-161 libcap2}} > {{ libdns1104 libfstrm0 libgeoip1 libgssapi-krb5-2 libgssrpc4 libicu63}} > {{ libisc1100 libisccc161 libisccfg163 libjson-c3 libk5crypto3}} > {{ libkadm5clnt-mit11 libkadm5srv-mit11 libkdb5-9 libkeyutils1 libkrb5-3}} > {{ libkrb5support0 liblmdb0 liblwres161 libnspr4 libprotobuf-c1 libsqlite3-0}} > {{ libxml2}} > {{Suggested packages:}} > {{ krb5-k5tls geoip-bin krb5-doc}} > {{The following NEW packages will be installed:}} > {{ bind9-host geoip-database krb5-config krb5-locales krb5-user libbind9-161}} > {{ libcap2 libdns1104 libfstrm0 libgeoip1 libgssapi-krb5-2 libgssrpc4 > libicu63}} > {{ libisc1100 libisccc161 libisccfg163 libjson-c3 libk5crypto3}} > {{ libkadm5clnt-mit11 libkadm5srv-mit11 libkdb5-9 libkeyutils1 libkrb5-3}} > {{ libkrb5support0 liblmdb0 liblwres161 libnspr4 libnss3 libprotobuf-c1}} > {{ libsqlite3-0 libxml2 tini}} > {{0 upgraded, 32 newly installed, 0 to remove and 2 not upgraded.}} > {{Need to get 18.1 MB of archives.}} > {{After this operation, 61.3 MB of additional disk space will be used.}} > {{Get:1 https://deb.debian.org/debian buster/main amd64 libcap2 amd64 > 1:2.25-2 [17.6 kB]}} > {{Get:2 https://deb.debian.org/debian buster/main amd64 libfstrm0 amd64 > 0.4.0-1 [20.8 kB]}} > {{Get:3 https://deb.debian.org/debian buster/main amd64 libgeoip1 amd64 > 1.6.12-1 [93.1 kB]}} > {{Get:4 https://deb.debian.org/debian buster/main amd64 libjson-c3 amd64 > 0.12.1+ds-2+deb10u1 [27.3 kB]}} > {{Get:5 https://deb.debian.org/debian buster/main amd64 liblmdb0 amd64 > 0.9.22-1 [45.0 kB]}} > {{Get:6 https://deb.debian.org/debian buster/main amd64 libprotobuf-c1 amd64 > 1.3.1-1+b1 [26.5 kB]}} > {{Get:7 https://deb.debian.org/debian buster/main amd64 libicu63 amd64 > 63.1-6+deb10u1 [8300 kB]}} > {{Get:8 https://deb.debian.org/debian buster/main amd64 libxml2 amd64 > 2.9.4+dfsg1-7+deb10u1 [689 kB]}} > {{Get:9 https://deb.debian.org/debian buster/main amd64 libisc1100 amd64 > 1:9.11.5.P4+dfsg-5.1+deb10u2 [458 kB]}} > {{Get:10 https://deb.debian.org/debian buster/main amd64 libkeyutils1 amd64 > 1.6-6 [15.0 kB]}} > {{Get:11
[jira] [Updated] (SPARK-34315) docker-image-tool.sh debconf trying to configure kerberos
[ https://issues.apache.org/jira/browse/SPARK-34315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Hughes updated SPARK-34315: --- Environment: # ## Full logs {{}}{{}} was: ## Full logs {{}}{{$ bin/docker-image-tool.sh build}} {{Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.}} {{STEP 1: FROM openjdk:8-jre-slim}} {{STEP 2: ARG spark_uid=185}} {{--> Using cache d24913e4f80a167a2682380bc0565b0eefac2e7e5b94f1491b99712e1154dd3b}} {{--> d24913e4f80}} {{STEP 3: RUN set -ex && sed -i 's/http:\/\/deb.\(.*\)/https:\/\/deb.\1/g' /etc/apt/sources.list && apt-get update && ln -s /lib /lib64 && apt install -y bash tini libc6 libpam-modules krb5-user libnss3 && mkdir -p /opt/spark && mkdir -p /opt/spark/examples && mkdir -p /opt/spark/work-dir && touch /opt/spark/RELEASE && rm /bin/sh && ln -sv /bin/bash /bin/sh && echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && chgrp root /etc/passwd && chmod ug+rw /etc/passwd && rm -rf /var/cache/apt/*}} {{+ sed -i s/http:\/\/deb.\(.*\)/https:\/\/deb.\1/g /etc/apt/sources.list}} {{+ apt-get update}} {{Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]}} {{Get:2 https://deb.debian.org/debian buster InRelease [121 kB] }} {{Get:3 https://deb.debian.org/debian buster-updates InRelease [51.9 kB]}} {{Get:4 http://security.debian.org/debian-security buster/updates/main amd64 Packages [271 kB]}} {{Get:5 https://deb.debian.org/debian buster/main amd64 Packages [7907 kB]}} {{Get:6 https://deb.debian.org/debian buster-updates/main amd64 Packages [7848 B]}} {{Fetched 8426 kB in 4s (1995 kB/s) }} {{Reading package lists... Done}} {{+ ln -s /lib /lib64}} {{+ apt install -y bash tini libc6 libpam-modules krb5-user libnss3}} {{Reading package lists... Done}} {{Building dependency tree }} {{Reading state information... Done}} {{bash is already the newest version (5.0-4).}} {{bash set to manually installed.}} {{libc6 is already the newest version (2.28-10).}} {{libc6 set to manually installed.}} {{libpam-modules is already the newest version (1.3.1-5).}} {{libpam-modules set to manually installed.}} {{The following package was automatically installed and is no longer required:}} {{ lsb-base}} {{Use 'apt autoremove' to remove it.}} {{The following additional packages will be installed:}} {{ bind9-host geoip-database krb5-config krb5-locales libbind9-161 libcap2}} {{ libdns1104 libfstrm0 libgeoip1 libgssapi-krb5-2 libgssrpc4 libicu63}} {{ libisc1100 libisccc161 libisccfg163 libjson-c3 libk5crypto3}} {{ libkadm5clnt-mit11 libkadm5srv-mit11 libkdb5-9 libkeyutils1 libkrb5-3}} {{ libkrb5support0 liblmdb0 liblwres161 libnspr4 libprotobuf-c1 libsqlite3-0}} {{ libxml2}} {{Suggested packages:}} {{ krb5-k5tls geoip-bin krb5-doc}} {{The following NEW packages will be installed:}} {{ bind9-host geoip-database krb5-config krb5-locales krb5-user libbind9-161}} {{ libcap2 libdns1104 libfstrm0 libgeoip1 libgssapi-krb5-2 libgssrpc4 libicu63}} {{ libisc1100 libisccc161 libisccfg163 libjson-c3 libk5crypto3}} {{ libkadm5clnt-mit11 libkadm5srv-mit11 libkdb5-9 libkeyutils1 libkrb5-3}} {{ libkrb5support0 liblmdb0 liblwres161 libnspr4 libnss3 libprotobuf-c1}} {{ libsqlite3-0 libxml2 tini}} {{0 upgraded, 32 newly installed, 0 to remove and 2 not upgraded.}} {{Need to get 18.1 MB of archives.}} {{After this operation, 61.3 MB of additional disk space will be used.}} {{Get:1 https://deb.debian.org/debian buster/main amd64 libcap2 amd64 1:2.25-2 [17.6 kB]}} {{Get:2 https://deb.debian.org/debian buster/main amd64 libfstrm0 amd64 0.4.0-1 [20.8 kB]}} {{Get:3 https://deb.debian.org/debian buster/main amd64 libgeoip1 amd64 1.6.12-1 [93.1 kB]}} {{Get:4 https://deb.debian.org/debian buster/main amd64 libjson-c3 amd64 0.12.1+ds-2+deb10u1 [27.3 kB]}} {{Get:5 https://deb.debian.org/debian buster/main amd64 liblmdb0 amd64 0.9.22-1 [45.0 kB]}} {{Get:6 https://deb.debian.org/debian buster/main amd64 libprotobuf-c1 amd64 1.3.1-1+b1 [26.5 kB]}} {{Get:7 https://deb.debian.org/debian buster/main amd64 libicu63 amd64 63.1-6+deb10u1 [8300 kB]}} {{Get:8 https://deb.debian.org/debian buster/main amd64 libxml2 amd64 2.9.4+dfsg1-7+deb10u1 [689 kB]}} {{Get:9 https://deb.debian.org/debian buster/main amd64 libisc1100 amd64 1:9.11.5.P4+dfsg-5.1+deb10u2 [458 kB]}} {{Get:10 https://deb.debian.org/debian buster/main amd64 libkeyutils1 amd64 1.6-6 [15.0 kB]}} {{Get:11 https://deb.debian.org/debian buster/main amd64 libkrb5support0 amd64 1.17-3+deb10u1 [65.8 kB]}} {{Get:12 https://deb.debian.org/debian buster/main amd64 libk5crypto3 amd64 1.17-3+deb10u1 [122 kB]}} {{Get:13 https://deb.debian.org/debian buster/main amd64 libkrb5-3 amd64 1.17-3+deb10u1 [369 kB]}} {{Get:14 https://deb.debian.org/debian buster/main amd64 libgssapi-krb5-2 amd64 1.17-3+deb10u1 [158 kB]}} {{Get:15 https://deb.debian.org/debian buster/main amd64 libdns1104 amd64 1:9.11.5.P4+dfsg-5.1+deb10u2 [1223 kB]}}
[jira] [Created] (SPARK-34315) docker-image-tool.sh debconf trying to configure kerberos
Tim Hughes created SPARK-34315: -- Summary: docker-image-tool.sh debconf trying to configure kerberos Key: SPARK-34315 URL: https://issues.apache.org/jira/browse/SPARK-34315 Project: Spark Issue Type: Bug Components: Kubernetes Affects Versions: 3.0.1 Environment: ## Full logs {{}}{{$ bin/docker-image-tool.sh build}} {{Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.}} {{STEP 1: FROM openjdk:8-jre-slim}} {{STEP 2: ARG spark_uid=185}} {{--> Using cache d24913e4f80a167a2682380bc0565b0eefac2e7e5b94f1491b99712e1154dd3b}} {{--> d24913e4f80}} {{STEP 3: RUN set -ex && sed -i 's/http:\/\/deb.\(.*\)/https:\/\/deb.\1/g' /etc/apt/sources.list && apt-get update && ln -s /lib /lib64 && apt install -y bash tini libc6 libpam-modules krb5-user libnss3 && mkdir -p /opt/spark && mkdir -p /opt/spark/examples && mkdir -p /opt/spark/work-dir && touch /opt/spark/RELEASE && rm /bin/sh && ln -sv /bin/bash /bin/sh && echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && chgrp root /etc/passwd && chmod ug+rw /etc/passwd && rm -rf /var/cache/apt/*}} {{+ sed -i s/http:\/\/deb.\(.*\)/https:\/\/deb.\1/g /etc/apt/sources.list}} {{+ apt-get update}} {{Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]}} {{Get:2 https://deb.debian.org/debian buster InRelease [121 kB] }} {{Get:3 https://deb.debian.org/debian buster-updates InRelease [51.9 kB]}} {{Get:4 http://security.debian.org/debian-security buster/updates/main amd64 Packages [271 kB]}} {{Get:5 https://deb.debian.org/debian buster/main amd64 Packages [7907 kB]}} {{Get:6 https://deb.debian.org/debian buster-updates/main amd64 Packages [7848 B]}} {{Fetched 8426 kB in 4s (1995 kB/s) }} {{Reading package lists... Done}} {{+ ln -s /lib /lib64}} {{+ apt install -y bash tini libc6 libpam-modules krb5-user libnss3}} {{Reading package lists... Done}} {{Building dependency tree }} {{Reading state information... Done}} {{bash is already the newest version (5.0-4).}} {{bash set to manually installed.}} {{libc6 is already the newest version (2.28-10).}} {{libc6 set to manually installed.}} {{libpam-modules is already the newest version (1.3.1-5).}} {{libpam-modules set to manually installed.}} {{The following package was automatically installed and is no longer required:}} {{ lsb-base}} {{Use 'apt autoremove' to remove it.}} {{The following additional packages will be installed:}} {{ bind9-host geoip-database krb5-config krb5-locales libbind9-161 libcap2}} {{ libdns1104 libfstrm0 libgeoip1 libgssapi-krb5-2 libgssrpc4 libicu63}} {{ libisc1100 libisccc161 libisccfg163 libjson-c3 libk5crypto3}} {{ libkadm5clnt-mit11 libkadm5srv-mit11 libkdb5-9 libkeyutils1 libkrb5-3}} {{ libkrb5support0 liblmdb0 liblwres161 libnspr4 libprotobuf-c1 libsqlite3-0}} {{ libxml2}} {{Suggested packages:}} {{ krb5-k5tls geoip-bin krb5-doc}} {{The following NEW packages will be installed:}} {{ bind9-host geoip-database krb5-config krb5-locales krb5-user libbind9-161}} {{ libcap2 libdns1104 libfstrm0 libgeoip1 libgssapi-krb5-2 libgssrpc4 libicu63}} {{ libisc1100 libisccc161 libisccfg163 libjson-c3 libk5crypto3}} {{ libkadm5clnt-mit11 libkadm5srv-mit11 libkdb5-9 libkeyutils1 libkrb5-3}} {{ libkrb5support0 liblmdb0 liblwres161 libnspr4 libnss3 libprotobuf-c1}} {{ libsqlite3-0 libxml2 tini}} {{0 upgraded, 32 newly installed, 0 to remove and 2 not upgraded.}} {{Need to get 18.1 MB of archives.}} {{After this operation, 61.3 MB of additional disk space will be used.}} {{Get:1 https://deb.debian.org/debian buster/main amd64 libcap2 amd64 1:2.25-2 [17.6 kB]}} {{Get:2 https://deb.debian.org/debian buster/main amd64 libfstrm0 amd64 0.4.0-1 [20.8 kB]}} {{Get:3 https://deb.debian.org/debian buster/main amd64 libgeoip1 amd64 1.6.12-1 [93.1 kB]}} {{Get:4 https://deb.debian.org/debian buster/main amd64 libjson-c3 amd64 0.12.1+ds-2+deb10u1 [27.3 kB]}} {{Get:5 https://deb.debian.org/debian buster/main amd64 liblmdb0 amd64 0.9.22-1 [45.0 kB]}} {{Get:6 https://deb.debian.org/debian buster/main amd64 libprotobuf-c1 amd64 1.3.1-1+b1 [26.5 kB]}} {{Get:7 https://deb.debian.org/debian buster/main amd64 libicu63 amd64 63.1-6+deb10u1 [8300 kB]}} {{Get:8 https://deb.debian.org/debian buster/main amd64 libxml2 amd64 2.9.4+dfsg1-7+deb10u1 [689 kB]}} {{Get:9 https://deb.debian.org/debian buster/main amd64 libisc1100 amd64 1:9.11.5.P4+dfsg-5.1+deb10u2 [458 kB]}} {{Get:10 https://deb.debian.org/debian buster/main amd64 libkeyutils1 amd64 1.6-6 [15.0 kB]}} {{Get:11 https://deb.debian.org/debian buster/main amd64 libkrb5support0 amd64 1.17-3+deb10u1 [65.8 kB]}} {{Get:12 https://deb.debian.org/debian buster/main amd64 libk5crypto3 amd64 1.17-3+deb10u1 [122 kB]}} {{Get:13 https://deb.debian.org/debian buster/main amd64 libkrb5-3 amd64 1.17-3+deb10u1 [369 kB]}} {{Get:14 https://deb.debian.org/debian buster/main amd64 libgssapi-krb5-2 amd64