[ https://issues.apache.org/jira/browse/SPARK-39500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571488#comment-17571488 ]
Dongjoon Hyun commented on SPARK-39500: --------------------------------------- For a record, we need the following to use the official IPv6 Maven Repo. {code:java} export DEFAULT_ARTIFACT_REPOSITORY=https://ipv6.repo1.maven.org/maven2/{code} > Ivy doesn't work correctly on IPv6-only environment > --------------------------------------------------- > > Key: SPARK-39500 > URL: https://issues.apache.org/jira/browse/SPARK-39500 > Project: Spark > Issue Type: Sub-task > Components: Spark Core > Affects Versions: 3.4.0 > Reporter: Dongjoon Hyun > Priority: Major > > Ivy doesn't work correctly on IPv6. > {code} > SparkSubmitUtils.resolveMavenCoordinates( > "org.apache.logging.log4j:log4j-api:2.17.2", > SparkSubmitUtils.buildIvySettings(None, Some("/tmp/ivy")), > transitive = true) > {code} > {code} > % bin/spark-shell > 22/06/16 22:22:12 WARN Utils: Your hostname, m1ipv6.local resolves to a > loopback address: 127.0.0.1; using 2600:1700:232e:3de0:0:0:0:b instead (on > interface en0) > 22/06/16 22:22:12 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to > another address > ===== https://ipv6.repo1.maven.org/maven2/ > =====https://maven-central.storage-download.googleapis.com/maven2/ > Setting default log level to "WARN". > To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use > setLogLevel(newLevel). > 22/06/16 22:22:14 WARN NativeCodeLoader: Unable to load native-hadoop library > for your platform... using builtin-java classes where applicable > Spark context Web UI available at http://unknown1498776019fa.attlocal.net:4040 > Spark context available as 'sc' (master = local[*], app id = > local-1655443334687). > Spark session available as 'spark'. > Welcome to > ____ __ > / __/__ ___ _____/ /__ > _\ \/ _ \/ _ `/ __/ '_/ > /___/ .__/\_,_/_/ /_/\_\ version 3.4.0-SNAPSHOT > /_/ > Using Scala version 2.12.16 (OpenJDK 64-Bit Server VM, Java 17.0.3) > Type in expressions to have them evaluated. > Type :help for more information. > scala> :paste -raw > // Entering paste mode (ctrl-D to finish) > package org.apache.spark.deploy > object Download { > SparkSubmitUtils.resolveMavenCoordinates( > "org.apache.logging.log4j:log4j-api:2.17.2", > SparkSubmitUtils.buildIvySettings(None, Some("/tmp/ivy")), > transitive = true) > } > // Exiting paste mode, now interpreting. > scala> org.apache.spark.deploy.Download > ===== https://ipv6.repo1.maven.org/maven2/ > =====https://maven-central.storage-download.googleapis.com/maven2/ > :: loading settings :: url = > jar:file:/Users/dongjoon/APACHE/spark/assembly/target/scala-2.12/jars/ivy-2.5.0.jar!/org/apache/ivy/core/settings/ivysettings.xml > Ivy Default Cache set to: /tmp/ivy/cache > The jars for the packages stored in: /tmp/ivy/jars > org.apache.logging.log4j#log4j-api added as a dependency > :: resolving dependencies :: > org.apache.spark#spark-submit-parent-f47b503f-897e-4b92-95da-3806c32c220f;1.0 > confs: [default] > :: resolution report :: resolve 95ms :: artifacts dl 0ms > :: modules in use: > --------------------------------------------------------------------- > | | modules || artifacts | > | conf | number| search|dwnlded|evicted|| number|dwnlded| > --------------------------------------------------------------------- > | default | 1 | 0 | 0 | 0 || 0 | 0 | > --------------------------------------------------------------------- > :: problems summary :: > :::: WARNINGS > module not found: org.apache.logging.log4j#log4j-api;2.17.2 > ==== local-m2-cache: tried > > file:/Users/dongjoon/.m2/repository/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.pom > -- artifact org.apache.logging.log4j#log4j-api;2.17.2!log4j-api.jar: > > file:/Users/dongjoon/.m2/repository/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar > ==== local-ivy-cache: tried > > /tmp/ivy/local/org.apache.logging.log4j/log4j-api/2.17.2/ivys/ivy.xml > -- artifact org.apache.logging.log4j#log4j-api;2.17.2!log4j-api.jar: > > /tmp/ivy/local/org.apache.logging.log4j/log4j-api/2.17.2/jars/log4j-api.jar > ==== ipv6: tried > > https://ipv6.repo1.maven.org/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.pom > -- artifact org.apache.logging.log4j#log4j-api;2.17.2!log4j-api.jar: > > https://ipv6.repo1.maven.org/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar > ==== central: tried > > https://maven-central.storage-download.googleapis.com/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.pom > -- artifact org.apache.logging.log4j#log4j-api;2.17.2!log4j-api.jar: > > https://maven-central.storage-download.googleapis.com/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar > ==== spark-packages: tried > > https://maven-central.storage-download.googleapis.com/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.pom > -- artifact org.apache.logging.log4j#log4j-api;2.17.2!log4j-api.jar: > > https://maven-central.storage-download.googleapis.com/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar > :::::::::::::::::::::::::::::::::::::::::::::: > :: UNRESOLVED DEPENDENCIES :: > :::::::::::::::::::::::::::::::::::::::::::::: > :: org.apache.logging.log4j#log4j-api;2.17.2: not found > :::::::::::::::::::::::::::::::::::::::::::::: > :::: ERRORS > Server access error at url > https://ipv6.repo1.maven.org/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.pom > (java.net.SocketException: Network is unreachable) > Server access error at url > https://ipv6.repo1.maven.org/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar > (java.net.SocketException: Network is unreachable) > Server access error at url > https://maven-central.storage-download.googleapis.com/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.pom > (java.net.SocketException: Network is unreachable) > Server access error at url > https://maven-central.storage-download.googleapis.com/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar > (java.net.SocketException: Network is unreachable) > Server access error at url > https://maven-central.storage-download.googleapis.com/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.pom > (java.net.SocketException: Network is unreachable) > Server access error at url > https://maven-central.storage-download.googleapis.com/maven2/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar > (java.net.SocketException: Network is unreachable) > :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS > java.lang.RuntimeException: [unresolved dependency: > org.apache.logging.log4j#log4j-api;2.17.2: not found] > at > org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1466) > at org.apache.spark.deploy.Download$.<init>(<pastie>:4) > at org.apache.spark.deploy.Download$.<clinit>(<pastie>) > ... 47 elided > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org