Thomas Dudziak created SPARK-11115: -------------------------------------- Summary: IPv6 regression Key: SPARK-11115 URL: https://issues.apache.org/jira/browse/SPARK-11115 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.5.1 Environment: CentOS 6.7, Java 1.8.0_25, dual stack IPv4 + IPv6 Reporter: Thomas Dudziak Priority: Critical
When running Spark with -Djava.net.preferIPv6Addresses=true, I get this error: 15/10/14 14:36:01 ERROR SparkContext: Error initializing SparkContext. java.lang.AssertionError: assertion failed: Expected hostname at scala.Predef$.assert(Predef.scala:179) at org.apache.spark.util.Utils$.checkHost(Utils.scala:805) at org.apache.spark.storage.BlockManagerId.<init>(BlockManagerId.scala:48) at org.apache.spark.storage.BlockManagerId$.apply(BlockManagerId.scala:107) at org.apache.spark.storage.BlockManager.initialize(BlockManager.scala:190) at org.apache.spark.SparkContext.<init>(SparkContext.scala:528) at org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:1017) Looking at the code in question, it seems that the code will only work for IPv4 as it assumes ':' can't be part of the hostname (which it clearly can for IPv6 addresses). Instead, the code should probably use Guava's HostAndPort class, i.e.: def checkHost(host: String, message: String = "") { assert(!HostAndPort.fromString(host).hasPort, message) } def checkHostPort(hostPort: String, message: String = "") { assert(HostAndPort.fromString(hostPort).hasPort, message) } -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org