Hi Kevin, Did you try adding a host name for the ipv6? I have a few ipv6 boxes, spark failed for me when i use just the ipv6 addresses, but it works fine when i use the host names.
Here's an entry in my /etc/hosts: 2607:5300:0100:0200:0000:0000:0000:0a4d hacked.work My spark-env.sh file: export SPARK_MASTER_IP="hacked.work" Here's the master listening on my v6: [image: Inline image 1] The Master UI with running spark-shell: [image: Inline image 2] I even ran a simple sc.parallelize(1 to 100).collect(). Thanks Best Regards On Wed, May 20, 2015 at 11:09 PM, Kevin Liu <kevin...@fb.com> wrote: > Hello, I have to work with IPv6 only servers and when I installed the > 1.3.1 hadoop 2.6 build, I couldn¹t get the example to run due to IPv6 > issues (errors below). I tried to add the > -Djava.net.preferIPv6Addresses=true setting but it still doesn¹t work. A > search on Spark¹s support for IPv6 is inconclusive. Can someone help > clarify the current status for IPv6? > > Thanks > Kevin > > > ‹‹ errors ‹ > > 5/05/20 10:17:30 INFO Executor: Fetching > http://2401:db00:2030:709b:face:0:9:0:51453 > /jars/spark-examples-1.3.1-hadoo > p2.6.0.jar with timestamp 1432142250197 > 15/05/20 10:17:30 INFO Executor: Fetching > http://2401:db00:2030:709b:face:0:9:0:51453 > /jars/spark-examples-1.3.1-hadoo > p2.6.0.jar with timestamp 1432142250197 > 15/05/20 10:17:30 ERROR Executor: Exception in task 5.0 in stage 0.0 (TID > 5) > java.net.MalformedURLException: For input string: > "db00:2030:709b:face:0:9:0:51453" > at java.net.URL.<init>(URL.java:620) > at java.net.URL.<init>(URL.java:483) > at java.net.URL.<init>(URL.java:432) > at org.apache.spark.util.Utils$.doFetchFile(Utils.scala:603) > at org.apache.spark.util.Utils$.fetchFile(Utils.scala:431) > at > org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Execu > tor$$updateDependencies$5.apply(Executor.scala:374) > at > org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Execu > tor$$updateDependencies$5.apply(Executor.scala:366) > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(Traver > sableLike.scala:772) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) > at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) > at > scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:7 > 71) > at > org.apache.spark.executor.Executor.org > $apache$spark$executor$Executor$$upda > teDependencies(Executor.scala:366) > at > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:184) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1 > 142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java: > 617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NumberFormatException: For input string: > "db00:2030:709b:face:0:9:0:51453" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:6 > 5) > at java.lang.Integer.parseInt(Integer.java:580) > at java.lang.Integer.parseInt(Integer.java:615) > at java.net.URLStreamHandler.parseURL(URLStreamHandler.java:216) > at java.net.URL.<init>(URL.java:615) > ... 18 more > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
