On Mon, Sep 14, 2015 at 6:55 AM, Adrian Bridgett <adr...@opensignal.com> wrote: > 15/09/14 13:00:25 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, > 10.1.200.245): java.lang.IllegalArgumentException: > java.net.UnknownHostException: nameservice1 > at > org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377) > at > org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
This looks like you're trying to connect to an HA HDFS service but you have not provided the proper hdfs-site.xml for your app; then, instead of recognizing "nameservice1" as an HA nameservice, it thinks it's an actual NN address, tries to connect to it, and fails. If you provide the correct hdfs-site.xml to your app (by placing it in $SPARK_HOME/conf or setting HADOOP_CONF_DIR to point to the conf directory), it should work. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
