Found it. Some how my host mapping was messing it up. Changing it to point to localhost worked.:
/etc/host #127.0.0.1 XXXXXX.com 127.0.0.1 localhost From: "Jain, Nishit" <nja...@underarmour.com<mailto:nja...@underarmour.com>> Date: Monday, December 19, 2016 at 2:54 PM To: "user@spark.apache.org<mailto:user@spark.apache.org>" <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: PySpark: [Errno 8] nodename nor servname provided, or not known Hi, I am using pre-built 'spark-2.0.1-bin-hadoop2.7’ and when I try to start pyspark, I get following message. Any ideas what could be wrong? I tried using python3, setting SPARK_LOCAL_IP to 127.0.0.1 but same error. ~ -> cd /Applications/spark-2.0.1-bin-hadoop2.7/bin/ /Applications/spark-2.0.1-bin-hadoop2.7/bin -> pyspark Python 2.7.12 (default, Oct 11 2016, 05:24:00) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin Type "help", "copyright", "credits" or "license" for more information. Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). 16/12/19 14:50:47 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 16/12/19 14:50:47 WARN Utils: Your hostname, XXXXXX.com resolves to a loopback address: 127.0.0.1; using XX.XX.XX.XXX instead (on interface en0) 16/12/19 14:50:47 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address Traceback (most recent call last): File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/shell.py", line 43, in <module> spark = SparkSession.builder\ File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/sql/session.py", line 169, in getOrCreate sc = SparkContext.getOrCreate(sparkConf) File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/context.py", line 294, in getOrCreate SparkContext(conf=conf or SparkConf()) File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/context.py", line 115, in __init__ conf, jsc, profiler_cls) File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/context.py", line 174, in _do_init self._accumulatorServer = accumulators._start_update_server() File "/Applications/spark-2.0.1-bin-hadoop2.7/python/pyspark/accumulators.py", line 259, in _start_update_server server = AccumulatorServer(("localhost", 0), _UpdateRequestHandler) File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 417, in __init__ self.server_bind() File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 431, in server_bind self.socket.bind(self.server_address) File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) socket.gaierror: [Errno 8] nodename nor servname provided, or not known Thanks, Nishit
