Re: SPARK_MASTER_IP actually expects a DNS name, not IP address

2015-10-18 Thread Robert Dodier
Nicholas, FWIW the --ip option seems to have been deprecated in commit d90d2af1, but that was a pretty big commit, lots of other stuff changed, and there isn't any hint in the log message as to the reason for changing --ip. best, Robert Dodier

Re: SPARK_MASTER_IP actually expects a DNS name, not IP address

2015-10-16 Thread Nicholas Chammas
JB, I am using spark-env.sh to define the master address instead of using spark-defaults.conf. I understand that that should work, and indeed it does, but only if SPARK_MASTER_IP is set to a DNS name and not an IP address. Perhaps I'm misunderstanding these configuration methods... Nick On

Re: SPARK_MASTER_IP actually expects a DNS name, not IP address

2015-10-16 Thread Jean-Baptiste Onofré
Hi Nick, there's the Spark master defined in conf/spark-defaults.conf and the -h option that you can provide to sbin/start-master.sh script. Did you try: sbin/start-master.sh -h xxx.xxx.xxx.xxx and then use the IP when you start the slaves: sbin/start-slave.sh spark://xxx.xxx.xxx.xxx.7077

Re: SPARK_MASTER_IP actually expects a DNS name, not IP address

2015-10-16 Thread Sean Owen
It's used in scripts like sbin/start-master.sh On Fri, Oct 16, 2015 at 5:01 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > I'd look into tracing a possible bug here, but I'm not sure where to look. > Searching the codebase for `SPARK_MASTER_IP`, amazingly, does not show it > being

Re: SPARK_MASTER_IP actually expects a DNS name, not IP address

2015-10-16 Thread Ted Yu
if [ "$SPARK_MASTER_IP" = "" ]; then SPARK_MASTER_IP=`hostname` --ip $SPARK_MASTER_IP --port $SPARK_MASTER_PORT --webui-port $SPARK_MASTER_WEBUI_PORT \ "$sbin"/../tachyon/bin/tachyon bootstrap-conf $SPARK_MASTER_IP ./sbin/start-master.sh if [ "$SPARK_MASTER_IP" = "" ]; then

Re: SPARK_MASTER_IP actually expects a DNS name, not IP address

2015-10-16 Thread Nicholas Chammas
I'd look into tracing a possible bug here, but I'm not sure where to look. Searching the codebase for `SPARK_MASTER_IP`, amazingly, does not show it being used in any place directly by Spark . Clearly, Spark is using this

Re: SPARK_MASTER_IP actually expects a DNS name, not IP address

2015-10-16 Thread Nicholas Chammas
Ah, my bad, I missed it since the GitHub search results preview only showed the first hit from start-master.sh and

Re: SPARK_MASTER_IP actually expects a DNS name, not IP address

2015-10-14 Thread Ted Yu
Some old bits: http://stackoverflow.com/questions/28162991/cant-run-spark-1-2-in-standalone-mode-on-mac http://stackoverflow.com/questions/29412157/passing-hostname-to-netty FYI On Wed, Oct 14, 2015 at 7:10 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > I’m setting the Spark