Hi Everyone i am new to spark ... I am posting some basic doubts i met while trying to create a standalone cluster for a small poc ...
1)My Corporate firewall blocked the port 7077, which is the default port of Master URL , So i used start-master.sh --port 8080 (also tried with several other ports and -p) but still the port used by master was unchanged. I resolved this problem by addingexport SPARK_MASTER_PORT="59468" in spark-env.sh . My first doubt is why -p PORT, --port PORT didn't work? 2)same result when tried with spark spark-class org.apache.spark.deploy.worker.Worker spark://IP:PORT 3)Also Why is by default the port random for worker ..Sorry if i am being stupid The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
