Re: Running spark-shell (or queries) over the network (not from master)
Have you tryied with ssh? It will be much secure (only 1 port open), and you'll be able to run spark-shell over the networ. I'm using that way in my project (https://github.com/data-tsunami/smoke) with good results. I can't make a try now, but something like this should work: ssh -tt ec2-user@YOUR-EC2-IP /path/to/spark-shell SPARK-SHELL-OPTIONS With this approach you are way more secure (without installing a VPN), you don't need spark/hadoop installed on your PC. You won't have acces to local files, but you haven't mentioned that as a requirement :-) Hope this help you. Horacio -- Web: http://www.data-tsunami.com Email: hgde...@gmail.com Cel: +54 9 3572 525359 LinkedIn: https://www.linkedin.com/in/hgdeoro - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Announce: Smoke - a web frontend to Spark
Hi everyone! I've been working on Smoke, a web frontend to interactively launch Spark jobs without compiling it (only support Scala right now, and launching the jobs on yarn-client mode). It works executing the Scala script using spark-shell in the Spark server. It's developed in Python, uses Celery/Redis to stream the job logs to the web and is easy to install. The project is on GitHub: https://github.com/data-tsunami/smoke (install instructions, screenshots, etc.). It's in an early development stage, but very usable. Thanks! Horacio -- Horacio G. de Oro Data Tsunami Email: hgde...@gmail.com Web: http://www.data-tsunami.com/english/ Cel: +54 9 3572 525359 LinkedIn: https://www.linkedin.com/in/hgdeoro - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
