Thank you for the suggestions, actually this project is already on spark-packages for 1~2 months. Then I think what I need is some promotions :P
2015-08-25 23:51 GMT+08:00 saurfang [via Apache Spark Developers List] < ml-node+s1001551n1380...@n3.nabble.com>: > This is very cool. I also have a sbt plugin that automates some aspects of > spark-submit but for a slightly different goal: > https://github.com/saurfang/sbt-spark-submit > > The hope there is to address the problem that one can have many Spark main > functions in a single jar and doing development often involves: change the > code, sbt assembly, scp the jar to cluster, run spark-submit with fully > qualified classpath and additional application arguments. > With my plugin, I'm able to capture all these steps into customizable > single sbt tasks that are easy to remember (and auto completes in sbt > console) so you can have multiple sbt tasks corresponding to different Main > functions, sub-projects and/or default arguments and make the > build/deploy/submit cycle straight through. > > > Currently this works great for YARN because YARN takes care of the jar > upload and master URL discovery. I have long wanted to make my plugin work > with spark-ec2 so I can upload jar and infer the master URL > programatically. > > Thanks for sharing and like Akhil said it'd be nice to have it on > spark-packages for discovery. > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-developers-list.1001551.n3.nabble.com/Introduce-a-sbt-plugin-to-deploy-and-submit-jobs-to-a-spark-cluster-on-ec2-tp13703p13809.html > To unsubscribe from Introduce a sbt plugin to deploy and submit jobs to a > spark cluster on ec2, click here > <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=13703&code=cGlzaGVuMDJAZ21haWwuY29tfDEzNzAzfC0xNjIyODI0MzY2> > . > NAML > <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Introduce-a-sbt-plugin-to-deploy-and-submit-jobs-to-a-spark-cluster-on-ec2-tp13703p13810.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
