Re: How to use spark-submit

2014-05-12 Thread Stephen Boesch
@Sonal - makes sense. Is the maven shade plugin runnable within sbt ? If so would you care to share those build.sbt (or .scala) lines? If not, are you aware of a similar plugin for sbt? 2014-05-11 23:53 GMT-07:00 Sonal Goyal sonalgoy...@gmail.com: Hi Stephen, I am using maven shade

Re: How to use spark-submit

2014-05-11 Thread Stephen Boesch
of spark-submit. Thanks On May 5, 2014, at 10:24 PM, Stephen Boesch java...@gmail.com wrote: I have a spark streaming application that uses the external streaming modules (e.g. kafka, mqtt, ..) as well. It is not clear how to properly invoke the spark-submit script: what

Invoke spark-shell without attempting to start the http server

2014-05-02 Thread Stephen Boesch
We have a spark server already running. When invoking spark-shell a new http server is attempted to be started spark.HttpServer: Starting HTTP Server But that attempts results in a BindException due to the preexisting server: java.net.BindException: Address already in use What is the

Optimal Server Design for Spark

2014-04-02 Thread Stephen Watt
Hi Folks I'm looking to buy some gear to run Spark. I'm quite well versed in Hadoop Server design but there does not seem to be much Spark related collateral around infrastructure guidelines (or at least I haven't been able to find them). My current thinking for server design is something

Spark Training Scripts (from Strata 09/13) Ec2 Deployment scripts having errors

2014-03-08 Thread Stephen Boesch
The spark-training scripts are not presently working 100%: the errors displayed when starting the slaves are shown below. Possibly a newer location for the files exists (I pulled from https://github.com/amplab/training-scripts an it is nearly 6 months old) cp: cannot create regular file

Re: Spark Training Scripts (from Strata 09/13) Ec2 Deployment scripts having errors

2014-03-08 Thread Stephen Boesch
: link_stat /root/mesos-ec2 failed: No such file or directory (2) But in this latest version the mesos errors appear not to be fatal: the cluster is in the process of coming up (copying wikipedia data now..) . 2014-03-08 6:26 GMT-08:00 Stephen Boesch java...@gmail.com: The spark-training scripts

<    1   2   3