Re: Sharing Spark Drivers

2015-02-24 Thread Chip Senkbeil
-with-the-Spark-Kernel Signed, Chip Senkbeil On Tue Feb 24 2015 at 8:04:08 AM John Omernik j...@omernik.com wrote: I have been posting on the Mesos list, as I am looking to see if it it's possible or not to share spark drivers. Obviously, in stand alone cluster mode, the Master handles requests

Re: spark driver behind firewall

2015-02-06 Thread Chip Senkbeil
send Scala code to interact with Apache Spark. You would still need to expose the Spark Kernel outside the firewall (similar to Kostas' suggestion about the jobserver), of course. Signed, Chip Senkbeil On Thu Feb 05 2015 at 11:07:28 PM Kostas Sakellis kos...@cloudera.com wrote: Yes, the driver has

Re: How to design a long live spark application

2015-02-05 Thread Chip Senkbeil
: https://github.com/ibm-et/spark-kernel/wiki/Getting-Started-with-the-Spark-Kernel If you have any more questions about the project, feel free to email me! Signed, Chip Senkbeil On Thu Feb 05 2015 at 10:58:01 AM Corey Nolet cjno...@gmail.com wrote: Here's another lightweight example of running

Re: SVD in pyspark ?

2015-01-25 Thread Chip Senkbeil
message protocol - used by IPython 3.0 - to communicate. See the getting started section here: https://github.com/ibm-et/spark-kernel/wiki/Getting-Started-with-the-Spark-Kernel It discusses getting IPython connected to a Spark Kernel. If you have any more questions, feel free to ask! Signed, Chip