Hi, Spark job server by ooyala is the right tool for the job. It exposes rest api so calling it from a web app is suitable. Is open source, you can find it on github
Best Paolo Platter ________________________________ Da: Ruebenacker, Oliver A<mailto:oliver.ruebenac...@altisource.com> Inviato: 16/09/2014 21.18 A: Matei Zaharia<mailto:matei.zaha...@gmail.com>; user@spark.apache.org<mailto:user@spark.apache.org> Oggetto: RE: Spark as a Library Hello, Thanks for the response and great to hear it is possible. But how do I connect to Spark without using the submit script? I know how to start up a master and some workers and then connect to the master by packaging the app that contains the SparkContext and then submitting the package with the spark-submit script in standalone-mode. But I don’t want to submit the app that contains the SparkContext via the script, because I want that app to be running on a web server. So, what are other ways to connect to Spark? I can’t find in the docs anything other than using the script. Thanks! Best, Oliver From: Matei Zaharia [mailto:matei.zaha...@gmail.com] Sent: Tuesday, September 16, 2014 1:31 PM To: Ruebenacker, Oliver A; user@spark.apache.org Subject: Re: Spark as a Library If you want to run the computation on just one machine (using Spark's local mode), it can probably run in a container. Otherwise you can create a SparkContext there and connect it to a cluster outside. Note that I haven't tried this though, so the security policies of the container might be too restrictive. In that case you'd have to run the app outside and expose an RPC interface between them. Matei On September 16, 2014 at 8:17:08 AM, Ruebenacker, Oliver A (oliver.ruebenac...@altisource.com<mailto:oliver.ruebenac...@altisource.com>) wrote: Hello, Suppose I want to use Spark from an application that I already submit to run in another container (e.g. Tomcat). Is this at all possible? Or do I have to split the app into two components, and submit one to Spark and one to the other container? In that case, what is the preferred way for the two components to communicate with each other? Thanks! Best, Oliver Oliver Ruebenacker | Solutions Architect Altisource™ 290 Congress St, 7th Floor | Boston, Massachusetts 02210 P: (617) 728-5582 | ext: 275585 oliver.ruebenac...@altisource.com<mailto:oliver.ruebenac...@altisource.com> | www.Altisource.com *********************************************************************************************************************** This email message and any attachments are intended solely for the use of the addressee. If you are not the intended recipient, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply email and immediately delete this message from your system. This message and any attachments may contain information that is confidential, privileged or exempt from disclosure. Delivery of this message to any person other than the intended recipient is not intended to waive any right or privilege. Message transmission is not guaranteed to be secure or free of software viruses. *********************************************************************************************************************** *********************************************************************************************************************** This email message and any attachments are intended solely for the use of the addressee. If you are not the intended recipient, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply email and immediately delete this message from your system. This message and any attachments may contain information that is confidential, privileged or exempt from disclosure. Delivery of this message to any person other than the intended recipient is not intended to waive any right or privilege. Message transmission is not guaranteed to be secure or free of software viruses. ***********************************************************************************************************************
