Thanks, will check too, however : just want to use Spark core RDD and standard data sources.
Le lun. 29 févr. 2016 à 14:54, Alex Dzhagriev <dzh...@gmail.com> a écrit : > Hi Moshir, > > Regarding the streaming, you can take a look at the spark streaming, the > micro-batching framework. If it satisfies your needs it has a bunch of > integrations. Thus, the source for the jobs could be Kafka, Flume or Akka. > > Cheers, Alex. > > On Mon, Feb 29, 2016 at 2:48 PM, moshir mikael <moshir.mik...@gmail.com> > wrote: > >> Hi Alex, >> thanks for the link. Will check it. >> Does someone know of a more streamlined approach ? >> >> >> >> >> Le lun. 29 févr. 2016 à 10:28, Alex Dzhagriev <dzh...@gmail.com> a >> écrit : >> >>> Hi Moshir, >>> >>> I think you can use the rest api provided with Spark: >>> https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala >>> >>> Unfortunately, I haven't find any documentation, but it looks fine. >>> Thanks, Alex. >>> >>> On Sun, Feb 28, 2016 at 3:25 PM, mms <moshir.mik...@gmail.com> wrote: >>> >>>> Hi, I cannot find a simple example showing how a typical application >>>> can 'connect' to a remote spark cluster and interact with it. Let's say I >>>> have a Python web application hosted somewhere *outside *a spark >>>> cluster, with just python installed on it. How can I talk to Spark without >>>> using a notebook, or using ssh to connect to a cluster master node ? I know >>>> of spark-submit and spark-shell, however forking a process on a remote host >>>> to execute a shell script seems like a lot of effort What are the >>>> recommended ways to connect and query Spark from a remote client ? Thanks >>>> Thx ! >>>> ------------------------------ >>>> View this message in context: Spark Integration Patterns >>>> <http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Integration-Patterns-tp26354.html> >>>> Sent from the Apache Spark User List mailing list archive >>>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >>>> >>> >>> >
