Re: How to run multiple Spark jobs as a workflow that takes input from a Streaming job in Oozie
Flume could be interesting for you. > On 19 Dec 2015, at 00:27, SRK wrote: > > Hi, > > How to run multiple Spark jobs that takes Spark Streaming data as the > input as a workflow in Oozie? We have to run our Streaming job first and > then have a workflow of Spark Batch jobs to process the data. > > Any suggestions on this would be of great help. > > Thanks! > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-run-multiple-Spark-jobs-as-a-workflow-that-takes-input-from-a-Streaming-job-in-Oozie-tp25739.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: How to run multiple Spark jobs as a workflow that takes input from a Streaming job in Oozie
Use Spark job server https://github.com/spark-jobserver/spark-jobserver Additional: 1. You can also write your on job server with spray (a Scala REST framework). 2. Create Thrift server and pass states of each job states (Thrift Object ) between your different Jobs. - Software Developer Sigmoid (SigmoidAnalytics), India -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-run-multiple-Spark-jobs-as-a-workflow-that-takes-input-from-a-Streaming-job-in-Oozie-tp25739p25745.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
How to run multiple Spark jobs as a workflow that takes input from a Streaming job in Oozie
Hi, How to run multiple Spark jobs that takes Spark Streaming data as the input as a workflow in Oozie? We have to run our Streaming job first and then have a workflow of Spark Batch jobs to process the data. Any suggestions on this would be of great help. Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-run-multiple-Spark-jobs-as-a-workflow-that-takes-input-from-a-Streaming-job-in-Oozie-tp25739.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
