Qiang, Did you look at Oozie? We use oozie to run spark jobs in production.
> On Feb 28, 2015, at 2:45 PM, Qiang Cao <caoqiang...@gmail.com> wrote: > > Hi Everyone, > > We need to deal with workflows on Spark. In our scenario, each workflow > consists of multiple processing steps. Among different steps, there could be > dependencies. I'm wondering if there are tools available that can help us > schedule and manage workflows on Spark. I'm looking for something like pig on > Hadoop, but it should fully function on Spark. > > Any suggestion? > > Thanks in advance! > > Qiang
