Thanks, Ashish! Is Oozie integrated with Spark? I knew it can accommodate some Hadoop jobs.
On Sat, Feb 28, 2015 at 6:07 PM, Ashish Nigam <ashnigamt...@gmail.com> wrote: > Qiang, > Did you look at Oozie? > We use oozie to run spark jobs in production. > > > On Feb 28, 2015, at 2:45 PM, Qiang Cao <caoqiang...@gmail.com> wrote: > > Hi Everyone, > > We need to deal with workflows on Spark. In our scenario, each workflow > consists of multiple processing steps. Among different steps, there could > be dependencies. I'm wondering if there are tools available that can > help us schedule and manage workflows on Spark. I'm looking for something > like pig on Hadoop, but it should fully function on Spark. > > Any suggestion? > > Thanks in advance! > > Qiang > > >
