, or leverage caching or temp table, going into another
Oozie action. You could either save output to file or put all Spark
processing into one Oozie action.
--- Original Message ---
From: Mayur Rustagi mayur.rust...@gmail.com
Sent: February 28, 2015 7:07 PM
To: Qiang Cao caoqiang...@gmail.com
Cc: Ted
Thanks, Ashish! Is Oozie integrated with Spark? I knew it can accommodate
some Hadoop jobs.
On Sat, Feb 28, 2015 at 6:07 PM, Ashish Nigam ashnigamt...@gmail.com
wrote:
Qiang,
Did you look at Oozie?
We use oozie to run spark jobs in production.
On Feb 28, 2015, at 2:45 PM, Qiang Cao
Hi Everyone,
We need to deal with workflows on Spark. In our scenario, each workflow
consists of multiple processing steps. Among different steps, there could
be dependencies. I'm wondering if there are tools available that can help
us schedule and manage workflows on Spark. I'm looking for
this link to get the idea for my implementation -
http://mail-archives.apache.org/mod_mbox/oozie-user/201404.mbox/%3CCAHCsPn-0Grq1rSXrAZu35yy_i4T=fvovdox2ugpcuhkwmjp...@mail.gmail.com%3E
On Feb 28, 2015, at 3:25 PM, Qiang Cao caoqiang...@gmail.com wrote:
Thanks, Ashish! Is Oozie integrated
28, 2015 at 6:26 PM, Qiang Cao caoqiang...@gmail.com wrote:
Thanks for the pointer, Ashish! I was also looking at Spork
https://github.com/sigmoidanalytics/spork Pig-on-Spark), but wasn't
sure if that's the right direction.
On Sat, Feb 28, 2015 at 6:36 PM, Ashish Nigam ashnigamt...@gmail.com
