Hi all,
Last week a bunch of us attended the Oozie Meetup prior to the Hadoop
Summit at the San Jose Convention Center. It was really nice to be able
to interact face-to-face with other Oozie contributors and users.
At the end of the meetup we went over a few items that gathered
consensus. I'd like to get the discussion going again on these so we can
get to more precise requirements and use cases, and maybe inspire a few
of us to get crackin'.
I'll start first, to get the ball rolling.
- one thing that was mentioned a lot is the ability to visually create
workflows and coordinators instead of having to write XML. Although
I personally don't rank this to be very high priority -- my use
case is less around ad-hoc workflows -- I do see a huge value in
this, if only to help spread Oozie.
- directly related to visualization, monitoring of the running
workflows is definitely at the top of my list. Being able to
visualize the DAG of a workflow, with the completed actions in
green, in progress action in yellow, failed action(s) in red would
be a huge help for real-time monitoring and debugging.
I've also written a command-line Python utility that helps managing a
set of coordinators that need to run in various Hadoop clusters,
exposing them as "applications" to the release manager. I'm not sure if
this has any value for Oozie, but if anybody's interested I might be
able to share the code for that.
/Maxime
--
{ name: 'Maxime Petazzoni', title: 'Sr. Platform Engineer',
company: { name: 'Turn, Inc', url: 'http://www.turn.com' } }
