[jira] Commented: (HIVE-1541) More general dataflow execution backend
[ https://issues.apache.org/jira/browse/HIVE-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12898812#action_12898812 ] Venkatesh S commented on HIVE-1541: --- Oozie should be a good candidate as well. More general dataflow execution backend --- Key: HIVE-1541 URL: https://issues.apache.org/jira/browse/HIVE-1541 Project: Hadoop Hive Issue Type: New Feature Reporter: Jeff Hammerbacher With the recent open source release of Mesos (http://github.com/mesos/mesos), experimentation at the query execution layer has become more feasible. Inspired by more general-purpose dataflow systems like Volcano, Dryad, and Dremel, it would be interesting to explore a more general-purpose dataflow execution system for Hive queries. One potential backend is the Hyracks project from UCI: http://code.google.com/p/hyracks. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1541) More general dataflow execution backend
[ https://issues.apache.org/jira/browse/HIVE-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12899290#action_12899290 ] Jeff Hammerbacher commented on HIVE-1541: - Hey Venkatesh, HIVE-1107 is aimed at getting Hive and Pig to express their sequence of MapReduce jobs as an Oozie workflow. For this JIRA, I meant an entirely different initialization routine and set of physical operators, similar to those used by an MPP relational database or Dremel, Whether Oozie is used to describe the workflow tying together these new physical operators is less of a concern to me. Thanks, Jeff More general dataflow execution backend --- Key: HIVE-1541 URL: https://issues.apache.org/jira/browse/HIVE-1541 Project: Hadoop Hive Issue Type: New Feature Reporter: Jeff Hammerbacher With the recent open source release of Mesos (http://github.com/mesos/mesos), experimentation at the query execution layer has become more feasible. Inspired by more general-purpose dataflow systems like Volcano, Dryad, and Dremel, it would be interesting to explore a more general-purpose dataflow execution system for Hive queries. One potential backend is the Hyracks project from UCI: http://code.google.com/p/hyracks. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1541) More general dataflow execution backend
[ https://issues.apache.org/jira/browse/HIVE-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12898654#action_12898654 ] Jeff Hammerbacher commented on HIVE-1541: - In particular, it would be nice to avoid the startup overhead of Hadoop MapReduce with this backend. More general dataflow execution backend --- Key: HIVE-1541 URL: https://issues.apache.org/jira/browse/HIVE-1541 Project: Hadoop Hive Issue Type: New Feature Reporter: Jeff Hammerbacher With the recent open source release of Mesos (http://github.com/mesos/mesos), experimentation at the query execution layer has become more feasible. Inspired by more general-purpose dataflow systems like Volcano, Dryad, and Dremel, it would be interesting to explore a more general-purpose dataflow execution system for Hive queries. One potential backend is the Hyracks project from UCI: http://code.google.com/p/hyracks. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
