Harish Butani created HIVE-7940:
-----------------------------------
Summary: Expose Machine Learning functions and Model application
in Hive
Key: HIVE-7940
URL: https://issues.apache.org/jira/browse/HIVE-7940
Project: Hive
Issue Type: New Feature
Reporter: Harish Butani
*Machine Learning functions*
# [HiveMall|https://github.com/myui/hivemall] has demonstrated how to do
machine learning in Hive. It has an extensive set of functions; it shows a way
through UDTFs and Amplify technique to do iterative computations. There is a
lot of interest in the Hive User community to use HiveMall.
# Other possible ways to expose machine learning functionality:
#* via Script Operator(Or Table Functions) that call out to a Machine Learning
service like [Oxdata|https://github.com/0xdata/h2o]. In this scheme the
service's nodes would communicate outside of hive, process the data in multiple
iterations and then return the result back into the hive pipeline.
#* At the language level, provide an iteration mechanism in Hive: this has more
general applications: to express Recursive CTEs and also to express Graph
Algorithms.
*Model Application*
Even when Regression/Classification models are build in other tools we should
provide a way to evaluate these models against the entire dataset residing in
Hive. These can be exposed as UDFs in Hive. A possible route could be a generic
PMML based module, for e.g. [JPMML-Hive|https://github.com/jpmml/jpmml-hive].
Or we should provide integration for specific libraries: Spark MLLib, R and
Python (SciPy/NumPy) seem the most popular toolkits.
The *goal* would be to provide Machine Learning functionality as a Feature of
Hive like [MadLib|http://madlib.net/] on Postgres, Pivotal, Impala etc.
Capturing this high level requirement in this jira.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
