Github user helenahm commented on the issue:
https://github.com/apache/incubator-hivemall/pull/93
Just added a MaxEntMixtureWeightUDAF to aggregate weights of models
obtained on each part of data.
create temporary function aggregate_classifiers as
'hivemall.smile.tools.MaxEntMixtureWeightUDAF';
tested it on EMR only:
add jar hivemall-core-0.4.2-rc.2-maxent-with-dependencies.jar;
add jar opennlp-maxent-3.0.0.jar;
source define-all.hive;
create temporary function train_maxent_classifier as
'hivemall.smile.classification.MaxEntUDTF';
create temporary function predict_maxent_classifier as
'hivemall.smile.tools.MaxEntPredictUDF';
create temporary function aggregate_classifiers as
'hivemall.smile.tools.MaxEntMixtureWeightUDAF';
select aggregate_classifiers(model) from tmodel5;
Where tmodel5 contains 5 lines of same model.
I will do more testing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
