[
https://issues.apache.org/jira/browse/HAMA-770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13724503#comment-13724503
]
Yexi Jiang commented on HAMA-770:
---------------------------------
I have finished the training (non-distributed version) algorithm for neural
network and the wrapper for linear regression and logistic regression.
I also tested these models on synthetic and real datasets.
A 3-layer neural network can achieve around 95% precision on the diabetes
dataset (a dataset for 2-class classification, available in Weka). The logistic
regression can achieve 80% precision on the same dataset.
As for linear regression, I tested it on the death rate dataset available at
http://people.sc.fsu.edu/~jburkardt/datasets/regression/regression.html
(x28.txt), and the mean percentage error around 7%.
Next step, I will implement the BSP training algorithm based on the
non-distributed version.
BTW, these model can be used in non-distributed way and can be trained
incrementally using the trainOnline method.
The snapshot version is available at https://github.com/yxjiang/hama/tree/ann.
> Use a unified model to represent linear regression, logistic regression, MLP,
> autoencoder, and deepNets
> -------------------------------------------------------------------------------------------------------
>
> Key: HAMA-770
> URL: https://issues.apache.org/jira/browse/HAMA-770
> Project: Hama
> Issue Type: Improvement
> Components: machine learning
> Reporter: Yexi Jiang
> Assignee: Yexi Jiang
>
> In principle, linear regression, logistic regression, MLP, autoencoder, and
> deepNets can be represented by a generic neural network model. Using a
> generic model and making the concrete models derive it can increase the
> reusability of the code.
> More concretely:
> Linear regression is a two level neural network (one input layer and one
> output layer) by setting the squashing function as identity function f( x ) =
> x, and cost function as squared error.
> Logistic regression is similar to linear regression, except that the
> squashing function is set as sigmoid and cost function is set as cross
> entropy.
> MLP is a neural nets with at least 2 layers of neurons. The squashing
> function can be sigmoid, tanh (may be more) and cost function can be cross
> entropy, squared error (may be more).
> (sparse) autoencoder can be used for dimensional reduction (nonlinear) and
> anomaly detection. Also, it can be used as the building block of deep nets.
> Generally it is a three layer neural networks, where the size of input layer
> is the same as output layer, and the size of hidden layer is typically less
> than that of the input/output layer. Its cost function is squared error + KL
> divergence.
> deepNets is used for deep learning, a simple architecture is to stack several
> autoencoder together.
> The steps:
> 1. Create package 'org.apache.hama.ml.ann'. This package is used to put the
> abstract models and trainer (an abstract trainer that defines the interfaces
> for the concrete model trainer). The concrete implementation of training is
> better to be detached from the model, so that new training method can easily
> be added in the future.
> 2. Move PerceptronTrainer from 'org.apache.hama.ml.perceptron' to
> 'org.apache.hama.ml.ann' and rename it to NeuralNetworkTrainer. The API
> defined in this class is generic enough to be reused by all the neural
> network trainers.
> 3. Add abstract NeuralNetwork, AbstractLayeredNeuralNetwork,
> SmallLayeredNeuralNetwork in to above package.
> a. NeuralNetwork defines the common behaviors of all neural network based
> models.
> b. AbstractLayeredNeuralNetwork defines the common behaviors of all layered
> neural network based model.
> c. SmallLayeredNeuralNetwork defines the common behaviors of all layered
> neural network whose topology can be loaded into one machine.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
