[
https://issues.apache.org/jira/browse/MADLIB-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16847072#comment-16847072
]
Frank McQuillan commented on MADLIB-1348:
-----------------------------------------
There are 2 possible sources for initial weights:
1) a previous madlib run (model weights will be in bytea in the model output
table)
2) numpy arrays
The model architecture table is currently looks like:
{code}
Output table
The output table contains the following columns:
model_id SERIAL PRIMARY KEY. Model ID.
model_arch JSON. JSON blob of the model architecture.
model_weights DOUBLE PRECISION[]. Weights of the model which may be use for
initialization.
__internal_madlib_id__ TEXT. Unique id for model arch. This is an id used
internally be MADlib.
{code}
so this will not support #1 very easily.
I wonder if we need to add a new column:
{code}
model_weights_bytea BYTEA. Weights of the model in PostgreSQL bytea format
which may be use for initialization.
{code}
and model weights can be in 1 format or the other, i.e., normally only 1 of the
2 columns will be normally populated depending on the source of the weights.
If for some reason both are, then we use bytea by priority rule.
> Weight initialization/transfer learning madlib_keras_fit()
> ----------------------------------------------------------
>
> Key: MADLIB-1348
> URL: https://issues.apache.org/jira/browse/MADLIB-1348
> Project: Apache MADlib
> Issue Type: Improvement
> Components: Deep Learning
> Reporter: Nikhil
> Priority: Major
> Fix For: v1.16
>
>
> Context
> Many deep neural nets are not trained from scratch, but rather initialized
> from weights generated by training related data sets using the same model
> architecture (particularly true for CNN).
> Story
> As a data scientist,
> I want to start training a model based on weights that I have,
> so that I don't have to start from scratch.
> * e.g, use weights from one dataset (e.g., VGG-16 on Imagenet) as starting
> point to training VGG-16 model on my data.
> Details
> 1. add support for optional param to load weights
> 2. add “name” , “description” to model arch table
> Interface
> {code}
> load_keras_model(
> keras_model_arch_table,
> model_arch,
> model_weights, -- OPTIONAL
> name, -- OPTIONAL
> description -- OPTIONAL
> )
> {code}
> Acceptance
> 1. Take a trained model with a known accuracy and load into the model arch
> table (can be simple).
> 2. Use it as input to training with fit() on the same data set it was trained
> on. Since it has already converged, it should show the same accuracy on the
> 1st or 2nd iteration as before.
> 3. Test load from keras library [2]. Pick any model, get the weights and
> test load into model arch table. Test for 1 or 2 iterations on any dataset
> to check that it runs.
> Reference
> [1] VGG16 and other pre-trained weights for Imagenet are built into Keras
> https://keras.io/getting-started/faq/#how-can-i-use-pre-trained-models-in-keras
> [2] http://cs231n.github.io/transfer-learning/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
