Nikhil Kak created MADLIB-1462:
----------------------------------
Summary: DL - Fit multiple does not print timing for validation
evaluate
Key: MADLIB-1462
URL: https://issues.apache.org/jira/browse/MADLIB-1462
Project: Apache MADlib
Issue Type: Improvement
Components: Deep Learning
Reporter: Nikhil Kak
Currently when running fit_multiple with validation dataset, we don't print the
timing for the validation runs
{code}
select madlib.madlib_keras_fit_multiple_model('cifar10_train_batched',
'cifar10_out', 'cifar10_mst_table', 100, TRUE, 'cifar10_train_batched', 1);
INFO:
Time for training in iteration 1: 33.6217501163 sec
DETAIL:
Training set after iteration 1:
mst_key=12: metric=0.260340005159, loss=2.13081121445
...
mst_key=2: metric=0.164859995246, loss=2.25495767593
Validation set after iteration 1:
mst_key=12: metric=0.260340005159, loss=2.13081121445
...
mst_key=2: metric=0.164859995246, loss=2.25495767593
CONTEXT: PL/Python function "madlib_keras_fit_multiple_model"
INFO:
Time for training in iteration 2: 24.7699511051 sec
DETAIL:
....
{code}
We should print the time it took to run validation evaluate for both training
and validation dataset
If the user specifies only the training dataset, then we will add the following
to the existing output
1. The cumulative time it took for all the msts to run eval for the training
dataset for that iteration
{code}
select
madlib.madlib_keras_fit_multiple_model('iris_data_packed','iris_multiple_model','mst_table_4row',2,
FALSE,NULL,1);
INFO:
Time for training in iteration 1: 2.24381709099 sec
DETAIL:
Training set after iteration 1:
mst_key=2: metric=0.333333343267, loss=1.33550834656
mst_key=1: metric=0.333333343267, loss=1.12043237686
mst_key=4: metric=0.333333343267, loss=3.90859818459
mst_key=3: metric=0.333333343267, loss=4.37875080109
Time for evaluating training dataset in iteration 1: 0.652065515518
CONTEXT: PL/Python function "madlib_keras_fit_multiple_model"
INFO:
Time for training in iteration 2: 2.32056617737 sec
DETAIL:
Training set after iteration 2:
mst_key=2: metric=0.666666686535, loss=1.14192306995
mst_key=1: metric=0.666666686535, loss=0.917088747025
mst_key=4: metric=0.340000003576, loss=2.98958563805
mst_key=3: metric=0.333333343267, loss=3.86314368248
Time for evaluating training dataset in iteration 2: 0.679529428482
{code}
If the user specifies a validation dataset, then we will add the following to
the existing output
1. The cumulative time it took for all the msts to run eval for the training
dataset for that iteration
1. The cumulative time it took for all the msts to run eval for the validation
dataset for that iteration
{code}
select
madlib.madlib_keras_fit_multiple_model('iris_data_packed','iris_multiple_model','mst_table_4row',2,
FALSE,'iris_data_packed',1);
INFO:
Time for training in iteration 1: 4.27021813393 sec
DETAIL:
Training set after iteration 1:
mst_key=2: metric=0.333333343267, loss=1.39633440971
mst_key=1: metric=0.333333343267, loss=1.04632723331
mst_key=4: metric=0.333333343267, loss=3.96611213684
mst_key=3: metric=0.333333343267, loss=4.38052940369
Time for evaluating training dataset in iteration 1: 0.649274587631
Validation set after iteration 1:
mst_key=2: metric=0.333333343267, loss=1.39633440971
mst_key=1: metric=0.333333343267, loss=1.04632723331
mst_key=4: metric=0.333333343267, loss=3.96611213684
mst_key=3: metric=0.333333343267, loss=4.38052940369
Time for evaluating validation dataset in iteration 1: 0.75797867775
CONTEXT: PL/Python function "madlib_keras_fit_multiple_model"
INFO:
Time for training in iteration 2: 2.1767308712 sec
DETAIL:
Training set after iteration 2:
mst_key=2: metric=0.666666686535, loss=1.10426521301
mst_key=1: metric=0.666666686535, loss=1.02108848095
mst_key=4: metric=0.333333343267, loss=3.10222005844
mst_key=3: metric=0.333333343267, loss=3.85620188713
Time for evaluating training dataset in iteration 2: 0.784633874893
Validation set after iteration 2:
mst_key=2: metric=0.666666686535, loss=1.10426521301
mst_key=1: metric=0.666666686535, loss=1.02108848095
mst_key=4: metric=0.333333343267, loss=3.10222005844
mst_key=3: metric=0.333333343267, loss=3.85620188713
Time for evaluating validation dataset in iteration 2: 0.639858007431
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
