fmcquillan99 edited a comment on pull request #511:
URL: https://github.com/apache/madlib/pull/511#issuecomment-679264964
re-testing
(3)
adding `loss_type` to info table:
```
madlib=# \d iris_multi_model_info
Table "public.iris_multi_model_info"
Column | Type | Modifiers
--------------------------+--------------------+-----------
mst_key | integer | not null
model_id | integer |
compile_params | text |
fit_params | text |
model_type | text |
model_size | double precision |
metrics_elapsed_time | double precision[] |
metrics_type | text[] |
loss_type | text |
training_metrics_final | double precision |
training_loss_final | double precision |
training_metrics | double precision[] |
training_loss | double precision[] |
validation_metrics_final | double precision |
validation_loss_final | double precision |
validation_metrics | double precision[] |
validation_loss | double precision[] |
```
```
SELECT * FROM iris_multi_model_info ORDER BY training_metrics_final DESC,
training_loss_final;
mst_key | model_id | compile_params
| fit_params | model_type | model_size | metrics_elapsed_time
| metrics_type | loss_type | training_metrics_final | training_loss_final |
training_metrics | training_loss | validation_metrics_final |
validation_loss_final | validation_metrics | validation_loss
---------+----------+-----------------------------------------------------------+-----------------------+--------------+--------------+----------------------+-----------------+-----------+------------------------+---------------------+---------------------+---------------------+--------------------------+-----------------------+--------------------+-----------------
5 | 1 |
loss=rmse,optimizer='Adam(lr=0.001)',metrics=['accuracy'] |
batch_size=4,epochs=1 | madlib_keras | 0.7900390625 | {36.2402780056} |
{accuracy} | rmse | 0.625 | 0.384403049946 |
{0.625} | {0.384403049945831} | |
| |
8 | 2 | loss=rmse,optimizer='Adam()',metrics=['squared_error']
| batch_size=8,epochs=1 | madlib_keras | 1.2197265625 | {35.9148938655853}
| {squared_error} | rmse | 0.448450267315 | 0.551081478596 |
{0.448450267314911} | {0.551081478595734} | |
| |
11 | 2 |
loss=rmse,optimizer='Adam(lr=0.001)',metrics=['accuracy'] |
batch_size=4,epochs=1 | madlib_keras | 1.2197265625 | {34.7360639572144} |
{accuracy} | rmse | 0.34999999404 | 0.373422443867 |
{0.349999994039536} | {0.37342244386673} | |
| |
12 | 2 |
loss=rmse,optimizer='Adam(lr=0.001)',metrics=['accuracy'] |
batch_size=8,epochs=1 | madlib_keras | 1.2197265625 | {35.4119808673859} |
{accuracy} | rmse | 0.34999999404 | 0.462386697531 |
{0.349999994039536} | {0.462386697530746} | |
| |
4 | 1 | loss=rmse, optimizer='Adam(lr=0.01)',metrics=['rmse']
| batch_size=8,epochs=1 | madlib_keras | 0.7900390625 | {35.0772519111633}
| {rmse} | rmse | 0.32794713974 | 0.32794713974 |
{0.32794713973999} | {0.32794713973999} | |
| |
6 | 1 |
loss=rmse,optimizer='Adam(lr=0.001)',metrics=['accuracy'] |
batch_size=8,epochs=1 | madlib_keras | 0.7900390625 | {36.3988900184631} |
{accuracy} | rmse | 0.324999988079 | 0.471819698811 |
{0.324999988079071} | {0.471819698810577} | |
| |
9 | 2 | loss=rmse, optimizer='Adam(lr=0.01)',metrics=['rmse']
| batch_size=4,epochs=1 | madlib_keras | 1.2197265625 | {35.5919768810272}
| {rmse} | rmse | 0.260808467865 | 0.260808467865 |
{0.26080846786499} | {0.26080846786499} | |
| |
10 | 2 | loss=rmse, optimizer='Adam(lr=0.01)',metrics=['rmse']
| batch_size=8,epochs=1 | madlib_keras | 1.2197265625 | {34.921012878418}
| {rmse} | rmse | 0.231591135263 | 0.231591135263 |
{0.231591135263443} | {0.231591135263443} | |
| |
7 | 2 | loss=rmse,optimizer='Adam()',metrics=['squared_error']
| batch_size=4,epochs=1 | madlib_keras | 1.2197265625 | {36.0883269309998}
| {squared_error} | rmse | 0.221946805716 | 0.468155503273 |
{0.221946805715561} | {0.46815550327301} | |
| |
3 | 1 | loss=rmse, optimizer='Adam(lr=0.01)',metrics=['rmse']
| batch_size=4,epochs=1 | madlib_keras | 0.7900390625 | {35.2352449893951}
| {rmse} | rmse | 0.181505993009 | 0.181505993009 |
{0.181505993008614} | {0.181505993008614} | |
| |
2 | 1 | loss=rmse,optimizer='Adam()',metrics=['squared_error']
| batch_size=8,epochs=1 | madlib_keras | 0.7900390625 | {34.5514559745789}
| {squared_error} | rmse | 0.173068851233 | 0.394461244345 |
{0.173068851232529} | {0.394461244344711} | |
| |
1 | 1 | loss=rmse,optimizer='Adam()',metrics=['squared_error']
| batch_size=4,epochs=1 | madlib_keras | 0.7900390625 | {35.7443828582764}
| {squared_error} | rmse | 0.156523838639 | 0.382581591606 |
{0.156523838639259} | {0.38258159160614} | |
| |
```
OK
(4)
```DROP TABLE IF EXISTS iris_model_cl, iris_model_cl_summary;
SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table
'iris_model_cl', -- model output
table
'model_arch_library', -- model arch table
1, -- model arch id
$$ loss=rmse, optimizer='adam',
metrics=['rmse'] $$, -- compile_params
$$ batch_size=5, epochs=3 $$, -- fit_params
3, -- num_iteration
NULL, -- use_gpus,
NULL, -- validation_table,
NULL, -- metrics_compute_frequency,
NULL, -- warm_start,
NULL, -- name
NULL, -- description
'test_custom_function_table' -- object table
);
```
produces
```
INFO:
Time for training in iteration 1: 2.89410400391 sec
CONTEXT: PL/Python function "madlib_keras_fit"
INFO:
Time for training in iteration 2: 0.0521700382233 sec
CONTEXT: PL/Python function "madlib_keras_fit"
INFO:
Time for training in iteration 3: 0.0579919815063 sec
DETAIL:
Time for evaluating training dataset in iteration 3: 0.225339174271 sec
Training set metric after iteration 3: 0.375931799412
Training set loss after iteration 3: 0.375931799412
CONTEXT: PL/Python function "madlib_keras_fit"
-[ RECORD 1 ]----+-
madlib_keras_fit |
Time: 6035.621 ms
```
OK
(5)
```
$$ loss=rmse, optimizer='adam', metrics=['squared_error'] $$, --
compile_params
$$ loss='rmse', optimizer='adam', metrics=['squared_error'] $$, --
compile_params
```
both work
```
$$ loss=rmse, optimizer='adam', metrics=[squared_error] $$, --
compile_params
```
does not but that is fine. Would required too many changes to parsers.
OK
so overall LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
