fmcquillan99 edited a comment on issue #461: DL: Add optional parameters for multi model training URL: https://github.com/apache/madlib/pull/461#issuecomment-561827179 LGTM from functional perspective (1) train for 3 iterations ``` DROP TABLE IF EXISTS iris_multi_model, iris_multi_model_summary, iris_multi_model_info; SELECT madlib.madlib_keras_fit_multiple_model('iris_train_packed', -- source_table 'iris_multi_model', -- model_output_table 'mst_table', -- model_selection_table 3, -- num_iterations 0, -- use gpus 'iris_test_packed', -- validation dataset 1, -- metrics compute frequency FALSE, -- warm start 'Sophie L.', -- name 'Simple MLP for iris dataset' -- description ); ``` Now train for 3 more iterations with warm start, then do it again with warm start, for a total of 9 iterations altogether ``` SELECT madlib.madlib_keras_fit_multiple_model('iris_train_packed', -- source_table 'iris_multi_model', -- model_output_table 'mst_table', -- model_selection_table 3, -- num_iterations 0, -- use gpus 'iris_test_packed', -- validation dataset 1, -- metrics compute frequency TRUE, -- warm start 'Sophie L.', -- name 'Simple MLP for iris dataset' -- description ); ``` Look at `mst_key=3` for 3 succesive runs: ``` training_loss [0.57945317029953, 0.363865971565247, 0.285590827465057] [0.254694432020187, 0.19344100356102, 0.146936997771263] [0.132369637489319, 0.191817075014114, 0.157660290598869] validation_loss [0.67350971698761, 0.443352043628693, 0.309820145368576] [0.261916130781174, 0.201081365346909, 0.127666547894478] [0.0899263918399811, 0.204310938715935, 0.0784771740436554] ``` Look at `mst_key=4` for 3 succesive runs: ``` training_loss [0.994381964206696, 0.746099174022675, 0.608452022075653] [0.51947021484375, 0.606200277805328, 0.340667217969894] [0.439217150211334, 0.332746595144272, 0.279199987649918] validation_loss [1.0290435552597, 0.814084649085999, 0.709287881851196] [0.614403426647186, 0.713598251342773, 0.380535960197449] [0.498434334993362, 0.354164630174637, 0.273105055093765] ``` This looks OK in principle since these losses seem to continue from the previous run. (2) drop summary table and run warm start ``` InternalError: (psycopg2.InternalError) plpy.Error: madlib_keras_fit_multiple error: Input table 'iris_multi_model_summary' does not exist. (plpython.c:5038) CONTEXT: Traceback (most recent call last): PL/Python function "madlib_keras_fit_multiple_model", line 21, in <module> fit_obj = madlib_keras_fit_multiple_model.FitMultipleModel(**globals()) PL/Python function "madlib_keras_fit_multiple_model", line 42, in wrapper PL/Python function "madlib_keras_fit_multiple_model", line 119, in __init__ PL/Python function "madlib_keras_fit_multiple_model", line 410, in __init__ PL/Python function "madlib_keras_fit_multiple_model", line 253, in __init__ PL/Python function "madlib_keras_fit_multiple_model", line 280, in _validate_common_args PL/Python function "madlib_keras_fit_multiple_model", line 674, in input_tbl_valid PL/Python function "madlib_keras_fit_multiple_model" [SQL: "SELECT madlib.madlib_keras_fit_multiple_model('iris_train_packed', -- source_table\n 'iris_multi_model', -- model_output_table\n 'mst_table', -- model_selection_table\n 3, -- num_iterations\n 0, -- use gpus\n 'iris_test_packed', -- validation dataset\n 1, -- metrics compute frequency\n TRUE, -- warm start\n 'Sophie L.', -- name\n 'Simple MLP for iris dataset' -- description\n );"] ``` OK error message (3) drop model table and run warm start ``` InternalError: (psycopg2.InternalError) plpy.Error: madlib_keras_fit_multiple error: Input table 'iris_multi_model' does not exist. (plpython.c:5038) CONTEXT: Traceback (most recent call last): PL/Python function "madlib_keras_fit_multiple_model", line 21, in <module> fit_obj = madlib_keras_fit_multiple_model.FitMultipleModel(**globals()) PL/Python function "madlib_keras_fit_multiple_model", line 42, in wrapper PL/Python function "madlib_keras_fit_multiple_model", line 119, in __init__ PL/Python function "madlib_keras_fit_multiple_model", line 410, in __init__ PL/Python function "madlib_keras_fit_multiple_model", line 253, in __init__ PL/Python function "madlib_keras_fit_multiple_model", line 279, in _validate_common_args PL/Python function "madlib_keras_fit_multiple_model", line 674, in input_tbl_valid PL/Python function "madlib_keras_fit_multiple_model" [SQL: "SELECT madlib.madlib_keras_fit_multiple_model('iris_train_packed', -- source_table\n 'iris_multi_model', -- model_output_table\n 'mst_table', -- model_selection_table\n 3, -- num_iterations\n 0, -- use gpus\n 'iris_test_packed', -- validation dataset\n 1, -- metrics compute frequency\n TRUE, -- warm start\n 'Sophie L.', -- name\n 'Simple MLP for iris dataset' -- description\n );"] ``` OK error message (4) try diff metrics compute frequency ``` DROP TABLE IF EXISTS iris_multi_model, iris_multi_model_summary, iris_multi_model_info; SELECT madlib.madlib_keras_fit_multiple_model('iris_train_packed', -- source_table 'iris_multi_model', -- model_output_table 'mst_table', -- model_selection_table 5, -- num_iterations 0, -- use gpus 'iris_test_packed', -- validation dataset 3, -- metrics compute frequency FALSE, -- warm start 'Sophie L.', -- name 'Simple MLP for iris dataset' -- description ); ``` Look at summary table ``` SELECT * FROM iris_multi_model_summary; -[ RECORD 1 ]-------------+--------------------------------------------- source_table | iris_train_packed validation_table | iris_test_packed model | iris_multi_model model_info | iris_multi_model_info dependent_varname | class_text independent_varname | attributes model_arch_table | model_arch_library num_iterations | 5 metrics_compute_frequency | 3 warm_start | f name | Sophie L. description | Simple MLP for iris dataset start_training_time | 2019-12-04 20:28:55.000868 end_training_time | 2019-12-04 20:29:50.155568 madlib_version | 1.17-dev num_classes | 3 class_values | {Iris-setosa,Iris-versicolor,Iris-virginica} dependent_vartype | character varying normalizing_const | 1 metrics_iters | {3,5} ``` OK on the {3,5}
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
