[ https://issues.apache.org/jira/browse/MADLIB-1363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Frank McQuillan updated MADLIB-1363: ------------------------------------ Description: fit() INFO and CONTEXT messages (1) no validation_table, metrics_compute_frequency=0 {code} SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table 'iris_model', -- model output table 'model_arch_library', -- model arch table 1, -- model arch id $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params $$ batch_size=5, epochs=3 $$, -- fit_params 10 -- num_iterations ); INFO: Processed 60 images: Fit took 0.567000865936 sec, Total was 0.757196903229 sec (seg0 slice1 10.128.0.41:40000 pid=13317) CONTEXT: PL/Python function "fit_transition" INFO: Processed 60 images: Fit took 0.55348110199 sec, Total was 0.741441011429 sec (seg1 slice1 10.128.0.41:40001 pid=13318) CONTEXT: PL/Python function "fit_transition" INFO: Time for training in iteration 1: 2.45737695694 sec CONTEXT: PL/Python function "madlib_keras_fit" {code} change to {code} INFO: Time for training in iteration 1: 2.45737695694 sec CONTEXT: PL/Python function "madlib_keras_fit" {code} (2) no validation_table, metrics_compute_frequency!=0 {code} SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table 'iris_model', -- model output table 'model_arch_library', -- model arch table 1, -- model arch id $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params $$ batch_size=5, epochs=3 $$, -- fit_params 10, -- num_iterations 0, -- gpus per host NULL, -- validation table 1 -- metrics compute frequency ); INFO: Processed 60 images: Fit took 0.534310817719 sec, Total was 0.712550878525 sec (seg0 slice1 10.128.0.41:40000 pid=14501) CONTEXT: PL/Python function "fit_transition" INFO: Processed 60 images: Fit took 0.564456939697 sec, Total was 0.751413106918 sec (seg1 slice1 10.128.0.41:40001 pid=14502) CONTEXT: PL/Python function "fit_transition" INFO: Time for training in iteration 1: 2.28858995438 sec CONTEXT: PL/Python function "madlib_keras_fit" INFO: Time for evaluation in iteration 1: 0.188971996307 sec. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Training set metric after iteration 1: 0.649999976158. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Training set loss after iteration 1: 1.1202558279. CONTEXT: PL/Python function "madlib_keras_fit" {code} change to {code} INFO: Time for training in iteration 1: 2.28858995438 sec Time for evaluation in iteration 1: 0.188971996307 sec Training set metric after iteration 1: 0.649999976158 Training set loss after iteration 1: 1.1202558279 CONTEXT: PL/Python function "madlib_keras_fit" {code} (3) yes validation_table, metrics_compute_frequency=0 {code} SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table 'iris_model', -- model output table 'model_arch_library', -- model arch table 1, -- model arch id $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params $$ batch_size=5, epochs=3 $$, -- fit_params 10, -- num_iterations 0, -- GPUs per host 'iris_test_packed' -- validation dataset ); INFO: Processed 60 images: Fit took 0.552575826645 sec, Total was 0.734694004059 sec (seg0 slice1 10.128.0.41:40000 pid=18431) CONTEXT: PL/Python function "fit_transition" INFO: Processed 60 images: Fit took 0.549551010132 sec, Total was 0.734927892685 sec (seg1 slice1 10.128.0.41:40001 pid=18432) CONTEXT: PL/Python function "fit_transition" INFO: Time for training in iteration 1: 2.36340904236 sec CONTEXT: PL/Python function "madlib_keras_fit" {code} change to {code} INFO: Time for training in iteration 1: 2.45737695694 sec CONTEXT: PL/Python function "madlib_keras_fit" {code} (4) yes validation_table, metrics_compute_frequency=!0 {code} SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table 'iris_model', -- model output table 'model_arch_library', -- model arch table 1, -- model arch id $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params $$ batch_size=5, epochs=3 $$, -- fit_params 10, -- num_iterations 0, -- GPUs per host 'iris_test_packed', -- validation dataset 1 -- metrics compute frequency ); INFO: Processed 60 images: Fit took 0.57217502594 sec, Total was 0.817452907562 sec (seg0 slice1 10.128.0.41:40000 pid=19573) CONTEXT: PL/Python function "fit_transition" INFO: Processed 60 images: Fit took 0.554927110672 sec, Total was 0.800101041794 sec (seg1 slice1 10.128.0.41:40001 pid=19574) CONTEXT: PL/Python function "fit_transition" INFO: Time for training in iteration 1: 2.43148899078 sec CONTEXT: PL/Python function "madlib_keras_fit" INFO: Time for evaluation in iteration 1: 0.217161893845 sec. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Training set metric after iteration 1: 0.524999976158. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Training set loss after iteration 1: 0.984773635864. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Time for evaluation in iteration 1: 0.205282926559 sec. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Validation set metric after iteration 1: 0.600000023842. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Validation set loss after iteration 1: 0.940379023552. CONTEXT: PL/Python function "madlib_keras_fit" {code} change to {code} INFO: Time for training in iteration 1: 2.43148899078 sec Time for evaluating training dataset in iteration 1: 0.217161893845 sec Training set metric after iteration 1: 0.524999976158 Training set loss after iteration 1: 0.984773635864 Time for evaluating validation dataset in iteration 1: 0.205282926559 sec Validation set metric after iteration 1: 0.600000023842 Validation set loss after iteration 1: 0.940379023552 CONTEXT: PL/Python function "madlib_keras_fit" {code} Note change in wording ^^^ because there are 2 evaluation times. was: fit() INFO and CONTEXT messages (1) no validation_table, metrics_compute_frequency=0 {code} SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table 'iris_model', -- model output table 'model_arch_library', -- model arch table 1, -- model arch id $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params $$ batch_size=5, epochs=3 $$, -- fit_params 10 -- num_iterations ); INFO: Processed 60 images: Fit took 0.567000865936 sec, Total was 0.757196903229 sec (seg0 slice1 10.128.0.41:40000 pid=13317) CONTEXT: PL/Python function "fit_transition" INFO: Processed 60 images: Fit took 0.55348110199 sec, Total was 0.741441011429 sec (seg1 slice1 10.128.0.41:40001 pid=13318) CONTEXT: PL/Python function "fit_transition" INFO: Time for training in iteration 1: 2.45737695694 sec CONTEXT: PL/Python function "madlib_keras_fit" {code} change to {code} INFO: Time for training in iteration 1: 2.45737695694 sec CONTEXT: PL/Python function "madlib_keras_fit" {code} (2) no validation_table, metrics_compute_frequency!=0 {code} SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table 'iris_model', -- model output table 'model_arch_library', -- model arch table 1, -- model arch id $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params $$ batch_size=5, epochs=3 $$, -- fit_params 10, -- num_iterations 0, -- gpus per host NULL, -- validation table 1 -- metrics compute frequency ); INFO: Processed 60 images: Fit took 0.534310817719 sec, Total was 0.712550878525 sec (seg0 slice1 10.128.0.41:40000 pid=14501) CONTEXT: PL/Python function "fit_transition" INFO: Processed 60 images: Fit took 0.564456939697 sec, Total was 0.751413106918 sec (seg1 slice1 10.128.0.41:40001 pid=14502) CONTEXT: PL/Python function "fit_transition" INFO: Time for training in iteration 1: 2.28858995438 sec CONTEXT: PL/Python function "madlib_keras_fit" INFO: Time for evaluation in iteration 1: 0.188971996307 sec. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Training set metric after iteration 1: 0.649999976158. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Training set loss after iteration 1: 1.1202558279. CONTEXT: PL/Python function "madlib_keras_fit" {code} change to {code} INFO: Time for training in iteration 1: 2.28858995438 sec Time for evaluation in iteration 1: 0.188971996307 sec Training set metric after iteration 1: 0.649999976158 Training set loss after iteration 1: 1.1202558279 CONTEXT: PL/Python function "madlib_keras_fit" {code} (3) yes validation_table, metrics_compute_frequency=0 SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table 'iris_model', -- model output table 'model_arch_library', -- model arch table 1, -- model arch id $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params $$ batch_size=5, epochs=3 $$, -- fit_params 10, -- num_iterations 0, -- GPUs per host 'iris_test_packed' -- validation dataset ); INFO: Processed 60 images: Fit took 0.552575826645 sec, Total was 0.734694004059 sec (seg0 slice1 10.128.0.41:40000 pid=18431) CONTEXT: PL/Python function "fit_transition" INFO: Processed 60 images: Fit took 0.549551010132 sec, Total was 0.734927892685 sec (seg1 slice1 10.128.0.41:40001 pid=18432) CONTEXT: PL/Python function "fit_transition" INFO: Time for training in iteration 1: 2.36340904236 sec CONTEXT: PL/Python function "madlib_keras_fit" {code} change to {code} INFO: Time for training in iteration 1: 2.45737695694 sec CONTEXT: PL/Python function "madlib_keras_fit" {code} (4) yes validation_table, metrics_compute_frequency=!0 {code} SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table 'iris_model', -- model output table 'model_arch_library', -- model arch table 1, -- model arch id $$ loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'] $$, -- compile_params $$ batch_size=5, epochs=3 $$, -- fit_params 10, -- num_iterations 0, -- GPUs per host 'iris_test_packed', -- validation dataset 1 -- metrics compute frequency ); INFO: Processed 60 images: Fit took 0.57217502594 sec, Total was 0.817452907562 sec (seg0 slice1 10.128.0.41:40000 pid=19573) CONTEXT: PL/Python function "fit_transition" INFO: Processed 60 images: Fit took 0.554927110672 sec, Total was 0.800101041794 sec (seg1 slice1 10.128.0.41:40001 pid=19574) CONTEXT: PL/Python function "fit_transition" INFO: Time for training in iteration 1: 2.43148899078 sec CONTEXT: PL/Python function "madlib_keras_fit" INFO: Time for evaluation in iteration 1: 0.217161893845 sec. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Training set metric after iteration 1: 0.524999976158. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Training set loss after iteration 1: 0.984773635864. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Time for evaluation in iteration 1: 0.205282926559 sec. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Validation set metric after iteration 1: 0.600000023842. CONTEXT: PL/Python function "madlib_keras_fit" INFO: Validation set loss after iteration 1: 0.940379023552. CONTEXT: PL/Python function "madlib_keras_fit" {code} change to {code} INFO: Time for training in iteration 1: 2.43148899078 sec Time for evaluating training dataset in iteration 1: 0.217161893845 sec Training set metric after iteration 1: 0.524999976158 Training set loss after iteration 1: 0.984773635864 Time for evaluating validation dataset in iteration 1: 0.205282926559 sec Validation set metric after iteration 1: 0.600000023842 Validation set loss after iteration 1: 0.940379023552 CONTEXT: PL/Python function "madlib_keras_fit" {code} Note change in wording ^^^ because there are 2 evaluation times. > Reduce verbose output to console with fit() > ------------------------------------------- > > Key: MADLIB-1363 > URL: https://issues.apache.org/jira/browse/MADLIB-1363 > Project: Apache MADlib > Issue Type: Improvement > Components: Deep Learning > Reporter: Frank McQuillan > Priority: Minor > Fix For: v1.16 > > > fit() INFO and CONTEXT messages > (1) no validation_table, metrics_compute_frequency=0 > {code} > SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table > 'iris_model', -- model output table > 'model_arch_library', -- model arch table > 1, -- model arch id > $$ loss='categorical_crossentropy', > optimizer='adam', metrics=['accuracy'] $$, -- compile_params > $$ batch_size=5, epochs=3 $$, -- fit_params > 10 -- num_iterations > ); > INFO: Processed 60 images: Fit took 0.567000865936 sec, Total was > 0.757196903229 sec (seg0 slice1 10.128.0.41:40000 pid=13317) > CONTEXT: PL/Python function "fit_transition" > INFO: Processed 60 images: Fit took 0.55348110199 sec, Total was > 0.741441011429 sec (seg1 slice1 10.128.0.41:40001 pid=13318) > CONTEXT: PL/Python function "fit_transition" > INFO: Time for training in iteration 1: 2.45737695694 sec > CONTEXT: PL/Python function "madlib_keras_fit" > {code} > change to > {code} > INFO: Time for training in iteration 1: 2.45737695694 sec > CONTEXT: PL/Python function "madlib_keras_fit" > {code} > (2) no validation_table, metrics_compute_frequency!=0 > {code} > SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table > 'iris_model', -- model output table > 'model_arch_library', -- model arch table > 1, -- model arch id > $$ loss='categorical_crossentropy', > optimizer='adam', metrics=['accuracy'] $$, -- compile_params > $$ batch_size=5, epochs=3 $$, -- fit_params > 10, -- num_iterations > 0, -- gpus per host > NULL, -- validation table > 1 -- metrics compute > frequency > ); > INFO: Processed 60 images: Fit took 0.534310817719 sec, Total was > 0.712550878525 sec (seg0 slice1 10.128.0.41:40000 pid=14501) > CONTEXT: PL/Python function "fit_transition" > INFO: Processed 60 images: Fit took 0.564456939697 sec, Total was > 0.751413106918 sec (seg1 slice1 10.128.0.41:40001 pid=14502) > CONTEXT: PL/Python function "fit_transition" > INFO: Time for training in iteration 1: 2.28858995438 sec > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Time for evaluation in iteration 1: 0.188971996307 sec. > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Training set metric after iteration 1: 0.649999976158. > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Training set loss after iteration 1: 1.1202558279. > CONTEXT: PL/Python function "madlib_keras_fit" > {code} > change to > {code} > INFO: Time for training in iteration 1: 2.28858995438 sec > Time for evaluation in iteration 1: 0.188971996307 sec > Training set metric after iteration 1: 0.649999976158 > Training set loss after iteration 1: 1.1202558279 > CONTEXT: PL/Python function "madlib_keras_fit" > {code} > (3) yes validation_table, metrics_compute_frequency=0 > {code} > SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table > 'iris_model', -- model output table > 'model_arch_library', -- model arch table > 1, -- model arch id > $$ loss='categorical_crossentropy', > optimizer='adam', metrics=['accuracy'] $$, -- compile_params > $$ batch_size=5, epochs=3 $$, -- fit_params > 10, -- num_iterations > 0, -- GPUs per host > 'iris_test_packed' -- validation dataset > ); > INFO: Processed 60 images: Fit took 0.552575826645 sec, Total was > 0.734694004059 sec (seg0 slice1 10.128.0.41:40000 pid=18431) > CONTEXT: PL/Python function "fit_transition" > INFO: Processed 60 images: Fit took 0.549551010132 sec, Total was > 0.734927892685 sec (seg1 slice1 10.128.0.41:40001 pid=18432) > CONTEXT: PL/Python function "fit_transition" > INFO: Time for training in iteration 1: 2.36340904236 sec > CONTEXT: PL/Python function "madlib_keras_fit" > {code} > change to > {code} > INFO: Time for training in iteration 1: 2.45737695694 sec > CONTEXT: PL/Python function "madlib_keras_fit" > {code} > (4) yes validation_table, metrics_compute_frequency=!0 > {code} > SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table > 'iris_model', -- model output table > 'model_arch_library', -- model arch table > 1, -- model arch id > $$ loss='categorical_crossentropy', > optimizer='adam', metrics=['accuracy'] $$, -- compile_params > $$ batch_size=5, epochs=3 $$, -- fit_params > 10, -- num_iterations > 0, -- GPUs per host > 'iris_test_packed', -- validation dataset > 1 -- metrics compute > frequency > ); > INFO: Processed 60 images: Fit took 0.57217502594 sec, Total was > 0.817452907562 sec (seg0 slice1 10.128.0.41:40000 pid=19573) > CONTEXT: PL/Python function "fit_transition" > INFO: Processed 60 images: Fit took 0.554927110672 sec, Total was > 0.800101041794 sec (seg1 slice1 10.128.0.41:40001 pid=19574) > CONTEXT: PL/Python function "fit_transition" > INFO: Time for training in iteration 1: 2.43148899078 sec > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Time for evaluation in iteration 1: 0.217161893845 sec. > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Training set metric after iteration 1: 0.524999976158. > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Training set loss after iteration 1: 0.984773635864. > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Time for evaluation in iteration 1: 0.205282926559 sec. > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Validation set metric after iteration 1: 0.600000023842. > CONTEXT: PL/Python function "madlib_keras_fit" > INFO: Validation set loss after iteration 1: 0.940379023552. > CONTEXT: PL/Python function "madlib_keras_fit" > {code} > change to > {code} > INFO: Time for training in iteration 1: 2.43148899078 sec > Time for evaluating training dataset in iteration 1: 0.217161893845 sec > Training set metric after iteration 1: 0.524999976158 > Training set loss after iteration 1: 0.984773635864 > Time for evaluating validation dataset in iteration 1: 0.205282926559 > sec > Validation set metric after iteration 1: 0.600000023842 > Validation set loss after iteration 1: 0.940379023552 > CONTEXT: PL/Python function "madlib_keras_fit" > {code} > Note change in wording ^^^ because there are 2 evaluation times. -- This message was sent by Atlassian JIRA (v7.6.3#76005)