njayaram2 commented on a change in pull request #389: DL: Convert the
keras_eval function from UDF to UDA
URL: https://github.com/apache/madlib/pull/389#discussion_r283045618
##########
File path: src/ports/postgres/modules/deep_learning/madlib_keras.py_in
##########
@@ -558,78 +529,118 @@ def evaluate1(schema_madlib, model_table, test_table,
id_col, model_arch_table,
plpy.info('evaluate result acc is {}'.format(loss_acc[1]))
def get_loss_acc_from_keras_eval(schema_madlib, table, dependent_varname,
- independent_varname, compile_params,
model_arch,
- model_data, gpus_per_host, segments_per_host,
- seg_ids_val,
- rows_per_seg_val, gp_segment_id_col):
+ independent_varname, compile_params,
+ model_arch, model_data, gpus_per_host,
+ segments_per_host, seg_ids, images_per_seg,
+ gp_segment_id_col):
"""
This function will call the internal keras evaluate function to get the
loss
and accuracy of each tuple which then gets averaged to get the final
result.
"""
evaluate_query = plpy.prepare("""
- select {schema_madlib}.array_avg(loss_acc, True) as final_loss_acc from
- (
- select ({schema_madlib}.internal_keras_evaluate({dependent_varname},
- {independent_varname},
+ -- TODO: really, we should not be casting integers and big integers to
smallint's
+ -- The right solution is either to change the datatype of the agg
function from
+ -- SMALLINT to INTEGER, or change the output of minibatch util to produce
SMALLINT
+ -- For the first, we should change fit_step also
+ select
({schema_madlib}.internal_keras_evaluate({dependent_varname}::SMALLINT[],
+ {independent_varname}::REAL[],
$MAD${model_arch}$MAD$,
- $1, {compile_params},
+ $1,
+ {compile_params},
{gpus_per_host},
{segments_per_host},
- ARRAY{seg_ids_val},
- ARRAY{rows_per_seg_val},
+ ARRAY{seg_ids},
+ ARRAY{images_per_seg},
{gp_segment_id_col})) as loss_acc
Review comment:
Can we indent this to the left to start at 4 tabs after the column that
`{schema_madlib}` is starting in, or any better indentation you see fit?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
