fmcquillan99 commented on pull request #511:
URL: https://github.com/apache/madlib/pull/511#issuecomment-678429057
(4)
if I use the same function for custom function for loss and metrics
```
# import database connector psycopg2 and create connection cursor
import psycopg2 as p2
conn = p2.connect('postgresql://gpadmin@localhost:8000/madlib')
cur = conn.cursor()
# import Dill and define functions
import dill
# custom loss
def msee(y_true, y_pred):
import keras.backend as K
loss=K.mean(K.sum(K.square(y_true-y_pred)))
return loss
pb_msee=dill.dumps(msee)
# custom metric
def rmse(y_true, y_pred):
import keras.backend as K
return K.sqrt(K.mean(K.square(y_pred - y_true), axis=-1))
pb_rmse=dill.dumps(rmse)
# call load function
cur.execute("DROP TABLE IF EXISTS test_custom_function_table")
cur.execute("SELECT
madlib.load_custom_function('test_custom_function_table', %s,'msee', 'mean
squared error loss')", [p2.Binary(pb_msee)])
cur.execute("SELECT
madlib.load_custom_function('test_custom_function_table', %s,'rmse', 'root
mean square error metric')", [p2.Binary(pb_rmse)])
conn.commit()
```
```
DROP TABLE IF EXISTS iris_model_cl, iris_model_cl_summary;
SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table
'iris_model_cl', -- model output
table
'model_arch_library', -- model arch table
1, -- model arch id
$$ loss=msee, optimizer='adam',
metrics=['msee'] $$, -- compile_params
$$ batch_size=5, epochs=3 $$, -- fit_params
3, -- num_iteration
NULL, -- use_gpus,
NULL, -- validation_table,
NULL, -- metrics_compute_frequency,
NULL, -- warm_start,
NULL, -- name
NULL, -- description
'test_custom_function_table' -- object table
);
```
produces
```
InternalError: (psycopg2.errors.InternalError_) plpy.Error: Custom function
['msee', 'msee'] not defined in object table 'test_custom_function_table'
(plpython.c:5038)
CONTEXT: Traceback (most recent call last):
PL/Python function "madlib_keras_fit", line 23, in <module>
madlib_keras.fit(**globals())
PL/Python function "madlib_keras_fit", line 42, in wrapper
PL/Python function "madlib_keras_fit", line 157, in fit
PL/Python function "madlib_keras_fit", line 397, in
query_custom_functions_map
PL/Python function "madlib_keras_fit"
[SQL: SELECT madlib.madlib_keras_fit('iris_train_packed', -- source table
'iris_model_cl', -- model output
table
'model_arch_library', -- model arch table
1, -- model arch id
$$ loss=msee, optimizer='adam',
metrics=['msee'] $$, -- compile_params
$$ batch_size=5, epochs=3 $$, -- fit_params
3, -- num_iteration
NULL, -- use_gpus,
NULL, -- validation_table,
NULL, -- metrics_compute_frequency,
NULL, -- warm_start,
NULL, -- name
NULL, -- description
'test_custom_function_table' -- object table
);]
(Background on this error at: http://sqlalche.me/e/2j85)
```
(5)
also it seems from https://keras.io/api/losses/ like that standard is to put
quotes around built-in functions like:
```
# pass optimizer by name: default parameters will be used
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')
```
but no quote for custom functions
```
loss_fn = keras.losses.SparseCategoricalCrossentropy()
model.compile(loss=loss_fn, optimizer='adam')
```
so I suggest we use the same convention
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
