[
https://issues.apache.org/jira/browse/MADLIB-1338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846993#comment-16846993
]
Nandish Jayaram commented on MADLIB-1338:
-----------------------------------------
Thank you for the comments [~fmcquillan]. Since the PR associated with this
JIRA is already merged, we can address comments in another related JIRA
(https://issues.apache.org/jira/browse/MADLIB-1349).
Please find the answers below:
1) As of now, we don't plan to support:
'sparse_categorical_accuracy',
'Sparse_categorical_crossentropy',
'top_k_categorical_accuracy',
'sparse_top_k_categorical_accuracy'
2) We will address this in MADLIB-1349.
3) Array might be ok to have, since we should eventually support multiple
metrics, and then this column will actually become a multi-dim array.
The first iteration normally takes longer then rest.
4) They are the timestamps at which loss/metrics were computed. It might be a
good idea to report cumulative time (in seconds) instead of timestamps. We will
try to do that in MADLIB-1349.
9) Yes, we will address this in MADLIB-1349.
11) We will address this in MADLIB-1349.
> DL: Add support for reporting various metrics in fit/evaluate
> -------------------------------------------------------------
>
> Key: MADLIB-1338
> URL: https://issues.apache.org/jira/browse/MADLIB-1338
> Project: Apache MADlib
> Issue Type: New Feature
> Components: Deep Learning
> Reporter: Nandish Jayaram
> Priority: Major
> Fix For: v1.16
>
>
> The current `madlib_keras.fit()` code reports accuracy as the only metric,
> along with loss value. But we could ask for different metrics in compile
> params (`mae`, `binary_accuracy ` etc.), then `Keras.evaluate()` would return
> back `loss` (by default) and `mean_absolute_error` or `binary_accuracy`
> (metrics).
> This JIRA requests support to be able to report any one of these metrics in
> the output table.
> Other requirements:
> 1. Remove training loss/accuracy computation from `fit_transition` and
> instead use the evaluate function to calculate the training loss/metric. See
> PR [https://github.com/apache/madlib/pull/388
> |https://github.com/apache/madlib/pull/388/files]for more details
> 2. metric param can be optional
> 3. Maybe we should rename all the related output column as metric instead of
> metrics
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
