[
https://issues.apache.org/jira/browse/MADLIB-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767635#comment-16767635
]
Frank McQuillan commented on MADLIB-1300:
-----------------------------------------
https://github.com/apache/madlib/pull/351
{code}
SELECT * FROM image_data_packed ORDER BY buffer_id;
-[ RECORD 1 ]-------+-------------------------------------
independent_var |
{{0.501961,0.2,0.286275,0.886275,0.407843,0.760784,0.286275,0.745098,0.384314,0.0901961,0.384314,0.396078},...}}
dependent_var |
{cat,dog,cat,cat,bird,cat,dog,cat,dog,dog,dog,bird,dog,dog,cat,cat,cat,cat,bird,bird,bird,cat,cat,bird,dog,dog}
buffer_id | 0
-[ RECORD 2 ]-------+-------------------------------------
independent_var |
{{0.694118,0.760784,0.72549,0.686275,0.168627,0.0627451,0.803922,0.360784,0.643137,0.509804,0.8,0.392157},...}}
dependent_var |
{cat,dog,dog,dog,cat,bird,bird,dog,cat,cat,dog,bird,cat,cat,dog,bird,cat,dog,cat,bird,cat,bird,bird,cat,dog,cat}
buffer_id | 1
{code}
LGTM
> Clarify dep and indep var column names in output table for deep learning
> minibatch preprocessor
> -----------------------------------------------------------------------------------------------
>
> Key: MADLIB-1300
> URL: https://issues.apache.org/jira/browse/MADLIB-1300
> Project: Apache MADlib
> Issue Type: Improvement
> Components: Module: Utilities
> Reporter: Frank McQuillan
> Assignee: Himanshu Pandey
> Priority: Minor
> Fix For: v1.16
>
>
> Follow on to this commit:
> Minibatch Preprocessor for Deep learning
> https://github.com/apache/madlib/commit/8de32ede33c48d2f4a440f0f639c94a277a359c1
> The output table produced by the deep mini-batch preprocessor contains the
> following columns:
> {code}
> ...
> dependent_varname FLOAT8[]. Packed array of dependent variables. If the
> dependent variable in the source table is categorical, the preprocessor will
> one-hot encode it.
> independent_varname FLOAT8[]. Packed array of independent variables.
> ...
> {code}
> This is misleading because these columns contain values not names, so we
> should rename these columns to:
> {code}
> ...
> dependent_var
> independent_var
> ...
> {code}
> The output summary table contains the following columns:
> {code}
> dependent_varname Dependent variable from the source table.
> independent_varname Independent variable from the source table.
> {code}
> This is OK since the columns actually do contain names.
> There is a related 2.0 story for the regular mini-batch preprocessor
> http://madlib.apache.org/docs/latest/group__grp__minibatch__preprocessing.html
> in JIRA https://issues.apache.org/jira/browse/MADLIB-1294 which we don't want
> to do in 1.16 since it will break semantic versioning
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
