[
https://issues.apache.org/jira/browse/MADLIB-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488048#comment-16488048
]
Nandish Jayaram commented on MADLIB-1239:
-----------------------------------------
w/ [~okislal]
A couple of questions:
1) The new column _feature_name_ in the output table is going to have the same
value for each row?
2) Should we assume the output column name to be _feature_vector_ all the time,
or should this also be a optional param?
There is also a concern regarding reversing what _cols2vec()_ does with
_vec2cols()_ when one of the input features is an array. We won't be able to
differentiate between original scalar and array columns once they are packed
into the _feature_vector_.
> Columns to Vector
> -----------------
>
> Key: MADLIB-1239
> URL: https://issues.apache.org/jira/browse/MADLIB-1239
> Project: Apache MADlib
> Issue Type: New Feature
> Components: Module: Utilities
> Reporter: Frank McQuillan
> Priority: Major
> Fix For: v1.15
>
>
> Columns to Vector
> Converts features from multiple columns of an input table into a feature
> array in a single column. Also outputs the names of the features into an
> array in a single column. This process can be reversed using the function
> vec2cols.
> {code}
> cols2vec(
> source_table,
> out_table,
> list_of_features,
> list_of_features_to_exclude,
> cols_to_output
> )
> source_table
> TEXT. Name of the table containing the source data.
> out_table
> TEXT. Name of the generated table containing the output. If a table with the
> same name already exists, an error will be returned.
> list_of_features
> TEXT. Comma-separated string of column names or expressions to put into
> array. Can also be a '*' implying all columns are to be put into array
> (except for the ones included in the next argument that lists exclusions).
> The types of the features should all be the same since PostgreSQL arrays only
> support elements of the same type. If multiple numeric types are present in
> the list of features, they will be cast to DOUBLE PRECISION in the feature
> array.
> Array columns can also be included in the list, and the array will be
> expanded to treat each element of the array as a separate feature.
> list_of_features_to_exclude (optional)
> TEXT, default NULL. Comma-separated string of column names to exclude from
> the feature array. Use only when list_of_features is '*'.
> cols_to_output (optional)
> TEXT, default NULL. Comma-separated string of column names from the source
> table to keep in the output table, in addition to the feature array. To keep
> all columns from the source table, use '*'.
> Output
> The output table produced by the cols2vec function contains the following
> columns:
> <...>
> Columns from source table, depending on which ones are kept (if any).
> feature_vector
> Array of features. Array type will depend on feature type in the source
> table.
> feature_names
> TEXT[] Array of names of features.
> {code}
> Open questions for cols2vec:
> 1) OK to cast to double if there are mixed numeric types? Or should we
> enforce that all feature columns be the exact same type? (Since elements of
> PostgreSQL arrays need to be the same type.)
> Aside
> The function
> http://pivotalsoftware.github.io/PDLTools/group__grp__array__utilities.html#cols2vec_example
> is similar but the proposed MADlib one has more options. To do the
> equivalent of the PDL Tools one in MADlib, you would do:
> {code}
> cols2vec(
> table_name,
> output_table,
> '*',
> exclude_columns,
> '*'
> )
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
