[
https://issues.apache.org/jira/browse/SPARK-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959978#comment-14959978
]
Yanbo Liang commented on SPARK-8418:
------------------------------------
[~josephkb] I don't think RFormula is the best way to resolve this issue
because it still use the pipeline chained transformers one by one to encode
multiple columns which is low performance.
I vote for strategy 2 of [~nburoojy] proposed. But I think we don't need to
reimplement all transformers to support a multi-value implementation because of
some feature transformers not needed.
I will firstly try to start with OneHotEncoder which is mostly common used.
> Add single- and multi-value support to ML Transformers
> ------------------------------------------------------
>
> Key: SPARK-8418
> URL: https://issues.apache.org/jira/browse/SPARK-8418
> Project: Spark
> Issue Type: Sub-task
> Components: ML
> Reporter: Joseph K. Bradley
>
> It would be convenient if all feature transformers supported transforming
> columns of single values and multiple values, specifically:
> * one column with one value (e.g., type {{Double}})
> * one column with multiple values (e.g., {{Array[Double]}} or {{Vector}})
> We could go as far as supporting multiple columns, but that may not be
> necessary since VectorAssembler could be used to handle that.
> Estimators under {{ml.feature}} should also support this.
> This will likely require a short design doc to describe:
> * how input and output columns will be specified
> * schema validation
> * code sharing to reduce duplication
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
