[jira] [Commented] (SPARK-6634) Allow replacing columns in Transformers
[ https://issues.apache.org/jira/browse/SPARK-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904445#comment-15904445 ] Tree Field commented on SPARK-6634: --- I want this feature too. because I often overwrite UnaryTransformer by myself to enable this. It seems it's only prevented in transformSchema method. Now, unlike before v1.4, dataframe's withColumn method used in UnaryTransformer allows replacing the input column. Any other reasons that is not allowed in transoformer, especially in UnaryTransformer. > Allow replacing columns in Transformers > --- > > Key: SPARK-6634 > URL: https://issues.apache.org/jira/browse/SPARK-6634 > Project: Spark > Issue Type: Improvement > Components: ML >Affects Versions: 1.3.0 >Reporter: Joseph K. Bradley >Priority: Minor > > Currently, Transformers do not allow input and output columns to share the > same name. (In fact, this is not allowed but also not even checked.) > Short-term proposal: Disallow input and output columns with the same name, > and add a check in transformSchema. > Long-term proposal: Allow input & output columns with the same name, and > where the behavior is that the output columns replace input columns with the > same name. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6634) Allow replacing columns in Transformers
[ https://issues.apache.org/jira/browse/SPARK-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516043#comment-14516043 ] Joseph K. Bradley commented on SPARK-6634: -- Not merged, but now feasible Allow replacing columns in Transformers --- Key: SPARK-6634 URL: https://issues.apache.org/jira/browse/SPARK-6634 Project: Spark Issue Type: Improvement Components: ML Affects Versions: 1.3.0 Reporter: Joseph K. Bradley Priority: Minor Currently, Transformers do not allow input and output columns to share the same name. (In fact, this is not allowed but also not even checked.) Short-term proposal: Disallow input and output columns with the same name, and add a check in transformSchema. Long-term proposal: Allow input output columns with the same name, and where the behavior is that the output columns replace input columns with the same name. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-6634) Allow replacing columns in Transformers
[ https://issues.apache.org/jira/browse/SPARK-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516039#comment-14516039 ] Xiangrui Meng commented on SPARK-6634: -- Is this feature merged, or at least feasible now? Allow replacing columns in Transformers --- Key: SPARK-6634 URL: https://issues.apache.org/jira/browse/SPARK-6634 Project: Spark Issue Type: Improvement Components: ML Affects Versions: 1.3.0 Reporter: Joseph K. Bradley Priority: Minor Currently, Transformers do not allow input and output columns to share the same name. (In fact, this is not allowed but also not even checked.) Short-term proposal: Disallow input and output columns with the same name, and add a check in transformSchema. Long-term proposal: Allow input output columns with the same name, and where the behavior is that the output columns replace input columns with the same name. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
