[jira] [Commented] (SPARK-7132) Add fit with validation set to spark.ml GBT
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448039#comment-16448039 ] Apache Spark commented on SPARK-7132: - User 'WeichenXu123' has created a pull request for this issue: https://github.com/apache/spark/pull/21129 > Add fit with validation set to spark.ml GBT > --- > > Key: SPARK-7132 > URL: https://issues.apache.org/jira/browse/SPARK-7132 > Project: Spark > Issue Type: Improvement > Components: ML >Reporter: Joseph K. Bradley >Priority: Minor > > In spark.mllib GradientBoostedTrees, we have a method runWithValidation which > takes a validation set. We should add that to the spark.ml API. > This will require a bit of thinking about how the Pipelines API should handle > a validation set (since Transformers and Estimators only take 1 input > DataFrame). The current plan is to include an extra column in the input > DataFrame which indicates whether the row is for training, validation, etc. > Goals > A [P0] Support efficient validation during training > B [P1] Support early stopping based on validation metrics > C [P0] Ensure validation data are preprocessed identically to training data > D [P1] Support complex Pipelines with multiple models using validation data > Proposal: column with indicator for train vs validation > Include an extra column in the input DataFrame which indicates whether the > row is for training or validation. Add a Param “validationFlagCol” used to > specify the extra column name. > A, B, C are easy. > D is doable. > Each estimator would need to have its validationFlagCol Param set to the same > column. > Complication: It would be ideal if we could prevent different estimators from > using different validation sets. (Joseph: There is not an obvious way IMO. > Maybe we can address this later by, e.g., having Pipelines take a > validationFlagCol Param and pass that to the sub-models in the Pipeline. > Let’s not worry about this for now.) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7132) Add fit with validation set to spark.ml GBT
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441864#comment-16441864 ] Weichen Xu commented on SPARK-7132: --- I dicussed with [~josephkb] and paste the proposal on JIRA. [~yanboliang] Do you agree with it or do you have other thoughts ? > Add fit with validation set to spark.ml GBT > --- > > Key: SPARK-7132 > URL: https://issues.apache.org/jira/browse/SPARK-7132 > Project: Spark > Issue Type: Improvement > Components: ML >Reporter: Joseph K. Bradley >Priority: Minor > > In spark.mllib GradientBoostedTrees, we have a method runWithValidation which > takes a validation set. We should add that to the spark.ml API. > This will require a bit of thinking about how the Pipelines API should handle > a validation set (since Transformers and Estimators only take 1 input > DataFrame). The current plan is to include an extra column in the input > DataFrame which indicates whether the row is for training, validation, etc. > Goals > A [P0] Support efficient validation during training > B [P1] Support early stopping based on validation metrics > C [P0] Ensure validation data are preprocessed identically to training data > D [P1] Support complex Pipelines with multiple models using validation data > Proposal: column with indicator for train vs validation > Include an extra column in the input DataFrame which indicates whether the > row is for training or validation. Add a Param “validationFlagCol” used to > specify the extra column name. > A, B, C are easy. > D is doable. > Each estimator would need to have its validationFlagCol Param set to the same > column. > Complication: It would be ideal if we could prevent different estimators from > using different validation sets. (Joseph: There is not an obvious way IMO. > Maybe we can address this later by, e.g., having Pipelines take a > validationFlagCol Param and pass that to the sub-models in the Pipeline. > Let’s not worry about this for now.) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7132) Add fit with validation set to spark.ml GBT
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877781#comment-15877781 ] Vincent commented on SPARK-7132: Hi All, any update on this issue? > Add fit with validation set to spark.ml GBT > --- > > Key: SPARK-7132 > URL: https://issues.apache.org/jira/browse/SPARK-7132 > Project: Spark > Issue Type: Improvement > Components: ML >Reporter: Joseph K. Bradley >Priority: Minor > > In spark.mllib GradientBoostedTrees, we have a method runWithValidation which > takes a validation set. We should add that to the spark.ml API. > This will require a bit of thinking about how the Pipelines API should handle > a validation set (since Transformers and Estimators only take 1 input > DataFrame). The current plan is to include an extra column in the input > DataFrame which indicates whether the row is for training, validation, etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7132) Add fit with validation set to spark.ml GBT
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15284119#comment-15284119 ] Mahmoud Rawas commented on SPARK-7132: -- Probably, it will be more beneficial for the user to specify splits as they may decide to cache or persist splits for better performance. > Add fit with validation set to spark.ml GBT > --- > > Key: SPARK-7132 > URL: https://issues.apache.org/jira/browse/SPARK-7132 > Project: Spark > Issue Type: Improvement > Components: ML >Reporter: Joseph K. Bradley >Priority: Minor > > In spark.mllib GradientBoostedTrees, we have a method runWithValidation which > takes a validation set. We should add that to the spark.ml API. > This will require a bit of thinking about how the Pipelines API should handle > a validation set (since Transformers and Estimators only take 1 input > DataFrame). The current plan is to include an extra column in the input > DataFrame which indicates whether the row is for training, validation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7132) Add fit with validation set to spark.ml GBT
[ https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735926#comment-14735926 ] Joseph K. Bradley commented on SPARK-7132: -- How would the split be chosen? It will be important for the user to be able to specify the split; for that, an extra Boolean column seems like a reasonable choice. > Add fit with validation set to spark.ml GBT > --- > > Key: SPARK-7132 > URL: https://issues.apache.org/jira/browse/SPARK-7132 > Project: Spark > Issue Type: Improvement > Components: ML >Reporter: Joseph K. Bradley >Priority: Minor > > In spark.mllib GradientBoostedTrees, we have a method runWithValidation which > takes a validation set. We should add that to the spark.ml API. > This will require a bit of thinking about how the Pipelines API should handle > a validation set (since Transformers and Estimators only take 1 input > DataFrame). The current plan is to include an extra column in the input > DataFrame which indicates whether the row is for training, validation, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7132) Add fit with validation set to spark.ml GBT
[
https://issues.apache.org/jira/browse/SPARK-7132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724889#comment-14724889
]
Yanbo Liang commented on SPARK-7132:
I will work on this issue.
[~josephkb]
I propose another way to resolve this issue.
The GBT Estimator remains take 1 input {DataFrame}, and we will split it into
training and validation dataset internal.
Because the runWithValidation interface will take RDD[LabeledPoint] as input,
it's easy to handle this.
And at the end of the GBT Estimator, we can also union these two dataset.
> Add fit with validation set to spark.ml GBT
> ---
>
> Key: SPARK-7132
> URL: https://issues.apache.org/jira/browse/SPARK-7132
> Project: Spark
> Issue Type: Improvement
> Components: ML
>Reporter: Joseph K. Bradley
>Priority: Minor
>
> In spark.mllib GradientBoostedTrees, we have a method runWithValidation which
> takes a validation set. We should add that to the spark.ml API.
> This will require a bit of thinking about how the Pipelines API should handle
> a validation set (since Transformers and Estimators only take 1 input
> DataFrame). The current plan is to include an extra column in the input
> DataFrame which indicates whether the row is for training, validation, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
