[ https://issues.apache.org/jira/browse/SPARK-19422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162008#comment-16162008 ]
Joseph K. Bradley commented on SPARK-19422: ------------------------------------------- Linking [SPARK-21972], which may interact with this PR. Let's be very careful about changing the behavior, though I agree this will be valuable to improve. > Cache input data in algorithms > ------------------------------ > > Key: SPARK-19422 > URL: https://issues.apache.org/jira/browse/SPARK-19422 > Project: Spark > Issue Type: Improvement > Components: ML > Affects Versions: 2.2.0 > Reporter: zhengruifeng > > Now some algorithms cache the input dataset if it was not cached any more > {{StorageLevel.NONE}}: > {{FeedForwardTrainer}}, {{LogisticRegression}}, {{OneVsRest}}, {{KMeans}}, > {{AFTSurvivalRegression}}, {{IsotonicRegression}}, {{LinearRegression}} with > non-WSL solver > It maybe reasonable to cache input for others: > {{DecisionTreeClassifier}}, {{GBTClassifier}}, {{RandomForestClassifier}}, > {{LinearSVC}} > {{BisectingKMeans}}, {{GaussianMixture}}, {{LDA}} > {{DecisionTreeRegressor}}, {{GBTRegressor}}, {{GeneralizedLinearRegression}} > with IRLS solver, {{RandomForestRegressor}} > {{NaiveBayes}} is not included since it only make one pass on the data. > {{MultilayerPerceptronClassifier}} is not included since the data is cached > in {{FeedForwardTrainer.train}} -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org