Ruifeng Zheng created SPARK-61361:
-------------------------------------
Summary: Support configurable intermediate storage levels in tree
estimators
Key: SPARK-61361
URL: https://issues.apache.org/jira/browse/SPARK-61361
Project: Spark
Issue Type: Improvement
Components: ML
Affects Versions: 5.0.0
Reporter: Ruifeng Zheng
Decision tree, random forest, and gradient-boosted tree classifiers and
regressors cache intermediate training datasets at hard-coded storage levels.
Expose the existing intermediateStorageLevel shared parameter on all six
spark.ml estimators in Scala and PySpark.
Pass the configured level through the shared RandomForest and
GradientBoostedTrees implementations, including bagged data, cached node IDs,
binned training and validation data, sampling counts, and prediction/error
caches. Keep MEMORY_AND_DISK as the default for existing callers.
Include PySpark constructor and setParams keyword support, and extend the
existing estimator tests.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]