Jason Wang created SPARK-27892: ---------------------------------- Summary: Saving/loading stages in PipelineModel should be parallel Key: SPARK-27892 URL: https://issues.apache.org/jira/browse/SPARK-27892 Project: Spark Issue Type: Improvement Components: ML Affects Versions: 2.4.3 Reporter: Jason Wang
When a PipelineModel is saved/loaded, all the stages are saved/loaded sequentially. When dealing with a PipelineModel with many stages, although each stage's save/load takes sub-second, the total time taken for the PipelineModel could be several minutes. It should be trivial to parallelize the save/load of stages in the SharedReadWrite object. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org