Ruifeng Zheng created SPARK-58587:
-------------------------------------
Summary: Use spillable sort for final IsotonicRegression pass
Key: SPARK-58587
URL: https://issues.apache.org/jira/browse/SPARK-58587
Project: Spark
Issue Type: Sub-task
Components: ML
Affects Versions: 5.0.0
Reporter: Ruifeng Zheng
Move the final IsotonicRegression Pool Adjacent Violators pass from the driver
to an executor after a spill-capable shuffle sort, reducing driver memory
pressure before the final model is collected.\n\nDraft PR:
https://github.com/apache/spark/pull/57788
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
