[
https://issues.apache.org/jira/browse/SPARK-23674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-23674.
----------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 23263
[https://github.com/apache/spark/pull/23263]
> Add Spark ML Listener for Tracking ML Pipeline Status
> -----------------------------------------------------
>
> Key: SPARK-23674
> URL: https://issues.apache.org/jira/browse/SPARK-23674
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 2.3.0
> Reporter: Mingjie Tang
> Assignee: Hyukjin Kwon
> Priority: Major
> Fix For: 3.0.0
>
>
> Currently, Spark provides status monitoring for different components of
> Spark, like spark history server, streaming listener, sql listener and etc.
> The use case would be (1) front UI to track the status of training coverage
> rate during iteration, then DS can understand how the job converge when
> training, like K-means, Logistic and other linear regression model. (2)
> tracking the data lineage for the input and output of training data.
> In this proposal, we hope to provide Spark ML pipeline listener to track the
> status of Spark ML pipeline status includes:
> # ML pipeline create and saved
> # ML pipeline model created, saved and load
> # ML model training status monitoring
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
