[
https://issues.apache.org/jira/browse/SPARK-23674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Jiang updated SPARK-23674:
-------------------------------
Description:
Currently, Spark provides status monitoring for different components of Spark,
like spark history server, streaming listener, sql listener and etc. The use
case would be (1) front UI to track the status of training coverage rate during
iteration, then DS can understand how the job converge when training, like
K-means, Logistic and other linear regression model. (2) tracking the data
lineage for the input and output of training data.
In this proposal, we hope to provide Spark ML pipeline listener to track the
status of Spark ML pipeline status includes:
# ML pipeline create and saved
# ML pipeline model created, saved and load
# ML model training status monitoring
was:
Currently, Spark provides status monitoring for different components of Spark,
like spark history server, streaming listener, sql listener and etc. The use
case would be (1) front UI to track the status of training coverage rate during
iteration, then DS can understand how the job converge when training, like
K-means, Logistic and other linear regression model. (2) tracking the data
lineage for the input and output of training data.
In this proposal, we hope to provide Spark ML pipeline listener to track the
status of Spark ML pipeline status includes:
# ML pipeline create and saved
# ML pipeline mode created, saved and load
# ML model training status monitoring
> Add Spark ML Listener for Tracking ML Pipeline Status
> -----------------------------------------------------
>
> Key: SPARK-23674
> URL: https://issues.apache.org/jira/browse/SPARK-23674
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 2.3.0
> Reporter: Mingjie Tang
> Priority: Major
>
> Currently, Spark provides status monitoring for different components of
> Spark, like spark history server, streaming listener, sql listener and etc.
> The use case would be (1) front UI to track the status of training coverage
> rate during iteration, then DS can understand how the job converge when
> training, like K-means, Logistic and other linear regression model. (2)
> tracking the data lineage for the input and output of training data.
> In this proposal, we hope to provide Spark ML pipeline listener to track the
> status of Spark ML pipeline status includes:
> # ML pipeline create and saved
> # ML pipeline model created, saved and load
> # ML model training status monitoring
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
