[
https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681895#comment-16681895
]
John Bauer commented on SPARK-21542:
------------------------------------
This is a) much more minimal, b) genuinely useful, and c) actually works with
save and load, for example:
impute.write().save("impute")
imp = ImputeNormal.load("impute")
imp.explainParams()
impute_model.write().save("impute_model")
impm = ImputeNormalModel.load("imputer_model")
impm = ImputeNormalModel.load("impute_model")
impm.getInputCol()
impm.getOutputCol()
impm.getMean()
impm.getStddev()
> Helper functions for custom Python Persistence
> ----------------------------------------------
>
> Key: SPARK-21542
> URL: https://issues.apache.org/jira/browse/SPARK-21542
> Project: Spark
> Issue Type: New Feature
> Components: ML, PySpark
> Affects Versions: 2.2.0
> Reporter: Ajay Saini
> Assignee: Ajay Saini
> Priority: Major
> Fix For: 2.3.0
>
>
> Currently, there is no way to easily persist Json-serializable parameters in
> Python only. All parameters in Python are persisted by converting them to
> Java objects and using the Java persistence implementation. In order to
> facilitate the creation of custom Python-only pipeline stages, it would be
> good to have a Python-only persistence framework so that these stages do not
> need to be implemented in Scala for persistence.
> This task involves:
> - Adding implementations for DefaultParamsReadable, DefaultParamsWriteable,
> DefaultParamsReader, and DefaultParamsWriter in pyspark.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
