Sorry for the wrong link, what you should refer is jpmml-sparkml (
https://github.com/jpmml/jpmml-sparkml).
Thanks
Yanbo
2016-07-24 4:46 GMT-07:00 Yanbo Liang :
> Spark does not support exporting ML models to PMML currently. You can try
> the third party jpmml-spark
Spark does not support exporting ML models to PMML currently. You can try
the third party jpmml-spark (https://github.com/jpmml/jpmml-spark) package
which supports a part of ML models.
Thanks
Yanbo
2016-07-20 11:14 GMT-07:00 Ajinkya Kale :
> Just found Google dataproc has
Just found Google dataproc has a preview of spark 2.0. Tried it and
save/load works! Thanks Shuai.
Followup question - is there a way to export the pyspark.ml models to PMML
? If not, what is the best way to integrate the model for inference in a
production service ?
On Tue, Jul 19, 2016 at 8:22
I am using google cloud dataproc which comes with spark 1.6.1. So upgrade
is not really an option.
No way / hack to save the models in spark 1.6.1 ?
On Tue, Jul 19, 2016 at 8:13 PM Shuai Lin wrote:
> It's added in not-released-yet 2.0.0 version.
>
>
It's added in not-released-yet 2.0.0 version.
https://issues.apache.org/jira/browse/SPARK-13036
https://github.com/apache/spark/commit/83302c3b
so i guess you need to wait for 2.0 release (or use the current rc4).
On Wed, Jul 20, 2016 at 6:54 AM, Ajinkya Kale wrote:
>
Is there a way to save a pyspark.ml.feature.PCA model ? I know mllib has
that but mllib does not have PCA afaik. How do people do model persistence
for inference using the pyspark ml models ? Did not find any documentation
on model persistency for ml.
--ajinkya