[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I have a project about model transformation with PMML, it needs to transform models with different types to pmml files. Moreover, JPMML(https://github.com/jpmml) has provided tools to do that,such as jpmml-sklearn, jpmml-xgboost etc. Our transformation API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema as metadata when do export? * The situations about machine learning libraries to jpmml are as the attached image, only xgboost and spark can't include schema info in exported model file. was: Hi all, I have a project about model transformation with PMML, it needs to transform models with different types to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema as metadata when do export? * The situations about machine learning libraries to jpmml are as the attached img, only xgboost and spark can't include schema info in exported model file. > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I have a project about model transformation with PMML, it needs to transform > models with different types to pmml files. > Moreover, JPMML(https://github.com/jpmml) has provided tools to do that,such > as jpmml-sklearn, jpmml-xgboost etc. Our transformation API parameters must > be concise and simple, in other words the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. However, JPMML-SPARK converter needs two > arguments: Data Schema and PipelineModel > *Can spark PipelineModel include input data schema as metadata when do > export? * > The situations about machine learning libraries to jpmml are as the attached > image, only xgboost and spark can't include schema info in exported model > file. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I have a project about model transformation with PMML, it needs to transform models with different types to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema as metadata when do export? * The situations about machine learning libraries to jpmml are as the attached img, only xgboost and spark can't include schema info in exported model file. was: Hi all, I recently did a research about pmml, and our project needs to transform many models with different types to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema as metadata when do export? * The situations about machine learning libraries to jpmml are as the attached img, only xgboost and spark can't include schema info in exported model file. > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I have a project about model transformation with PMML, it needs to transform > models with different types to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API > parameters must be concise and simple, in other words the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. However, JPMML-SPARK converter needs two > arguments: Data Schema and PipelineModel > *Can spark PipelineModel include input data schema as metadata when do > export? * > The situations about machine learning libraries to jpmml are as the attached > img, only xgboost and spark can't include schema info in exported model file. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and our project needs to transform many models with different types to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema as metadata when do export? * The situations about machine learning libraries to jpmml are as the attached img, only xgboost and spark can't include schema info in exported model file. was: Hi all, I recently did a research about pmml, and our project needs to transform many models with different types to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as the attached img > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and our project needs to transform many > models with different types to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API > parameters must be concise and simple, in other words the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. However, JPMML-SPARK converter needs two > arguments: Data Schema and PipelineModel > *Can spark PipelineModel include input data schema as metadata when do > export? * > The situations about machine learning libraries to jpmml are as the attached > img, only xgboost and spark can't include schema info in exported model file. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and our project needs to transform many models with different types to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as the attached img was: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as the attached img > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and our project needs to transform many > models with different types to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API > parameters must be concise and simple, in other words the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. However, JPMML-SPARK converter needs two > arguments: Data Schema and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as the attached > img -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API parameters must be concise and simple, in other words the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as the attached img was: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as the attached img > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API > parameters must be concise and simple, in other words the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. However, JPMML-SPARK converter needs two > arguments: Data Schema and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as the attached > img -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. However, JPMML-SPARK converter needs two arguments: Data Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as the attached img was: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as the attached img > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. However, JPMML-SPARK converter needs two > arguments: Data Schema and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as the attached > img -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as the attached img was: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: !jpmml-research.jpg|thumbnail! > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as the attached > img -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Attachment: jpmml-research.jpg [^jpmml-research.jpg] > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > !jpmml-research.jpg|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Attachment: (was: jpmml-research.jpg) > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > !jpmml-research.jpg|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: !jpmml-research.jpg|thumbnail! was: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: !jpmml-research.png|thumbnail! > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > !jpmml-research.jpg|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: !jpmml-research.png|thumbnail! was: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: !jpmml-research.jpg|thumbnail! > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > !jpmml-research.png|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Attachment: jpmml-research.jpg > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg)!attachment-name.jpg|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: !jpmml-research.jpg|thumbnail! was: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg)!attachment-name.jpg|thumbnail! > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > Attachments: jpmml-research.jpg > > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > !jpmml-research.jpg|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Attachment: ppmml-design2.jpg > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg)!attachment-name.jpg|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Attachment: (was: ppmml-design2.jpg) > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg)!attachment-name.jpg|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg)!attachment-name.jpg|thumbnail! was: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg) > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg)!attachment-name.jpg|thumbnail! -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-22872) Spark ML Pipeline Model Persistent Support Save Schema Info
[ https://issues.apache.org/jira/browse/SPARK-22872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyanny updated SPARK-22872: --- Description: Hi all, I recently did a research about pmml, and my project needs to transform many models with different type to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg) was: Hi all, I recently did a research about pmml, and my project need to transform many model types to pmml files. Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) provides many tools to do that. I need to provide a uniform API for user, the API arguments are the less the better. I came with a issue that, sklearn, tensorflow, and lightgbm can produce only one model file, including schema info and model data info. but Spark PipelineModel only export a model file in parquet, there is no schema info in the model file. And JPMML-SPARK needs two arguments: Schema and PipelineModel *Can spark PipelineModel include input data schema when export to a file? * I found a solution, use dataframe API to export schema: dataframe.limit(1).write.format("parquet").save("./model.schema") *Are there any solutions to get the PipelineModel input schema?* The situations about machine learning libraries to jpmml are as follows: !http://i65.tinypic.com/2q1dxtd.jpg! > Spark ML Pipeline Model Persistent Support Save Schema Info > --- > > Key: SPARK-22872 > URL: https://issues.apache.org/jira/browse/SPARK-22872 > Project: Spark > Issue Type: IT Help > Components: ML >Affects Versions: 2.2.0 >Reporter: Cyanny >Priority: Minor > > Hi all, > I recently did a research about pmml, and my project needs to transform many > models with different type to pmml files. > Moreover, JPMML(https://github.com/jpmml/jpmml-sparkml) has provided many > tools to do that. I need to provide a uniform API for user, the API arguments > are the less the better. > I came with a issue that, sklearn, tensorflow, and lightgbm can produce only > one model file, including schema info and model data info. > but Spark PipelineModel only export a model file in parquet, there is no > schema info in the model file. And JPMML-SPARK needs two arguments: Schema > and PipelineModel > *Can spark PipelineModel include input data schema when export to a file? * > I found a solution, use dataframe API to export schema: > ```dataframe.limit(1).write.format("parquet").save("./model.schema")``` > *Are there any solutions to get the PipelineModel input schema?* > The situations about machine learning libraries to jpmml are as follows: > ![jpmml](http://i65.tinypic.com/2q1dxtd.jpg) -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org