[ 
https://issues.apache.org/jira/browse/SPARK-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph K. Bradley updated SPARK-15526:
--------------------------------------
    Comment: was deleted

(was: [~srowen] I just found that PMML is now listed as a dependency in a lot 
(all?) of the Spark modules: catalyst, mllib-local, etc.  I haven't yet tested 
to see if these 2 PRs caused that change, but it seems likely.

Update: Maybe I built it incorrectly...I don't see this issue in the artifacts 
from RC4.)

> Shade JPMML
> -----------
>
>                 Key: SPARK-15526
>                 URL: https://issues.apache.org/jira/browse/SPARK-15526
>             Project: Spark
>          Issue Type: Dependency upgrade
>          Components: ML, MLlib
>    Affects Versions: 2.0.0
>            Reporter: Villu Ruusmann
>            Assignee: Sean Owen
>            Priority: Minor
>             Fix For: 2.3.0
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> The Spark-MLlib module depends on the JPMML-Model library 
> (org.jpmml:pmml-model:1.2.7) for its PMML export capabilities. The 
> JPMML-Model library is included in the Apache Spark assembly, which makes it 
> very difficult to build and deploy competing PMML exporters that may wish to 
> depend on different versions (typically much newer) of the same library.
> JPMML-Model library classes are not part of Apache Spark public APIs, so it 
> shouldn't be a problem if they are relocated by prepending a prefix 
> "org.spark_project" to their package names using Maven Shade Plugin. The 
> requested treatment is identical to how Google Guava and Jetty dependencies 
> are shaded in the final assembly.
> This issue is raised in relation to the JPMML-SparkML project 
> (https://github.com/jpmml/jpmml-sparkml), which provides PMML export 
> capabilities for Spark ML Pipelines. Currently, application developers who 
> wish to use it must tweak their application classpath, which assumes 
> familiarity with build internals.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to