[
https://issues.apache.org/jira/browse/SPARK-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14328358#comment-14328358
]
Xiangrui Meng commented on SPARK-5669:
--------------------------------------
Agree that users should maintain their dependencies. In 1.3, if users want to
use JBLAS to call native routines, there will be two JBLAS jars on the
classpath, one with native libraries from user and one without from Spark. I
don't know what would happen in this case. If we remove JBLAS in 1.4, there
will be only one BLAS jar. Btw, I sent out a PR for SPARK-5814. Maybe it is too
late to merge it into branch-1.3 now, but it should be a good change. Let's ask
[~pwendell].
> Spark assembly includes incompatibly licensed libgfortran, libgcc code via
> JBLAS
> --------------------------------------------------------------------------------
>
> Key: SPARK-5669
> URL: https://issues.apache.org/jira/browse/SPARK-5669
> Project: Spark
> Issue Type: Bug
> Components: Build
> Reporter: Sean Owen
> Assignee: Sean Owen
> Priority: Blocker
> Fix For: 1.3.0
>
>
> Sorry for "Blocker", but it's a license issue. The Spark assembly includes
> the following, from JBLAS:
> {code}
> lib/
> lib/static/
> lib/static/Mac OS X/
> lib/static/Mac OS X/x86_64/
> lib/static/Mac OS X/x86_64/libjblas_arch_flavor.jnilib
> lib/static/Mac OS X/x86_64/sse3/
> lib/static/Mac OS X/x86_64/sse3/libjblas.jnilib
> lib/static/Windows/
> lib/static/Windows/x86/
> lib/static/Windows/x86/libgfortran-3.dll
> lib/static/Windows/x86/libgcc_s_dw2-1.dll
> lib/static/Windows/x86/jblas_arch_flavor.dll
> lib/static/Windows/x86/sse3/
> lib/static/Windows/x86/sse3/jblas.dll
> lib/static/Windows/amd64/
> lib/static/Windows/amd64/libgfortran-3.dll
> lib/static/Windows/amd64/jblas.dll
> lib/static/Windows/amd64/libgcc_s_sjlj-1.dll
> lib/static/Windows/amd64/jblas_arch_flavor.dll
> lib/static/Linux/
> lib/static/Linux/i386/
> lib/static/Linux/i386/sse3/
> lib/static/Linux/i386/sse3/libjblas.so
> lib/static/Linux/i386/libjblas_arch_flavor.so
> lib/static/Linux/amd64/
> lib/static/Linux/amd64/sse3/
> lib/static/Linux/amd64/sse3/libjblas.so
> lib/static/Linux/amd64/libjblas_arch_flavor.so
> {code}
> Unfortunately the libgfortran and libgcc libraries included for Windows are
> not licensed in a way that's compatible with Spark and the AL2 -- LGPL at
> least.
> It's easy to exclude them. I'm not clear what it does to running on Windows;
> I assume it can still work but the libs would have to be made available
> locally and put on the shared library path manually. I don't think there's a
> package manager as in Linux that would make it easily available. I'm not able
> to test on Windows.
> If it doesn't work, the follow-up question is whether that means JBLAS has to
> be removed on the double, or treated as a known issue for 1.3.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
