libgfortran.x86_64 4.1.2-52.el5_8.1 comes with libgfortran.so.1 but
not libgfortran.so.3. JBLAS requires the latter. If you have root
access, you can try to install a newer version of libgfortran.
Otherwise, maybe you can try Spark 1.3, which doesn't use JBLAS in
ALS. -Xiangrui
On Tue, Mar 3, 2015 at 11:21 AM, Prashant Sharma
<sharma.prashant.1...@gmail.com> wrote:
> Hi Folks,
>
> We are trying to run the following code from the spark shell in a CDH 5.3
> cluster running on RHEL 5.8.
>
> spark-shell --master yarn --deploy-mode client --num-executors 15
> --executor-cores 6 --executor-memory 12G
> import org.apache.spark.mllib.recommendation.ALS
> import org.apache.spark.mllib.recommendation.Rating
> val users_item_score_clean =
> sc.textFile("/tmp/spark_mllib_test").map(_.split(","))
> val ratings = users_item_score_clean.map(x=> Rating(x(0).toInt, x(1).toInt,
> x(2).toDouble))
> val rank = 10
> val numIterations = 20
> val alpha = 1.0
> val lambda = 0.01
> val model = ALS.trainImplicit(ratings, rank, numIterations, lambda,alpha)
>
>
>
> We are getting the following error (detailed error is attached in
> error.log):
>
>
> -- org.jblas ERROR Couldn't load copied link file:
> java.lang.UnsatisfiedLinkError:
> /u08/hadoop/yarn/nm/usercache/sharma.p/appcache/application_1425015707226_0128/
> container_e12_1425015707226_0128_01_000010/tmp/jblas7605371780736016929libjblas_arch_flavor.so:
> libgfortran.so.3:
> cannot open shared object file: No such file or directory.
>
> On Linux 64bit, you need additional support libraries.
> You need to install libgfortran3.
>
> For example for debian or Ubuntu, type "sudo apt-get install libgfortran3"
>
> For more information, see
> https://github.com/mikiobraun/jblas/wiki/Missing-Libraries
> 15/03/02 14:50:25 ERROR executor.Executor: Exception in task 22.0 in stage
> 6.0 (TID 374)
> java.lang.UnsatisfiedLinkError: org.jblas.NativeBlas.dposv(CII[DII[DII)I
> at org.jblas.NativeBlas.dposv(Native Method)
> at org.jblas.SimpleBlas.posv(SimpleBlas.java:369)
>
>
> This exact code runs fine on another CDH 5.3 cluster which runs on RHEL 6.5.
>
> "libgfortran.so.3" is not present on the problematic cluster.
>
> [root@node04 ~]# find / -name libgfortran.so.3 2>/dev/null
>
>
> I am able to find "libgfortran.so.3 " on the cluster where the above code
> works:
>
> [root@workingclusternode04 ~]# find / -name libgfortran.so.3 2>/dev/null
> /usr/lib64/libgfortran.so.3
>
>
>
> The following output shows that the fortran packages are installed on both
> the clusters:
>
> On the cluster where this is not working
>
> [root@node04 ~]# yum list | grep -i fortran
> gcc-gfortran.x86_64 4.1.2-52.el5_8.1 installed
> libgfortran.i386 4.1.2-52.el5_8.1 installed
> libgfortran.x86_64 4.1.2-52.el5_8.1 installed
>
>
> On the cluster where the spark job is this working
>
> [root@ workingclusternode04 ~]# yum list | grep -i fortran
> Repository 'bda' is missing name in configuration, using id
> compat-libgfortran-41.x86_64 4.1.2-39.el6 @bda
> gcc-gfortran.x86_64 4.4.7-4.el6 @bda
> libgfortran.x86_64 4.4.7-4.el6 @bda
>
>
> Has anybody run into this? Any pointers are much appreciated.
>
> Regards,
> Prashant
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
