Not of course...all machines in HDInsight are Windows 64bit server. And I have made sure all my DLLs are for 64bit machines. I have managed to get those DLLs loade on my local machine which is also Windows 64bit.
[image: --] Xi Shen [image: http://]about.me/davidshen <http://about.me/davidshen?promo=email_sig> <http://about.me/davidshen> On Thu, Mar 26, 2015 at 11:11 AM, DB Tsai <dbt...@dbtsai.com> wrote: > Are you deploying the windows dll to linux machine? > > Sincerely, > > DB Tsai > ------------------------------------------------------- > Blog: https://www.dbtsai.com > > > On Wed, Mar 25, 2015 at 3:57 AM, Xi Shen <davidshe...@gmail.com> wrote: > > I think you meant to use the "--files" to deploy the DLLs. I gave a try, > but > > it did not work. > > > > From the Spark UI, Environment tab, I can see > > > > spark.yarn.dist.files > > > > > file:/c:/openblas/libgcc_s_seh-1.dll,file:/c:/openblas/libblas3.dll,file:/c:/openblas/libgfortran-3.dll,file:/c:/openblas/liblapack3.dll,file:/c:/openblas/libquadmath-0.dll > > > > I think my DLLs are all deployed. But I still got the warn message that > > native BLAS library cannot be load. > > > > And idea? > > > > > > Thanks, > > David > > > > > > On Wed, Mar 25, 2015 at 5:40 AM DB Tsai <dbt...@dbtsai.com> wrote: > >> > >> I would recommend to upload those jars to HDFS, and use add jars > >> option in spark-submit with URI from HDFS instead of URI from local > >> filesystem. Thus, it can avoid the problem of fetching jars from > >> driver which can be a bottleneck. > >> > >> Sincerely, > >> > >> DB Tsai > >> ------------------------------------------------------- > >> Blog: https://www.dbtsai.com > >> > >> > >> On Tue, Mar 24, 2015 at 4:13 AM, Xi Shen <davidshe...@gmail.com> wrote: > >> > Hi, > >> > > >> > I am doing ML using Spark mllib. However, I do not have full control > to > >> > the > >> > cluster. I am using Microsoft Azure HDInsight > >> > > >> > I want to deploy the BLAS or whatever required dependencies to > >> > accelerate > >> > the computation. But I don't know how to deploy those DLLs when I > submit > >> > my > >> > JAR to the cluster. > >> > > >> > I know how to pack those DLLs into a jar. The real challenge is how to > >> > let > >> > the system find them... > >> > > >> > > >> > Thanks, > >> > David > >> > >
