Re: Export BLAS module on Spark MLlib

2015-11-30 Thread Burak Yavuz
Or you could also use reflection like in this Spark Package: https://github.com/brkyvz/lazy-linalg/blob/master/src/main/scala/com/brkyvz/spark/linalg/BLASUtils.scala Best, Burak On Mon, Nov 30, 2015 at 12:48 PM, DB Tsai wrote: > The workaround is have your code in the same

Re: Export BLAS module on Spark MLlib

2015-11-30 Thread DB Tsai
The workaround is have your code in the same package, or write some utility wrapper in the same package so you can use them in your code. Mostly we implement those BLAS for our own need, and we don't have general use-case in mind. As a result, if we open them up prematurely, it will add our api

Re: Export BLAS module on Spark MLlib

2015-11-30 Thread DB Tsai
I used reflection initially, but I found it's very slow especially in a tight loop. Maybe caching the reflection can help which I never try. Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0xAF08DF8D On Mon, Nov 30, 2015 at

Export BLAS module on Spark MLlib

2015-11-28 Thread Sasaki Kai
Hello I'm developing a Spark package that manipulates Vector and Matrix for machine learning. This package uses mllib.linalg.Vector and mllib.linalg.Matrix in order to achieve compatible interface to mllib itself. But mllib.linalg.BLAS module is private inside spark package. We cannot use