Usage of JDK Vector API in ML/MLLib

2020-12-15 Thread Ludovic Henry
Hello, I’ve, over the past few days, looked into using the new Vector API [1] to accelerate some BLAS operations straight from Java. You can find a gist at [2] containing most of the changes in mllib-local/src/main/scala/org/apache/spark/ml/linalg/BLAS.scala. To measure performance, I’ve a

Re: Usage of JDK Vector API in ML/MLLib

2020-12-15 Thread Sean Owen
Yes it's intriguing, though as you say not readily available in the wild yet. I would also expect native BLAS to outperform f2j also, so yeah that's the interesting question, whether this is a win over native code or not. I suppose the upside is eventually, we may expect this API to be available in

Re: Usage of JDK Vector API in ML/MLLib

2020-12-15 Thread Erik Krogen
Regarding selective compilation, you can hide sources behind a Maven profile such as `-Pvectorized`. Check out what we do to switch between the `hive-1.2` and `hive-2.3` profiles where different source directories are grabbed at compile-time (the hive-1.2 profile was recently removed so you might h