[
https://issues.apache.org/jira/browse/SPARK-21688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121209#comment-16121209
]
Vincent commented on SPARK-21688:
---------------------------------
currently, there are certain places in ML/MLLib, such as in mllib/SVM, blas
operations (dot, axpy, etc..)are bound with f2j, there is no chance to use
native blas. We understand it was due to performance issue for blas level I api
to go with F2J, but that's mainly because multi-thread native blas issue, with
proper settings, we wont be bothered with such issue. So, maybe we should
change the f2j-binding calls in the current implementation. [~srowen]
> performance improvement in mllib SVM with native BLAS
> ------------------------------------------------------
>
> Key: SPARK-21688
> URL: https://issues.apache.org/jira/browse/SPARK-21688
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 2.2.0
> Environment: 4 nodes: 1 master node, 3 worker nodes
> model name : Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz
> Memory : 180G
> num of core per node: 10
> Reporter: Vincent
> Attachments: mllib svm training.png, svm1.png, svm2.png,
> svm-mkl-1.png, svm-mkl-2.png
>
>
> in current mllib SVM implementation, we found that the CPU is not fully
> utilized, one reason is that f2j blas is set to be used in the HingeGradient
> computation. As we found out earlier
> (https://issues.apache.org/jira/browse/SPARK-21305) that with proper
> settings, native blas is generally better than f2j on the uni-test level,
> here we make the blas operations in SVM go with MKL blas and get an end to
> end performance report showing that in most cases native blas outperformance
> f2j blas up to 50%.
> So, we suggest removing those f2j-fixed calling and going for native blas if
> available. If this proposal is acceptable, we will move on to benchmark other
> algorithms impacted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
