[
https://issues.apache.org/jira/browse/MAHOUT-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850541#comment-15850541
]
Andrew Palumbo commented on MAHOUT-1934:
----------------------------------------
ok so VCL jars are picked up in {{mahout-shell in local}} mode:
{code}
$MASTER=local[2] mahout spark-shell
mahout> :load
/home/andy/sandbox/mahout/examples/bin/SparseSparseDrmTimer.mscala
Loading /home/andy/sandbox/mahout/examples/bin/SparseSparseDrmTimer.mscala...
timeSparseDRMMMul: (m: Int, n: Int, s: Int, para: Int, pctDense: Double, seed:
Long)Long
mahout> timeSparseDRMMMul(500, 500, 500, 10)
[Stage 2:=============================> (5 + 2) /
10][INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
[INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
[WARN] Unable to create class GPUMMul: attempting OpenMP version
[INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
[WARN] Unable to create class GPUMMul: attempting OpenMP version
[INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
[INFO] Successfully created org.apache.mahout.viennacl.openmp.OMPMMul solver
[INFO] Successfully created org.apache.mahout.viennacl.openmp.OMPMMul solver
ompRWCW
ompRWCW
{code}
the {{ompRWCW}} "trace" (println) is confirmation that the Drm Matrix
Multiplication is executed by OpenMP.
> OpenMP jars aren't being picked up in distributed Spark
> -------------------------------------------------------
>
> Key: MAHOUT-1934
> URL: https://issues.apache.org/jira/browse/MAHOUT-1934
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.13.0
> Reporter: Trevor Grant
> Assignee: Andrew Palumbo
> Priority: Blocker
> Fix For: 0.13.0
>
>
> When executing the following:
> ```
> import org.apache.mahout.math.decompositions._
> drmX.checkpoint()
> val (U, V, s) = dspca(drmX, k = 30, q = 1)
> ```
> I see the following:
> ```
> import org.apache.mahout.math.decompositions._
> res1: org.apache.mahout.math.drm.CheckpointedDrm[Int] =
> org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@2e8dd198
> [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
> [WARN] Unable to create class GPUMMul: attempting OpenMP version
> [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
> org.apache.mahout.viennacl.openmp.OMPMMul$
> [INFO] Unable to create class OMPMMul: falling back to java version
> drmReduxX: org.apache.mahout.math.drm.DrmLike[Int] =
> OpMapBlock(OpTimesRightMatrix(org.apache.mahout.s...
> ```
> The code executes fine, but it seems the OpenMP is not getting picked up.
> To be clear I build these jars with -Pviennacl-omp
> I the issue persists through other operations as well. I am running on a 3
> node Yarn cluster/ spark 1.6.3
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
