[
https://issues.apache.org/jira/browse/MAHOUT-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938552#comment-13938552
]
Hudson commented on MAHOUT-1346:
--------------------------------
SUCCESS: Integrated in Mahout-Quality #2526 (See
[https://builds.apache.org/job/Mahout-Quality/2526/])
MAHOUT-1346: (D)-SPCA and other additions and fixes.
Squashed commit of the following:
commit 1ed6267a3cc550d1d648346fca7e152c63909667
Merge: 95da094 c0260b5
Author: Dmitriy Lyubimov <[email protected]>
Date: Mon Mar 17 11:38:33 2014 -0700
Merge branch 'trunk' into dev-1.0-spark
commit 95da094a38d7fbbd654e95bbfb5723c5e1e48c55
Author: Dmitriy Lyubimov <[email protected]>
Date: Mon Mar 17 11:24:28 2014 -0700
D-SPCA fixes. test is passing (in-core and out-of-core tests give the same
result). However, the test seems to be still rank-deficient (rank=20 instead of
desired 100). Not sure why -- the Cholesky seems to be too sensitive.
commit c907be0f9b35436cf623e1a92941f924382b531d
Author: Dmitriy Lyubimov <[email protected]>
Date: Mon Mar 17 00:10:05 2014 -0700
Fixes for D-SPCA: q=0 works, q>0 still doesn't
commit 0e24fe88cbbae2db57c06b4d85f5df1b5a2925e5
Author: Dmitriy Lyubimov <[email protected]>
Date: Sun Mar 16 18:16:27 2014 -0700
added test for colSums, colMeans
commit 1012d65ba360cfe944f2f559cf032c7546e19072
Author: Dmitriy Lyubimov <[email protected]>
Date: Sun Mar 16 18:08:25 2014 -0700
D-SPCA WIP. Test is not yet working with -q0
commit 5560f1e928380280b23dbd35efec39b8566eb1a3
Author: Dmitriy Lyubimov <[email protected]>
Date: Sun Mar 9 18:12:03 2014 -0700
Fixng random gen in SPCA test codegen
commit 740cac1eec6a4e1bf256066d86b7b5681f88bd4c
Author: Dmitriy Lyubimov <[email protected]>
Date: Sun Mar 9 18:08:30 2014 -0700
removing check for rank deficiency (so pca can complete). User can check
the results for that if needed.
Adding in-core s-pca test.
commit f1abfe430987b60a62a0e65cf234ebdc40135275
Author: Dmitriy Lyubimov <[email protected]>
Date: Sun Mar 9 16:19:32 2014 -0700
perhaps a better ssvd test assertion
commit e26596fd6cecb74055c38d838a8b02e9698b17f8
Author: Dmitriy Lyubimov <[email protected]>
Date: Sat Mar 8 22:01:58 2014 -0800
first write-up of in-core stochastic PCA
commit eb3bf98d6abcd8f480e892c74bd7f61b32b33bdf
Author: Dmitriy Lyubimov <[email protected]>
Date: Sat Mar 8 20:35:24 2014 -0800
rowsums, colsums, rowmeans, colmeans for in-core + tests (dlyubimov: rev
1578527)
* /mahout/trunk/math-scala/pom.xml
*
/mahout/trunk/math-scala/src/main/scala/org/apache/mahout/math/scalabindings/MatrixOps.scala
*
/mahout/trunk/math-scala/src/main/scala/org/apache/mahout/math/scalabindings/SSVD.scala
*
/mahout/trunk/math-scala/src/main/scala/org/apache/mahout/math/scalabindings/VectorOps.scala
*
/mahout/trunk/math-scala/src/main/scala/org/apache/mahout/math/scalabindings/package.scala
*
/mahout/trunk/math-scala/src/test/scala/org/apache/mahout/math/scalabindings/MathSuite.scala
*
/mahout/trunk/math-scala/src/test/scala/org/apache/mahout/math/scalabindings/MatlabLikeMatrixOpsSuite.scala
*
/mahout/trunk/math-scala/src/test/scala/org/apache/mahout/math/scalabindings/MatrixOpsSuite.scala
*
/mahout/trunk/math-scala/src/test/scala/org/apache/mahout/math/scalabindings/RLikeMatrixOpsSuite.scala
*
/mahout/trunk/math-scala/src/test/scala/org/apache/mahout/math/scalabindings/RLikeVectorOpsSuite.scala
*
/mahout/trunk/math-scala/src/test/scala/org/apache/mahout/math/scalabindings/VectorOpsSuite.scala
* /mahout/trunk/math-scala/src/test/scala/org/apache/mahout/test
*
/mahout/trunk/math-scala/src/test/scala/org/apache/mahout/test/LoggerConfiguration.scala
*
/mahout/trunk/math-scala/src/test/scala/org/apache/mahout/test/MahoutSuite.scala
* /mahout/trunk/spark/pom.xml
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AinCoreB.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AtB.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/blas/Ax.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrm.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrmBase.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedOps.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/DrmLike.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/RLikeDrmOps.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/decompositions/DQR.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/decompositions/DSPCA.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/decompositions/DSSVD.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/package.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/CheckpointAction.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAewB.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAewScalar.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAt.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAtAnyKey.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAtx.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpAx.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpMapBlock.scala
*
/mahout/trunk/spark/src/main/scala/org/apache/mahout/sparkbindings/drm/plan/OpRowRange.scala
*
/mahout/trunk/spark/src/test/scala/org/apache/mahout/sparkbindings/drm/DrmLikeOpsSuite.scala
*
/mahout/trunk/spark/src/test/scala/org/apache/mahout/sparkbindings/drm/RLikeDrmOpsSuite.scala
*
/mahout/trunk/spark/src/test/scala/org/apache/mahout/sparkbindings/drm/decompositions/MathSuite.scala
*
/mahout/trunk/spark/src/test/scala/org/apache/mahout/sparkbindings/test/LoggerConfiguration.scala
*
/mahout/trunk/spark/src/test/scala/org/apache/mahout/sparkbindings/test/MahoutLocalContext.scala
> Spark Bindings (DRM)
> --------------------
>
> Key: MAHOUT-1346
> URL: https://issues.apache.org/jira/browse/MAHOUT-1346
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.9
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Fix For: 1.0
>
> Attachments: ScalaSparkBindings.pdf
>
>
> Spark bindings for Mahout DRM.
> DRM DSL.
> Disclaimer. This will all be experimental at this point.
> The idea is to wrap DRM by Spark RDD with support of some basic
> functionality, perhaps some humble beginning of Cost-based optimizer
> (0) Spark serialization support for Vector, Matrix
> (1) Bagel transposition
> (2) slim X'X
> (2a) not-so-slim X'X
> (3) blockify() (compose RDD containing vertical blocks of original input)
> (4) read/write Mahout DRM off HDFS
> (5) A'B
> ...
--
This message was sent by Atlassian JIRA
(v6.2#6252)
