Hi,
If I take the Movielens data and run the default ALS with regularization as
0.0, I am hitting exception from LAPACK that the gram matrix is not
positive definite. This is on the master branch.
This is how I run it :
./bin/spark-submit --total-executor-cores 1 --master spark://
tusca09lmlvt00c.uswin.ad.vzwcorp.com:7077 --jars
/Users/v606014/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar
--class org.apache.spark.examples.mllib.MovieLensALS
./examples/target/spark-examples_2.10-1.1.0-SNAPSHOT.jar --rank 20
--numIterations 20 --lambda 0.0 --kryo
hdfs://localhost:8020/sandbox/movielens/
Error from LAPACK:
WARN TaskSetManager: Lost task 0.0 in stage 11.0 (TID 22,
tusca09lmlvt00c.uswin.ad.vzwcorp.com):
org.jblas.exceptions.LapackArgumentException: LAPACK DPOSV: Leading minor
of order i of A is not positive definite.
>From the maths it's not expected right ?
||r - wi'hj||^{2} has to be positive definite...
I think the tests are not running any 0.0 regularization tests otherwise we
should have caught it as well...
For the sparse coding NMF variant that I am running, I have to turn off L2
regularization when I run a L1 on products to extract sparse topics...
Thanks.
Deb
