Re: Regarding Online Recommenders

[Sebastian Schelter]

Hi Gokhan,

I like your proposals and I think this is an important discussion. Peng
is also interested in working on online recommenders, so we should try
to team up our efforts. I'd like to extend the discussion a little to
related API changes, that I think are necessary.

What do you think about completely removing the setPreference() and
removePreference() methods from Recommender? I think they don't belong
there for two reasons: First,  they duplicate functionality from
DataModel and second, a lot of recommenders are read-only/train-once and
cannot handle single preference updates anyway.

I think we should have a DataModel implementation that can be updated
and an online learning recommender should be able to register to be
notified with updates.

We should further more split up the DataModel interface into a hierarchy
of three parts:

First, a simple readonly interface that allows sequential access to the
data (similar to FactorizablePreferences). This allows us to create
memory efficient implementations. E.g. Cheng reported in MAHOUT-1272
that the current DataModel needs 12GB heap for the Netflix dataset (100M
ratings) which is unacceptable. I was able to fit the KDD Music dataset
(250M ratings) into 3GB with FactorizablePreferences.

The second interface would extend the readonly interface and should
resemble what DataModel is today: An easy-to-use in-memory
implementation that trades high memory consumption for convenient random
access.

And finally the third interface would extend the second and provide
tooling for online updates of the data.

What do you think of that? Does it sound reasonable?

--sebastian


> The DataModel I imagine would follow the current API, where underlying
> preference storage is replaced with a matrix.
> 
> A Recommender would then use the DataModel and the OnlineLearner, where
> Recommender#setPreference is delegated to DataModel#setPreference (like it
> does now), and DataModel#setPreference triggers OnlineLearner#train.

[jira] [Commented] (MAHOUT-1272) Parallel SGD matrix factorizer for SVDrecommender

[Sebastian Schelter (JIRA)]

[ 
https://issues.apache.org/jira/browse/MAHOUT-1272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13709456#comment-13709456
 ] 

Sebastian Schelter commented on MAHOUT-1272:


I think we should rework the datamodel first. It makes no sense to have to 
allocate 12GB heap for a 1GB dataset.



> Parallel SGD matrix factorizer for SVDrecommender
> -
>
> Key: MAHOUT-1272
> URL: https://issues.apache.org/jira/browse/MAHOUT-1272
> Project: Mahout
>  Issue Type: New Feature
>  Components: Collaborative Filtering
>Reporter: Peng Cheng
>Assignee: Sean Owen
>  Labels: features, patch, test
> Fix For: 0.8
>
> Attachments: GroupLensSVDRecomenderEvaluatorRunner.java, 
> libimsetiSVDRecomenderEvaluatorRunner.java, mahout.patch, 
> NetflixRecomenderEvaluatorRunner.java, ParallelSGDFactorizer.java, 
> ParallelSGDFactorizer.java, ParallelSGDFactorizerTest.java, 
> ParallelSGDFactorizerTest.java
>
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>
> a parallel factorizer based on MAHOUT-1089 may achieve better performance on 
> multicore processor.
> existing code is single-thread and perhaps may still be outperformed by the 
> default ALS-WR.
> In addition, its hardcoded online-to-batch-conversion prevents it to be used 
> by an online recommender. An online SGD implementation may help build 
> high-performance online recommender as a replacement of the outdated 
> slope-one.
> The new factorizer can implement either DSGD 
> (http://www.mpi-inf.mpg.de/~rgemulla/publications/gemulla11dsgd.pdf) or 
> hogwild! (www.cs.wisc.edu/~brecht/papers/hogwildTR.pdf).
> Related discussion has been carried on for a while but remain inconclusive:
> http://web.archiveorange.com/archive/v/z6zxQUSahofuPKEzZkzl

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Cholesky test failures...

[Ted Dunning]

On Mon, Jul 15, 2013 at 5:16 PM, Dmitriy Lyubimov  wrote:

> On Mon, Jul 15, 2013 at 5:13 PM, Ted Dunning 
> wrote:
>
> > It sounds like you missed the part of the API contract that says that you
> > have to fix the bugs before using the code.
> >
> > :-)
> >
> > More seriously, with pivot=true, the result can be hard to understand
> > because they are permuted.  Could that be the problem?
> >
>
> ok, but solveLeft() in this case should still work as expected? or not ? I
> don't understand the "permuted" part -- i guess it is some sort of indexing
> overlay over L matrix -- but whatever solution is, shouldn't the api be
> agnostic of the method used?
>

Yes.  It should.  And there should be a test case that proves it.

I am surprised that there isn't.

The idea behind the pivoting is that you get AP = QR so

(AP)'(AP) = R'R = P' A'A P

thus

A' A = P R' R P' = (R P')' (R P')

What you get in place of R is RP' (R with columns rearranged).

Re: Cholesky test failures...

[Dmitriy Lyubimov]

On Mon, Jul 15, 2013 at 5:13 PM, Ted Dunning  wrote:

> It sounds like you missed the part of the API contract that says that you
> have to fix the bugs before using the code.
>
> :-)
>
> More seriously, with pivot=true, the result can be hard to understand
> because they are permuted.  Could that be the problem?
>

ok, but solveLeft() in this case should still work as expected? or not ? I
don't understand the "permuted" part -- i guess it is some sort of indexing
overlay over L matrix -- but whatever solution is, shouldn't the api be
agnostic of the method used?


>
>
> On Mon, Jul 15, 2013 at 5:11 PM, Dmitriy Lyubimov 
> wrote:
>
> > Hm.
> >
> > if i specify pivoted=false, everything works.
> > In addition it seems i have created singular input by chance, but making
> it
> > non-singular
> >
> >
> >  val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5.5))
> >
> >  still doesn't help with pivoted=true (which is default) , my test fails.
> >
> > do pivoted results require some special handling?
> >
> >
> >
> >
> > On Mon, Jul 15, 2013 at 5:03 PM, Dmitriy Lyubimov 
> > wrote:
> >
> > > should read
> > >
> > > val axmb = (a %*% x) - b
> > >
> > > of course but it doesn't make difference, the Cholesky output doesn't
> > make
> > > sense to me even before that
> > >
> > >
> > > On Mon, Jul 15, 2013 at 5:00 PM, Dmitriy Lyubimov  > >wrote:
> > >
> > >> Hi Ted,
> > >>
> > >> I am getting Cholesky test failures when trying to solve of Ax=B
> > >>
> > >> The L matrix and solveLeft() output do not make much sense to me. For
> > >> once, L doesn't even have the expected L-shape:
> > >>
> > >> Do you have an idea where i go wrong? (the test is wrapped into scala
> > DSL
> > >> but it is Mahout's cholesky underwraps) .
> > >>
> > >> test code:
> > >>
> > >>   test("chol") {
> > >>
> > >> // try to solve Ax=b with cholesky:
> > >> // this requires
> > >> // (LL')x = B
> > >> // L'x= (L^-1)B
> > >> // x=(L'^-1)(L^-1)B
> > >>
> > >> val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5))
> > >>
> > >> // make sure it is symmetric for a valid solution
> > >> a := a.t %*% a
> > >>
> > >> printf("A= \n%s\n", a)
> > >>
> > >> val b = dense((9, 8, 7)).t
> > >>
> > >> printf ("b = \n%s\n", b)
> > >>
> > >> val ch = chol(a)
> > >>
> > >> printf ("L = \n%s\n", ch.getL)
> > >>
> > >> printf ( "(L^-1)b =\n%s\n", ch.solveLeft(b))
> > >>
> > >> val x = ch.solveRight(diag(1,3)) %*% ch.solveLeft(b)
> > >>
> > >> printf("x = \n%s\n", x.toString)
> > >>
> > >> val axmb = (a %*% b) - b
> > >>
> > >> printf("AX - B = \n%s\n", axmb.toString)
> > >>
> > >> assert(axmb.norm < 1e-10)
> > >>
> > >>   }
> > >>
> > >>
> > >>
> > >> Output:
> > >>
> > >> A=
> > >> {
> > >>   0  => {0:14.0,1:20.0,2:26.0}
> > >>   1  => {0:20.0,1:29.0,2:38.0}
> > >>   2  => {0:26.0,1:38.0,2:50.0}
> > >> }
> > >> b =
> > >> {
> > >>   0  => {0:9.0}
> > >>   1  => {0:8.0}
> > >>   2  => {0:7.0}
> > >> }
> > >>
> > >> L =
> > >> {
> > >>   0  => {0:0.6928203230275511,2:3.676955262170047}
> > >>   1  => {0:0.3464101615137781,2:5.374011537017761}
> > >>   2  => {2:7.0710678118654755}
> > >> }
> > >> (L^-1)b =
> > >> {
> > >>   0  => {0:1.2727922061357855}
> > >>   1  => {0:11.547005383792511}
> > >>   2  => {}
> > >> }
> > >> X =
> > >> {
> > >>   0  => {0:0.18}
> > >>   1  => {0:5.119661282874144}
> > >>   2  => {}
> > >> }
> > >> AX - B =
> > >> {
> > >>   0  => {0:459.0}
> > >>   1  => {0:670.0}
> > >>   2  => {0:881.0}
> > >> }
> > >>
> > >> org.scalatest.exceptions.TestFailedException was thrown.
> > >>
> > >>
> > >
> >
>

Re: Cholesky test failures...

[Dmitriy Lyubimov]

On Mon, Jul 15, 2013 at 5:11 PM, Ted Dunning  wrote:

> Does it help to know that the matrix A has rank 2 instead of 3?
>

yes, already figured that in my previous email, but full-rank doesn't work
with pivoted=true

>
> > chol(a)
> > Error in chol.default(a) :
> >   the leading minor of order 3 is not positive definite
> > > qr.R(qr(a))
> >   [,1][,2]  [,3]
> > [1,] -35.66511 -51.8153470 -6.796559e+01
> > [2,]   0.0  -0.4120817 -8.241634e-01
> > [3,]   0.0   0.000  4.218847e-15
> > > svd(a)$d
> > [1] 9.261128e+01 3.887216e-01 5.102882e-15
>
>
> I don't remember if I implemented a pivoting Cholesky or not.
>

ok so pivoted=true is the default and if it doesn't work, it probably
shouldn't be the default..


>
>
> On Mon, Jul 15, 2013 at 5:03 PM, Dmitriy Lyubimov 
> wrote:
>
> > should read
> >
> > val axmb = (a %*% x) - b
> >
> > of course but it doesn't make difference, the Cholesky output doesn't
> make
> > sense to me even before that
> >
> >
> > On Mon, Jul 15, 2013 at 5:00 PM, Dmitriy Lyubimov 
> > wrote:
> >
> > > Hi Ted,
> > >
> > > I am getting Cholesky test failures when trying to solve of Ax=B
> > >
> > > The L matrix and solveLeft() output do not make much sense to me. For
> > > once, L doesn't even have the expected L-shape:
> > >
> > > Do you have an idea where i go wrong? (the test is wrapped into scala
> DSL
> > > but it is Mahout's cholesky underwraps) .
> > >
> > > test code:
> > >
> > >   test("chol") {
> > >
> > > // try to solve Ax=b with cholesky:
> > > // this requires
> > > // (LL')x = B
> > > // L'x= (L^-1)B
> > > // x=(L'^-1)(L^-1)B
> > >
> > > val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5))
> > >
> > > // make sure it is symmetric for a valid solution
> > > a := a.t %*% a
> > >
> > > printf("A= \n%s\n", a)
> > >
> > > val b = dense((9, 8, 7)).t
> > >
> > > printf ("b = \n%s\n", b)
> > >
> > > val ch = chol(a)
> > >
> > > printf ("L = \n%s\n", ch.getL)
> > >
> > > printf ( "(L^-1)b =\n%s\n", ch.solveLeft(b))
> > >
> > > val x = ch.solveRight(diag(1,3)) %*% ch.solveLeft(b)
> > >
> > > printf("x = \n%s\n", x.toString)
> > >
> > > val axmb = (a %*% b) - b
> > >
> > > printf("AX - B = \n%s\n", axmb.toString)
> > >
> > > assert(axmb.norm < 1e-10)
> > >
> > >   }
> > >
> > >
> > >
> > > Output:
> > >
> > > A=
> > > {
> > >   0  => {0:14.0,1:20.0,2:26.0}
> > >   1  => {0:20.0,1:29.0,2:38.0}
> > >   2  => {0:26.0,1:38.0,2:50.0}
> > > }
> > > b =
> > > {
> > >   0  => {0:9.0}
> > >   1  => {0:8.0}
> > >   2  => {0:7.0}
> > > }
> > >
> > > L =
> > > {
> > >   0  => {0:0.6928203230275511,2:3.676955262170047}
> > >   1  => {0:0.3464101615137781,2:5.374011537017761}
> > >   2  => {2:7.0710678118654755}
> > > }
> > > (L^-1)b =
> > > {
> > >   0  => {0:1.2727922061357855}
> > >   1  => {0:11.547005383792511}
> > >   2  => {}
> > > }
> > > X =
> > > {
> > >   0  => {0:0.18}
> > >   1  => {0:5.119661282874144}
> > >   2  => {}
> > > }
> > > AX - B =
> > > {
> > >   0  => {0:459.0}
> > >   1  => {0:670.0}
> > >   2  => {0:881.0}
> > > }
> > >
> > > org.scalatest.exceptions.TestFailedException was thrown.
> > >
> > >
> >
>

Re: Cholesky test failures...

[Ted Dunning]

It sounds like you missed the part of the API contract that says that you
have to fix the bugs before using the code.

:-)

More seriously, with pivot=true, the result can be hard to understand
because they are permuted.  Could that be the problem?



On Mon, Jul 15, 2013 at 5:11 PM, Dmitriy Lyubimov  wrote:

> Hm.
>
> if i specify pivoted=false, everything works.
> In addition it seems i have created singular input by chance, but making it
> non-singular
>
>
>  val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5.5))
>
>  still doesn't help with pivoted=true (which is default) , my test fails.
>
> do pivoted results require some special handling?
>
>
>
>
> On Mon, Jul 15, 2013 at 5:03 PM, Dmitriy Lyubimov 
> wrote:
>
> > should read
> >
> > val axmb = (a %*% x) - b
> >
> > of course but it doesn't make difference, the Cholesky output doesn't
> make
> > sense to me even before that
> >
> >
> > On Mon, Jul 15, 2013 at 5:00 PM, Dmitriy Lyubimov  >wrote:
> >
> >> Hi Ted,
> >>
> >> I am getting Cholesky test failures when trying to solve of Ax=B
> >>
> >> The L matrix and solveLeft() output do not make much sense to me. For
> >> once, L doesn't even have the expected L-shape:
> >>
> >> Do you have an idea where i go wrong? (the test is wrapped into scala
> DSL
> >> but it is Mahout's cholesky underwraps) .
> >>
> >> test code:
> >>
> >>   test("chol") {
> >>
> >> // try to solve Ax=b with cholesky:
> >> // this requires
> >> // (LL')x = B
> >> // L'x= (L^-1)B
> >> // x=(L'^-1)(L^-1)B
> >>
> >> val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5))
> >>
> >> // make sure it is symmetric for a valid solution
> >> a := a.t %*% a
> >>
> >> printf("A= \n%s\n", a)
> >>
> >> val b = dense((9, 8, 7)).t
> >>
> >> printf ("b = \n%s\n", b)
> >>
> >> val ch = chol(a)
> >>
> >> printf ("L = \n%s\n", ch.getL)
> >>
> >> printf ( "(L^-1)b =\n%s\n", ch.solveLeft(b))
> >>
> >> val x = ch.solveRight(diag(1,3)) %*% ch.solveLeft(b)
> >>
> >> printf("x = \n%s\n", x.toString)
> >>
> >> val axmb = (a %*% b) - b
> >>
> >> printf("AX - B = \n%s\n", axmb.toString)
> >>
> >> assert(axmb.norm < 1e-10)
> >>
> >>   }
> >>
> >>
> >>
> >> Output:
> >>
> >> A=
> >> {
> >>   0  => {0:14.0,1:20.0,2:26.0}
> >>   1  => {0:20.0,1:29.0,2:38.0}
> >>   2  => {0:26.0,1:38.0,2:50.0}
> >> }
> >> b =
> >> {
> >>   0  => {0:9.0}
> >>   1  => {0:8.0}
> >>   2  => {0:7.0}
> >> }
> >>
> >> L =
> >> {
> >>   0  => {0:0.6928203230275511,2:3.676955262170047}
> >>   1  => {0:0.3464101615137781,2:5.374011537017761}
> >>   2  => {2:7.0710678118654755}
> >> }
> >> (L^-1)b =
> >> {
> >>   0  => {0:1.2727922061357855}
> >>   1  => {0:11.547005383792511}
> >>   2  => {}
> >> }
> >> X =
> >> {
> >>   0  => {0:0.18}
> >>   1  => {0:5.119661282874144}
> >>   2  => {}
> >> }
> >> AX - B =
> >> {
> >>   0  => {0:459.0}
> >>   1  => {0:670.0}
> >>   2  => {0:881.0}
> >> }
> >>
> >> org.scalatest.exceptions.TestFailedException was thrown.
> >>
> >>
> >
>

Re: Cholesky test failures...

[Ted Dunning]

Does it help to know that the matrix A has rank 2 instead of 3?

> chol(a)
> Error in chol.default(a) :
>   the leading minor of order 3 is not positive definite
> > qr.R(qr(a))
>   [,1][,2]  [,3]
> [1,] -35.66511 -51.8153470 -6.796559e+01
> [2,]   0.0  -0.4120817 -8.241634e-01
> [3,]   0.0   0.000  4.218847e-15
> > svd(a)$d
> [1] 9.261128e+01 3.887216e-01 5.102882e-15


I don't remember if I implemented a pivoting Cholesky or not.



On Mon, Jul 15, 2013 at 5:03 PM, Dmitriy Lyubimov  wrote:

> should read
>
> val axmb = (a %*% x) - b
>
> of course but it doesn't make difference, the Cholesky output doesn't make
> sense to me even before that
>
>
> On Mon, Jul 15, 2013 at 5:00 PM, Dmitriy Lyubimov 
> wrote:
>
> > Hi Ted,
> >
> > I am getting Cholesky test failures when trying to solve of Ax=B
> >
> > The L matrix and solveLeft() output do not make much sense to me. For
> > once, L doesn't even have the expected L-shape:
> >
> > Do you have an idea where i go wrong? (the test is wrapped into scala DSL
> > but it is Mahout's cholesky underwraps) .
> >
> > test code:
> >
> >   test("chol") {
> >
> > // try to solve Ax=b with cholesky:
> > // this requires
> > // (LL')x = B
> > // L'x= (L^-1)B
> > // x=(L'^-1)(L^-1)B
> >
> > val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5))
> >
> > // make sure it is symmetric for a valid solution
> > a := a.t %*% a
> >
> > printf("A= \n%s\n", a)
> >
> > val b = dense((9, 8, 7)).t
> >
> > printf ("b = \n%s\n", b)
> >
> > val ch = chol(a)
> >
> > printf ("L = \n%s\n", ch.getL)
> >
> > printf ( "(L^-1)b =\n%s\n", ch.solveLeft(b))
> >
> > val x = ch.solveRight(diag(1,3)) %*% ch.solveLeft(b)
> >
> > printf("x = \n%s\n", x.toString)
> >
> > val axmb = (a %*% b) - b
> >
> > printf("AX - B = \n%s\n", axmb.toString)
> >
> > assert(axmb.norm < 1e-10)
> >
> >   }
> >
> >
> >
> > Output:
> >
> > A=
> > {
> >   0  => {0:14.0,1:20.0,2:26.0}
> >   1  => {0:20.0,1:29.0,2:38.0}
> >   2  => {0:26.0,1:38.0,2:50.0}
> > }
> > b =
> > {
> >   0  => {0:9.0}
> >   1  => {0:8.0}
> >   2  => {0:7.0}
> > }
> >
> > L =
> > {
> >   0  => {0:0.6928203230275511,2:3.676955262170047}
> >   1  => {0:0.3464101615137781,2:5.374011537017761}
> >   2  => {2:7.0710678118654755}
> > }
> > (L^-1)b =
> > {
> >   0  => {0:1.2727922061357855}
> >   1  => {0:11.547005383792511}
> >   2  => {}
> > }
> > X =
> > {
> >   0  => {0:0.18}
> >   1  => {0:5.119661282874144}
> >   2  => {}
> > }
> > AX - B =
> > {
> >   0  => {0:459.0}
> >   1  => {0:670.0}
> >   2  => {0:881.0}
> > }
> >
> > org.scalatest.exceptions.TestFailedException was thrown.
> >
> >
>

Re: Cholesky test failures...

[Dmitriy Lyubimov]

Hm.

if i specify pivoted=false, everything works.
In addition it seems i have created singular input by chance, but making it
non-singular


 val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5.5))

 still doesn't help with pivoted=true (which is default) , my test fails.

do pivoted results require some special handling?




On Mon, Jul 15, 2013 at 5:03 PM, Dmitriy Lyubimov  wrote:

> should read
>
> val axmb = (a %*% x) - b
>
> of course but it doesn't make difference, the Cholesky output doesn't make
> sense to me even before that
>
>
> On Mon, Jul 15, 2013 at 5:00 PM, Dmitriy Lyubimov wrote:
>
>> Hi Ted,
>>
>> I am getting Cholesky test failures when trying to solve of Ax=B
>>
>> The L matrix and solveLeft() output do not make much sense to me. For
>> once, L doesn't even have the expected L-shape:
>>
>> Do you have an idea where i go wrong? (the test is wrapped into scala DSL
>> but it is Mahout's cholesky underwraps) .
>>
>> test code:
>>
>>   test("chol") {
>>
>> // try to solve Ax=b with cholesky:
>> // this requires
>> // (LL')x = B
>> // L'x= (L^-1)B
>> // x=(L'^-1)(L^-1)B
>>
>> val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5))
>>
>> // make sure it is symmetric for a valid solution
>> a := a.t %*% a
>>
>> printf("A= \n%s\n", a)
>>
>> val b = dense((9, 8, 7)).t
>>
>> printf ("b = \n%s\n", b)
>>
>> val ch = chol(a)
>>
>> printf ("L = \n%s\n", ch.getL)
>>
>> printf ( "(L^-1)b =\n%s\n", ch.solveLeft(b))
>>
>> val x = ch.solveRight(diag(1,3)) %*% ch.solveLeft(b)
>>
>> printf("x = \n%s\n", x.toString)
>>
>> val axmb = (a %*% b) - b
>>
>> printf("AX - B = \n%s\n", axmb.toString)
>>
>> assert(axmb.norm < 1e-10)
>>
>>   }
>>
>>
>>
>> Output:
>>
>> A=
>> {
>>   0  => {0:14.0,1:20.0,2:26.0}
>>   1  => {0:20.0,1:29.0,2:38.0}
>>   2  => {0:26.0,1:38.0,2:50.0}
>> }
>> b =
>> {
>>   0  => {0:9.0}
>>   1  => {0:8.0}
>>   2  => {0:7.0}
>> }
>>
>> L =
>> {
>>   0  => {0:0.6928203230275511,2:3.676955262170047}
>>   1  => {0:0.3464101615137781,2:5.374011537017761}
>>   2  => {2:7.0710678118654755}
>> }
>> (L^-1)b =
>> {
>>   0  => {0:1.2727922061357855}
>>   1  => {0:11.547005383792511}
>>   2  => {}
>> }
>> X =
>> {
>>   0  => {0:0.18}
>>   1  => {0:5.119661282874144}
>>   2  => {}
>> }
>> AX - B =
>> {
>>   0  => {0:459.0}
>>   1  => {0:670.0}
>>   2  => {0:881.0}
>> }
>>
>> org.scalatest.exceptions.TestFailedException was thrown.
>>
>>
>

Re: Cholesky test failures...

[Dmitriy Lyubimov]

should read

val axmb = (a %*% x) - b

of course but it doesn't make difference, the Cholesky output doesn't make
sense to me even before that


On Mon, Jul 15, 2013 at 5:00 PM, Dmitriy Lyubimov  wrote:

> Hi Ted,
>
> I am getting Cholesky test failures when trying to solve of Ax=B
>
> The L matrix and solveLeft() output do not make much sense to me. For
> once, L doesn't even have the expected L-shape:
>
> Do you have an idea where i go wrong? (the test is wrapped into scala DSL
> but it is Mahout's cholesky underwraps) .
>
> test code:
>
>   test("chol") {
>
> // try to solve Ax=b with cholesky:
> // this requires
> // (LL')x = B
> // L'x= (L^-1)B
> // x=(L'^-1)(L^-1)B
>
> val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5))
>
> // make sure it is symmetric for a valid solution
> a := a.t %*% a
>
> printf("A= \n%s\n", a)
>
> val b = dense((9, 8, 7)).t
>
> printf ("b = \n%s\n", b)
>
> val ch = chol(a)
>
> printf ("L = \n%s\n", ch.getL)
>
> printf ( "(L^-1)b =\n%s\n", ch.solveLeft(b))
>
> val x = ch.solveRight(diag(1,3)) %*% ch.solveLeft(b)
>
> printf("x = \n%s\n", x.toString)
>
> val axmb = (a %*% b) - b
>
> printf("AX - B = \n%s\n", axmb.toString)
>
> assert(axmb.norm < 1e-10)
>
>   }
>
>
>
> Output:
>
> A=
> {
>   0  => {0:14.0,1:20.0,2:26.0}
>   1  => {0:20.0,1:29.0,2:38.0}
>   2  => {0:26.0,1:38.0,2:50.0}
> }
> b =
> {
>   0  => {0:9.0}
>   1  => {0:8.0}
>   2  => {0:7.0}
> }
>
> L =
> {
>   0  => {0:0.6928203230275511,2:3.676955262170047}
>   1  => {0:0.3464101615137781,2:5.374011537017761}
>   2  => {2:7.0710678118654755}
> }
> (L^-1)b =
> {
>   0  => {0:1.2727922061357855}
>   1  => {0:11.547005383792511}
>   2  => {}
> }
> X =
> {
>   0  => {0:0.18}
>   1  => {0:5.119661282874144}
>   2  => {}
> }
> AX - B =
> {
>   0  => {0:459.0}
>   1  => {0:670.0}
>   2  => {0:881.0}
> }
>
> org.scalatest.exceptions.TestFailedException was thrown.
>
>

Cholesky test failures...

[Dmitriy Lyubimov]

Hi Ted,

I am getting Cholesky test failures when trying to solve of Ax=B

The L matrix and solveLeft() output do not make much sense to me. For once,
L doesn't even have the expected L-shape:

Do you have an idea where i go wrong? (the test is wrapped into scala DSL
but it is Mahout's cholesky underwraps) .

test code:

  test("chol") {

// try to solve Ax=b with cholesky:
// this requires
// (LL')x = B
// L'x= (L^-1)B
// x=(L'^-1)(L^-1)B

val a = dense((1, 2, 3), (2, 3, 4), (3, 4, 5))

// make sure it is symmetric for a valid solution
a := a.t %*% a

printf("A= \n%s\n", a)

val b = dense((9, 8, 7)).t

printf ("b = \n%s\n", b)

val ch = chol(a)

printf ("L = \n%s\n", ch.getL)

printf ( "(L^-1)b =\n%s\n", ch.solveLeft(b))

val x = ch.solveRight(diag(1,3)) %*% ch.solveLeft(b)

printf("x = \n%s\n", x.toString)

val axmb = (a %*% b) - b

printf("AX - B = \n%s\n", axmb.toString)

assert(axmb.norm < 1e-10)

  }



Output:

A=
{
  0  => {0:14.0,1:20.0,2:26.0}
  1  => {0:20.0,1:29.0,2:38.0}
  2  => {0:26.0,1:38.0,2:50.0}
}
b =
{
  0  => {0:9.0}
  1  => {0:8.0}
  2  => {0:7.0}
}

L =
{
  0  => {0:0.6928203230275511,2:3.676955262170047}
  1  => {0:0.3464101615137781,2:5.374011537017761}
  2  => {2:7.0710678118654755}
}
(L^-1)b =
{
  0  => {0:1.2727922061357855}
  1  => {0:11.547005383792511}
  2  => {}
}
X =
{
  0  => {0:0.18}
  1  => {0:5.119661282874144}
  2  => {}
}
AX - B =
{
  0  => {0:459.0}
  1  => {0:670.0}
  2  => {0:881.0}
}

org.scalatest.exceptions.TestFailedException was thrown.

Build failed in Jenkins: mahout-nightly #1292

[Apache Jenkins Server]

See 

--
[...truncated 2525 lines...]
Running org.apache.mahout.math.CholeskyDecompositionTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.378 sec - in 
org.apache.mahout.math.CholeskyDecompositionTest
Running org.apache.mahout.math.decomposer.lanczos.TestLanczosSolver
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.977 sec - in 
org.apache.mahout.math.decomposer.lanczos.TestLanczosSolver
Running org.apache.mahout.math.decomposer.hebbian.TestHebbianSolver
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.844 sec - in 
org.apache.mahout.math.decomposer.hebbian.TestHebbianSolver
Running org.apache.mahout.math.OldQRDecompositionTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.095 sec - in 
org.apache.mahout.math.OldQRDecompositionTest
Running org.apache.mahout.math.TestOrderedIntDoubleMapping
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec - in 
org.apache.mahout.math.TestOrderedIntDoubleMapping
Running org.apache.mahout.math.WeightedVectorTest
Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.64 sec - in 
org.apache.mahout.math.WeightedVectorTest
Running org.apache.mahout.math.map.OpenCharByteHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.024 sec - in 
org.apache.mahout.math.map.OpenCharByteHashMapTest
Running org.apache.mahout.math.map.OpenByteShortHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.024 sec - in 
org.apache.mahout.math.map.OpenByteShortHashMapTest
Running org.apache.mahout.math.map.OpenDoubleCharHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 sec - in 
org.apache.mahout.math.map.OpenDoubleCharHashMapTest
Running org.apache.mahout.math.map.OpenByteFloatHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.028 sec - in 
org.apache.mahout.math.map.OpenByteFloatHashMapTest
Running org.apache.mahout.math.map.OpenByteDoubleHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 sec - in 
org.apache.mahout.math.map.OpenByteDoubleHashMapTest
Running org.apache.mahout.math.map.OpenCharCharHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.022 sec - in 
org.apache.mahout.math.map.OpenCharCharHashMapTest
Running org.apache.mahout.math.map.OpenIntByteHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 sec - in 
org.apache.mahout.math.map.OpenIntByteHashMapTest
Running org.apache.mahout.math.map.OpenIntObjectHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 sec - in 
org.apache.mahout.math.map.OpenIntObjectHashMapTest
Running org.apache.mahout.math.map.OpenShortFloatHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.026 sec - in 
org.apache.mahout.math.map.OpenShortFloatHashMapTest
Running org.apache.mahout.math.map.OpenCharLongHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 sec - in 
org.apache.mahout.math.map.OpenCharLongHashMapTest
Running org.apache.mahout.math.map.OpenByteByteHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 sec - in 
org.apache.mahout.math.map.OpenByteByteHashMapTest
Running org.apache.mahout.math.map.OpenCharFloatHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 sec - in 
org.apache.mahout.math.map.OpenCharFloatHashMapTest
Running org.apache.mahout.math.map.OpenIntLongHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.024 sec - in 
org.apache.mahout.math.map.OpenIntLongHashMapTest
Running org.apache.mahout.math.map.OpenShortIntHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 sec - in 
org.apache.mahout.math.map.OpenShortIntHashMapTest
Running org.apache.mahout.math.map.OpenByteIntHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 sec - in 
org.apache.mahout.math.map.OpenByteIntHashMapTest
Running org.apache.mahout.math.map.OpenLongShortHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.024 sec - in 
org.apache.mahout.math.map.OpenLongShortHashMapTest
Running org.apache.mahout.math.map.OpenDoubleIntHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 sec - in 
org.apache.mahout.math.map.OpenDoubleIntHashMapTest
Running org.apache.mahout.math.map.OpenIntShortHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.024 sec - in 
org.apache.mahout.math.map.OpenIntShortHashMapTest
Running org.apache.mahout.math.map.OpenByteObjectHashMapTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.026 sec - in 
org.apache.mahout.math.map.OpenByteObjectHashMapTest
Running org.apache.mahout.math.map.OpenDoubleDoubleHashMapTest
Tests run: 1

Regarding Online Recommenders

[Gokhan Capan]

Based on the conversation in MAHOUT-1274, I put some code here:

https://github.com/gcapan/mahout/tree/onlinerec
I hope that would initiate a discussion on OnlineRecommender approaches.

I think the OnlineRecommender would require (similar to what Sebastian
commented there):

1- A DataModel that allows adding new users/items and performs fast
iteration
2- An online learning interface that allows updating the model with a
feedback, and make predictions based on the latest model

The code is a "very early" effort for the latter, and it contains a matrix
factorization-based implementation where training is done by SGD.

The model is stored in a DenseMatrix --it should be replaced with a matrix
that allows adding new rows and doesn't allocate space for empty rows
(please search for DenseRowMatrix and BlockSparseMatrix in the dev-list,
and see MAHOUT-1193 for relevant issue).

I didn't try that on a dataset yet.

The DataModel I imagine would follow the current API, where underlying
preference storage is replaced with a matrix.

A Recommender would then use the DataModel and the OnlineLearner, where
Recommender#setPreference is delegated to DataModel#setPreference (like it
does now), and DataModel#setPreference triggers OnlineLearner#train.

Gokhan

[jira] [Updated] (MAHOUT-1285) Arff loader can misparse string data as double

[Neil Walkinshaw (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MAHOUT-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neil Walkinshaw updated MAHOUT-1285:


Attachment: tempArff

This file causes the exception

> Arff loader can misparse string data as double
> --
>
> Key: MAHOUT-1285
> URL: https://issues.apache.org/jira/browse/MAHOUT-1285
> Project: Mahout
>  Issue Type: Bug
>Affects Versions: 0.9
> Environment: Linux Ubuntu 12.4
>Reporter: Neil Walkinshaw
> Attachments: tempArff
>
>
> Have successfully loaded numerous ARFF files with Mahout (originally 
> generated via WEKA). The files contain randomly generated data. For a 
> specific random seed, the following exception is thrown:
> java.lang.NumberFormatException: For input string: 
> "b1shkt70694difsmmmdv0ikmoh"
>   at 
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1241)
>   at java.lang.Double.parseDouble(Double.java:540)
>   at 
> org.apache.mahout.utils.vectors.arff.MapBackedARFFModel.processNumeric(MapBackedARFFModel.java:146)
>   at 
> org.apache.mahout.utils.vectors.arff.MapBackedARFFModel.getValue(MapBackedARFFModel.java:97)
>   at 
> org.apache.mahout.utils.vectors.arff.ARFFIterator.computeNext(ARFFIterator.java:77)
>   at 
> org.apache.mahout.utils.vectors.arff.ARFFIterator.computeNext(ARFFIterator.java:30)
>   at 
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
>   at 
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
>   at 
> org.apache.mahout.utils.vectors.io.SequenceFileVectorWriter.write(SequenceFileVectorWriter.java:44)
>   at 
> org.apache.mahout.utils.vectors.arff.Driver.writeFile(Driver.java:251)
>   at org.apache.mahout.utils.vectors.arff.Driver.main(Driver.java:145)
>   at 
> libInterfaces.MahoutTraceBuilder.generateMahoutFile(MahoutTraceBuilder.java:38)
>   at 
> libInterfaces.MahoutTraceBuilder.generateMahoutReader(MahoutTraceBuilder.java:42)
>   at tests.InputTester.testMahoutMeansShift(InputTester.java:111)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAHOUT-1285) Arff loader can misparse string data as double

[Neil Walkinshaw (JIRA)]

Neil Walkinshaw created MAHOUT-1285:
---

 Summary: Arff loader can misparse string data as double
 Key: MAHOUT-1285
 URL: https://issues.apache.org/jira/browse/MAHOUT-1285
 Project: Mahout
  Issue Type: Bug
Affects Versions: 0.9
 Environment: Linux Ubuntu 12.4
Reporter: Neil Walkinshaw


Have successfully loaded numerous ARFF files with Mahout (originally generated 
via WEKA). The files contain randomly generated data. For a specific random 
seed, the following exception is thrown:

java.lang.NumberFormatException: For input string: "b1shkt70694difsmmmdv0ikmoh"
at 
sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1241)
at java.lang.Double.parseDouble(Double.java:540)
at 
org.apache.mahout.utils.vectors.arff.MapBackedARFFModel.processNumeric(MapBackedARFFModel.java:146)
at 
org.apache.mahout.utils.vectors.arff.MapBackedARFFModel.getValue(MapBackedARFFModel.java:97)
at 
org.apache.mahout.utils.vectors.arff.ARFFIterator.computeNext(ARFFIterator.java:77)
at 
org.apache.mahout.utils.vectors.arff.ARFFIterator.computeNext(ARFFIterator.java:30)
at 
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at 
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at 
org.apache.mahout.utils.vectors.io.SequenceFileVectorWriter.write(SequenceFileVectorWriter.java:44)
at 
org.apache.mahout.utils.vectors.arff.Driver.writeFile(Driver.java:251)
at org.apache.mahout.utils.vectors.arff.Driver.main(Driver.java:145)
at 
libInterfaces.MahoutTraceBuilder.generateMahoutFile(MahoutTraceBuilder.java:38)
at 
libInterfaces.MahoutTraceBuilder.generateMahoutReader(MahoutTraceBuilder.java:42)
at tests.InputTester.testMahoutMeansShift(InputTester.java:111)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Mahout-Examples-Cluster-Reuters-II #543

[Apache Jenkins Server]

See 

--
[...truncated 2223 lines...]
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractIntByteMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractIntCharMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractIntIntMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractIntShortMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractIntLongMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractIntFloatMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractIntDoubleMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractShortByteMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractShortCharMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractShortIntMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractShortShortMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractShortLongMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractShortFloatMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractShortDoubleMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractLongByteMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractLongCharMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractLongIntMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractLongShortMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractLongLongMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractLongFloatMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractLongDoubleMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahout/math/map/AbstractFloatByteMap.java
[INFO] Writing to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Mahout-Examples-Cluster-Reuters-II/trunk/math/target/generated-sources/mahout/org/apache/mahou

Re: Mahout release process

[Dmitriy Lyubimov]

no, doesnt look so.


On Mon, Jul 15, 2013 at 10:10 AM, Grant Ingersoll wrote:

>
> On Jul 14, 2013, at 7:27 PM, Dmitriy Lyubimov  wrote:
>
> >> I'd say go for it.  Of course, my preference would be that time spent on
> >> Mahout right now is focused on testing 0.8, but you are free to do as
> you
> >> wish.
> >>
> >
> > it looks good on my part. I found however that a bug was (re-?)
> introduced
> > into UpperTriangular matrix( breaks row count property in certain form of
> > constructor) which however did not seem to affect any of existing
> solvers.
> > this is fixed as a part of M-1281
>
> Do we need to respin?
>
>
>

Re: Mahout release process

[Grant Ingersoll]

On Jul 14, 2013, at 7:27 PM, Dmitriy Lyubimov  wrote:

>> I'd say go for it.  Of course, my preference would be that time spent on
>> Mahout right now is focused on testing 0.8, but you are free to do as you
>> wish.
>> 
> 
> it looks good on my part. I found however that a bug was (re-?) introduced
> into UpperTriangular matrix( breaks row count property in certain form of
> constructor) which however did not seem to affect any of existing solvers.
> this is fixed as a part of M-1281

Do we need to respin?