On Fri, Apr 5, 2013 at 8:07 AM, Sean Owen <sro...@gmail.com> wrote: > OK yes you're on to something here. I should clarify. Koobas you are > right that the ALS algorithm itself is fine here as far as my > knowledge takes me. The thing it inverts to solve for a row of X is > something like (Y' * Cu * Y + lambda * I). No problem there, and > indeed I see why the regularization term is part of that. > > I'm talking about a later step, after the factorization. You get a new > row in A and want to solve A = X*Y' for X, given the current Y. (And > vice versa). I'm using a QR decomposition for that, but not to > directly solve the system (and this may be the issue), but instead to > compute and save off (Y' * Y)^-1 so that we can figure A * Y * > (Y'*Y)^-1 very fast at runtime. That is to say the problem centers > around the inverse of Y'*Y and in this example, it does not even > exist. > > I don't see why the inverse of Y'*Y does not exist. What Y do you end up with?
> I am not sure it's just a numerical precision thing since using an SVD > to get the inverse gives the same result. > > But I certainly have examples where the data (A) is most certainly > rank >> k and get this bad behavior -- for example, when lambda is > very *high*. > > > On Fri, Apr 5, 2013 at 6:57 AM, Ted Dunning <ted.dunn...@gmail.com> wrote: > > On Fri, Apr 5, 2013 at 2:40 AM, Koobas <koo...@gmail.com> wrote: > > > >> Anyways, I saw no particular reason for the method to fail with k > >> approaching or exceeding m and n. > >> It does if there is no regularization. > >> But with regularization in place, k can be pretty much anything. > >> > > > > Ahh... this is an important point and it should handle all of the issues > of > > poor conditioning. > > > > The regularizer takes the rank deficient A and makes it reasonably well > > conditioned. How well conditioned depends on the choice of lambda, the > > regularizing scale constant. >
