So b = 0.89 0.42 0.0 0.88 0.97 The solution at the bottom is the solution to Ax = b solved using Gaussian elimination. I guess another question is, is there another way to solve this problem? I'm trying to solve the least squares fit with a huge A (5MM x 1MM)
x = inverse(A-transpose*A)*A-transose*b but I didn't see any functions for matrix inversion I suppose I can use an iterative solver but I didn't see that either which is why I chose the QR decomposition , solve for Q and then Q-transpose*b = d and the solve Lx = d which would give the solution. But I don't think this would work either since the matrices are local copies and not RDD data structures. Any advice would be appreciated... Iman P.S. I also looked in the linear regression class in the mlib but I haven't seen any examples with sparse matrix and sparse vectors as the input just 'Dataset' If you have a code example of this this would work?? On Tue, Nov 8, 2016 at 6:41 AM Iman Mohtashemi <iman.mohtash...@gmail.com> wrote: > Hi Sean, > Here you go: > > sparsematrix.txt = > > row, col ,val > 0,0,.42 > 0,1,.28 > 0,2,.89 > 1,0,.83 > 1,1,.34 > 1,2,.42 > 2,0,.23 > 3,0,.42 > 3,1,.98 > 3,2,.88 > 4,0,.23 > 4,1,.36 > 4,2,.97 > > The vector is just the third column of the matrix which should give the > trivial solution of [0,0,1] > > This translates to this which is correct > There are zeros in the matrix (Not really sparse but just an example) > 0.42 0.28 0.89 > 0.83 0.34 0.42 > 0.23 0.0 0.0 > 0.42 0.98 0.88 > 0.23 0.36 0.97 > > > Here is what I get for the Q and R > > Q: -0.21470961288429483 0.23590615093828807 0.6784910613691661 > -0.3920784235278427 -0.06171221388256143 0.5847874866876442 > -0.7748216464954987 -0.4003560542230838 -0.29392323671555354 > -0.3920784235278427 0.8517909521421976 -0.31435038559403217 > -0.21470961288429483 -0.23389547730301666 -0.11165321782745863 > R: -1.0712142642814275 -0.8347536340918976 -1.227672225670157 > 0.0 0.7662808691141717 0.7553315911660984 > 0.0 0.0 0.7785210939368136 > > When running this in matlab the numbers are the same but row 1 is the last > row and the last row is interchanged with row 3 > > > > On Mon, Nov 7, 2016 at 11:35 PM Sean Owen <so...@cloudera.com> wrote: > > Rather than post a large section of code, please post a small example of > the input matrix and its decomposition, to illustrate what you're saying is > out of order. > > On Tue, Nov 8, 2016 at 3:50 AM im281 <iman.mohtash...@gmail.com> wrote: > > I am getting the correct rows but they are out of order. Is this a bug or > am > I doing something wrong? > > >
