Re: TallSkinnyQR

2016-12-30 Thread Iman Mohtashemi
Hi guys, Are your changes/bug fixes reflected in the Spark 2.1 release? Iman On Dec 2, 2016 3:03 PM, "Iman Mohtashemi" <iman.mohtash...@gmail.com> wrote: > Thanks again! This is very helpful! > Best regards, > Iman > > On Dec 2, 2016 2:49 PM, "Huamin Li"

Re: Running spark from Eclipse and then Jar

2016-12-10 Thread Iman Mohtashemi
an, Galway, Ireland > Web: http://www.reza-analytics.eu/index.html > <http://139.59.184.114/index.html> > > On 8 December 2016 at 01:15, Iman Mohtashemi <iman.mohtash...@gmail.com> > wrote: > > yes exactly. I run mine fine in Eclipse but when I run it from a > cor

Re: Running spark from Eclipse and then Jar

2016-12-07 Thread Iman Mohtashemi
onal University of Ireland, Galway > IDA Business Park, Dangan, Galway, Ireland > Web: http://www.reza-analytics.eu/index.html > <http://139.59.184.114/index.html> > > On 7 December 2016 at 23:39, Iman Mohtashemi <iman.mohtash...@gmail.com> > wrote: > > No but I tried t

Re: Running spark from Eclipse and then Jar

2016-12-07 Thread Iman Mohtashemi
No but I tried that too and still didn't work. Where are the files being read from? From the local machine or HDFS? Do I need to get the files to HDFS first? In Eclipse I just point to the location of the directory? On Wed, Dec 7, 2016 at 3:34 PM Md. Rezaul Karim <

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
ith me! > > I implemented these codes for my summer intern project with Mark Tygert, > and we are currently testing the performance of the new codes. > > Best, > Huamin > > On Fri, Dec 2, 2016 at 2:07 PM, Iman Mohtashemi <iman.mohtash...@gmail.com > > wrote: > >>

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
tallSkinnyQR are giving the correct results for > indexedRowMatrix when using multiple executors or workers. Let me know if > I should do a pull request for this. > > Best, > Huamin > > On Fri, Dec 2, 2016 at 11:23 AM, Iman Mohtashemi < > iman.mohtash...@gmail.com> wrot

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
ctor.WriteClusterKey(clusters.get(i)); String cValue = detector.WriteClusterValue(clusters.get(i)); context.write(new Text(cKey), new Text(cValue)); } } } On Fri, Dec 2, 2016 at 8:23 AM Iman Mohtashemi <iman.mohtash...@gmai

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
it ordered rows if you're > going to use a method like the QR decomposition. That works. I don't think > the QR method should ever have been on this class though, for this reason. > > On Fri, Dec 2, 2016 at 4:13 PM Iman Mohtashemi <iman.mohtash...@gmail.com> > wrote: > > Hi

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
Hi guys, Was this bug ever resolved? Iman On Fri, Nov 11, 2016 at 9:59 AM Iman Mohtashemi <iman.mohtash...@gmail.com> wrote: > Yes this would be helpful, otherwise the Q part of the decomposition is > useless. One can use that to solve the system by transposing it and > mul

Re: TallSkinnyQR

2016-11-11 Thread Iman Mohtashemi
gt; existence of use cases like yours. For example, at least > CoordinateMatrix.toIndexedRowMatrix could sort? that is less surprising. > > In any event you should be able to make it work by manually getting the > RDD[IndexedRow] out of IndexedRowMatrix, sorting by index, then mapping it >

Re: TallSkinnyQR

2016-11-08 Thread Iman Mohtashemi
dex, then mapping it > to Vectors and making a RowMatrix from it. > > > > On Tue, Nov 8, 2016 at 2:41 PM Iman Mohtashemi <iman.mohtash...@gmail.com> > wrote: > >> Hi Sean, >> Here you go: >> >> sparsematrix.txt = >> >> row, col ,val >

Re: TallSkinnyQR

2016-11-08 Thread Iman Mohtashemi
of this this would work?? On Tue, Nov 8, 2016 at 6:41 AM Iman Mohtashemi <iman.mohtash...@gmail.com> wrote: > Hi Sean, > Here you go: > > sparsematrix.txt = > > row, col ,val > 0,0,.42 > 0,1,.28 > 0,2,.89 > 1,0,.83 > 1,1,.34 > 1,2,.42 > 2,0,.23 > 3,0,.

Re: TallSkinnyQR

2016-11-08 Thread Iman Mohtashemi
Hi Sean, Here you go: sparsematrix.txt = row, col ,val 0,0,.42 0,1,.28 0,2,.89 1,0,.83 1,1,.34 1,2,.42 2,0,.23 3,0,.42 3,1,.98 3,2,.88 4,0,.23 4,1,.36 4,2,.97 The vector is just the third column of the matrix which should give the trivial solution of [0,0,1] This translates to this which is