Re: TallSkinnyQR

2016-12-30 Thread Iman Mohtashemi
Hi guys, Are your changes/bug fixes reflected in the Spark 2.1 release? Iman On Dec 2, 2016 3:03 PM, "Iman Mohtashemi" wrote: > Thanks again! This is very helpful! > Best regards, > Iman > > On Dec 2, 2016 2:49 PM, "Huamin Li" <3eri...@gmail.com> wrote

Re: Running spark from Eclipse and then Jar

2016-12-10 Thread Iman Mohtashemi
Dangan, Galway, Ireland > Web: http://www.reza-analytics.eu/index.html > <http://139.59.184.114/index.html> > > On 8 December 2016 at 01:15, Iman Mohtashemi > wrote: > > yes exactly. I run mine fine in Eclipse but when I run it from a > corresponding jar I get the same

Re: Running spark from Eclipse and then Jar

2016-12-07 Thread Iman Mohtashemi
> National University of Ireland, Galway > IDA Business Park, Dangan, Galway, Ireland > Web: http://www.reza-analytics.eu/index.html > <http://139.59.184.114/index.html> > > On 7 December 2016 at 23:39, Iman Mohtashemi > wrote: > > No but I tried that too and still did

Re: Running spark from Eclipse and then Jar

2016-12-07 Thread Iman Mohtashemi
No but I tried that too and still didn't work. Where are the files being read from? From the local machine or HDFS? Do I need to get the files to HDFS first? In Eclipse I just point to the location of the directory? On Wed, Dec 7, 2016 at 3:34 PM Md. Rezaul Karim < rezaul.ka...@insight-centre.org>

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
ith me! > > I implemented these codes for my summer intern project with Mark Tygert, > and we are currently testing the performance of the new codes. > > Best, > Huamin > > On Fri, Dec 2, 2016 at 2:07 PM, Iman Mohtashemi > wrote: > >> Great thanks! Where can I g

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
tallSkinnyQR are giving the correct results for > indexedRowMatrix when using multiple executors or workers. Let me know if > I should do a pull request for this. > > Best, > Huamin > > On Fri, Dec 2, 2016 at 11:23 AM, Iman Mohtashemi < > iman.mohtash...@gmail.com> wro

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
= detector.WriteClusterValue(clusters.get(i)); context.write(new Text(cKey), new Text(cValue)); } } } On Fri, Dec 2, 2016 at 8:23 AM Iman Mohtashemi wrote: > Ok thanks. > > On Fri, Dec 2, 2016 at 8:19 AM Sean Owen wrote: > > I tried

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
> going to use a method like the QR decomposition. That works. I don't think > the QR method should ever have been on this class though, for this reason. > > On Fri, Dec 2, 2016 at 4:13 PM Iman Mohtashemi > wrote: > > Hi guys, > Was this bug ever resolved? >

Re: TallSkinnyQR

2016-12-02 Thread Iman Mohtashemi
Hi guys, Was this bug ever resolved? Iman On Fri, Nov 11, 2016 at 9:59 AM Iman Mohtashemi wrote: > Yes this would be helpful, otherwise the Q part of the decomposition is > useless. One can use that to solve the system by transposing it and > multiplying with b and solving for x (Ax =

Re: TallSkinnyQR

2016-11-11 Thread Iman Mohtashemi
ke yours. For example, at least > CoordinateMatrix.toIndexedRowMatrix could sort? that is less surprising. > > In any event you should be able to make it work by manually getting the > RDD[IndexedRow] out of IndexedRowMatrix, sorting by index, then mapping it > to Vectors and making a

Re: TallSkinnyQR

2016-11-08 Thread Iman Mohtashemi
still to at least sort the rows, given the > existence of use cases like yours. For example, at least > CoordinateMatrix.toIndexedRowMatrix > could sort? that is less surprising. > > In any event you should be able to make it work by manually getting the > RDD[IndexedRow] out of IndexedRowMatrix, sorting by in

Re: TallSkinnyQR

2016-11-08 Thread Iman Mohtashemi
#x27; If you have a code example of this this would work?? On Tue, Nov 8, 2016 at 6:41 AM Iman Mohtashemi wrote: > Hi Sean, > Here you go: > > sparsematrix.txt = > > row, col ,val > 0,0,.42 > 0,1,.28 > 0,2,.89 > 1,0,.83 > 1,1,.34 > 1,2,.42 > 2,0,.23 > 3,0

Re: TallSkinnyQR

2016-11-08 Thread Iman Mohtashemi
Hi Sean, Here you go: sparsematrix.txt = row, col ,val 0,0,.42 0,1,.28 0,2,.89 1,0,.83 1,1,.34 1,2,.42 2,0,.23 3,0,.42 3,1,.98 3,2,.88 4,0,.23 4,1,.36 4,2,.97 The vector is just the third column of the matrix which should give the trivial solution of [0,0,1] This translates to this which is cor