Re: Problems running SSVD directly from Java

2015-04-28 Thread Dmitriy Lyubimov
On Tue, Apr 28, 2015 at 1:14 PM, Mihai Dascalu wrote: > Indeed, it’s in local mode - but to setup hadoop on my Mac for the task at > hand did not seem necessary (the SVD uses a sparse matrix of 11MB). > oh. Then it is a wrong tool. try bidMat, I promise you won't be dissapointed. https://github

Re: Problems running SSVD directly from Java

2015-04-28 Thread Dmitriy Lyubimov
also it seems like you are running a local mode. This project doesn't test for hadoop local mode as a part of its release. reason being hadoop local mode has so many limitations and incompatibilities (e.g. can't run more than 1 reducer, for one) that it was never worth it. On Tue, Apr 28, 2015

Re: Problems running SSVD directly from Java

2015-04-28 Thread Dmitriy Lyubimov
i suggested to start with -k 10 not 100 to iron stuff out. that will reduce debug cycle quite a bit. the problem with classpath is definitely something reproducible and functional, it is not a known problem to me. But... there are so many flavors of hadoop that various issues like that pop up all

Re: Problems running SSVD directly from Java

2015-04-28 Thread Mihai Dascalu
I got the same problem with k=100 & p=15, aBlockRows=20 faster now (around 20minutes) I just realized that it’s at a final step in the processing (I’ve attached the end part of the log) Any suggestions? In my Eclipse project I have imported: httpclient-4.2.5.jar mahout-hdfs-0.10.0.jar mahou

Re: Problems running SSVD directly from Java

2015-04-28 Thread Dmitriy Lyubimov
if your run time gets too high, try to start with low -k (like 10 or something) and -q=0, that will significantly reduce complexity of the problem. if this works, you need to find optimal levers that suit your hardware/input size/ runtime requirements. ( I can tell you right away that (k+p) value

Re: Problems running SSVD directly from Java

2015-04-28 Thread Mihai Dascalu
I’ve created a SWING interface around the invocation, but it is not a classpath setting as the SVD runs for more than 1h. Afterwards I have the runtime error in the HTTPclient, which is really strange. Also I have a lot of map operations in the console, but no reduce operations are logged. Than

Problems running SSVD directly from Java

2015-04-28 Thread Mihai Dascalu
Hi! I’ve been experimenting with the SSVDSolver and unfortunately, during runtime, I encounter this error: 10648576 [Thread-13] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local1958711697_0001 java.lang.NoClassDefFoundError: org/apache/commons/httpclient/HttpMethod at org.apac