Hi, My cluster size is 4 machines and I trying to multiply 2 7*7 matrices. I think there is some problem with some setting/ configuration. Where is the heap size checked ? Probably I can go that file and see what is the problem ?
Thank You Abhishek Agrawal SUNY- Buffalo (716-435-7122) On Wed 11/04/09 12:19 AM , "Edward J. Yoon" [email protected] sent: > I don't see any error during perform matrix multiplication of a matrix > with itself. > > How large your matrix is? (and, let me know your cluster information) > > On Wed, Nov 4, 2009 at 11:40 AM, Edward J. Yoon [email protected]> wrote:> OK, > That error appears *only when* perform the > matrix multiplication> of a matrix with itself? > > > > I'll check it. > > > > On Wed, Nov 4, 2009 at 5:38 AM, >  lo.edu> wrote:>> Hi, > >>  As I was telling you yesterday > I am getting an error when I call the matrix>> multiplication function from my program. The > exact error I am getting is as>> under. The call to the function > is>> > >>    >     >  DenseMatrix resultMatrix= > covMatrix.mult(covMatrix);>> > >> I am multiplying the matrix covMatrix by > itself just to check if the>> configuration is done correctly. But then I > am getting an exception. I tried to>> run the default matrix multiplication > program bin/hama examples mult matA matA>> but that worked perfectly well. SO > I think > its not that the heap is full rather I>> have not done some setting > properly.>> > >> had...@zeus:~ > /HAMA$ java Markowitz.MarkowitzMain>> In matrix Multiplication > >> 03/04/26 20:29:05 INFO jvm.JvmMetrics: > Initializing JVM Metrics with>> processName=JobTracker, > sessionId=>> 03/04/26 20:29:05 WARN mapred.JobClient: Use > GenericOptionsParser for parsing the>> arguments. Applications should > implement > Tool for the same.>> 03/04/26 20:29:05 WARN mapred.JobClient: No > job jar file set.  User classes may>> not be found. See JobConf(Class) or > JobConf#setJar(String).>> 03/04/26 20:29:05 INFO > mapred.HTableInputFormatBase: split: 0->aphrodite:,>> 03/04/26 20:29:05 INFO mapred.JobClient: > Running job: job_local_0001>> 03/04/26 20:29:06 INFO > mapred.HTableInputFormatBase: split: 0->aphrodite:,>> 03/04/26 20:29:06 INFO mapred.MapTask: > numReduceTasks: 1>> 03/04/26 20:29:06 INFO mapred.MapTask: > io.sort.mb = 100>> 03/04/26 20:29:06 WARN > mapred.LocalJobRunner: job_local_0001>> java.lang.OutOfMemoryError: Java heap > space>>    >  at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:498)>>    >  at > org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)>>    >  at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)>> Exception in thread "Thread-14" > java.lang.NoClassDefFoundError:>> > org/apache/commons/httpclient/HttpMethod>>    >  at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:196)>> Caused by: java.lang.ClassNotFoundException: > org.apache.commons.httpclient.HttpMethod>>    >  at > java.net.URLClassLoader$1.run(URLClassLoader.java:200)>>    >  at java.security.AccessController.doPrivileged(Native > Method)>>    >  at > java.net.URLClassLoader.findClass(URLClassLoader.java:188)>>    >  at > java.lang.ClassLoader.loadClass(ClassLoader.java:306)>>    >  at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)>>    >  at > java.lang.ClassLoader.loadClass(ClassLoader.java:251)>>    >  at > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)>>    >  ... 1 more>> Exception in thread "main" > java.io.IOException: Job failed!>>    >  at > org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)>>    >  at > org.apache.hama.util.JobManager.execute(JobManager.java:52)>>    >  at > org.apache.hama.DenseMatrix.mult(DenseMatrix.java:554)>>    >  at > Markowitz.MarkowitzMain.main(MarkowitzMain.java:32)>> had...@zeus:~ > /HAMA$>> > >> Thank You > >> > >> > >> Abhishek Agrawal > >> > >> SUNY- Buffalo > >> (716-435-7122) > >> > >> On Tue 11/03/09  2:58 AM , > "Edward J. Yoon" edwardy > [email protected] sent:>>> Hi, > >>> > >>> # The maximum amount of heap to use, in > MB. Default is 1000.>>> # export HBASE_HEAPSIZE=1000 > >>> > >>> Check above option and regarding > temporary tables, there is no method>>> yet. But, You can delete them using > HbaseShell's disable and delete>>> command If you have no important data in > HDFS, re-format HDFS is good>>> idea. It's just simple tip. > >>> > >>> On Tue, Nov 3, 2009 at 4:39 PM, >  lo.edu> wrote:> Hi,>>> > � I have > another doubt. I am running>>> a program of mine which results in> > creation of quite a few tables in Hbase(397 to>>> be exact). Now when I am running a> > matrix-matrix multiplication job I am getting a>>> heap size not enough exception.> What > should I do to delete these 397 table>>> easily ?> > >>> > Thank You > >>> > > >>> > Abhishek Agrawal > >>> > > >>> > SUNY- Buffalo > >>> > (716-435-7122) > >>> > > >>> > On Sat 10/31/09 > � 3:25 AM ,>>> "Edward J. Yoon" > edwardy>>> [email protected] > sent:>> Sounds good. Thank you, I'll update>>> wiki.>> > >>> >> (you can also change the wiki > pages, please>>> feel free to > contribute!)>>>>> >> On Sat, Oct 31, 2009 at 2:28 > PM,>>> � > lo.edu> > wrote:> Hey,>>> >> > > ���>>> > ��� I just ran a test. > I>> put a println statement and it worked.>>> Thanks a lot for> your help. > From > my> experience, I feel addition>>> >> of the following statements to > the> wiki>>> page would make it easier for new users > to>> install/ get started with>>> hama.>>> > 1. Absolute paths > have to be given in>>> the config>> file> 2. To see > the O/P, changes have to>>> be made to>> the source code, the > code has to be>>>> compiled again and the necessary > files> copied to>>> >> $HAMA_HOME> > >>> >> > Thank You > >>> >> > > >>> >> > Abhishek > Agrawal>>> >> > > >>> >> > SUNY- Buffalo > >>> >> > (716-435-7122) > >>> >> > > >>> >> > >>> >> -- > >>> >> Best Regards, Edward J. Yoon @ > NHN,>>> corp.>> edwardy > >>> >> [email protected]http://blog.udanax.org>>>>> >> > >>> >> > >>> > > >>> > > >>> > >>> > >>> > >>> -- > >>> Best Regards, Edward J. Yoon @ NHN, > corp.>>> edwardy > >>> [email protected]http://blog.udanax.org>>> > >>> > >>> > >> > >> > > > > > > > > -- > > Best Regards, Edward J. Yoon @ NHN, > corp.> edwardy > [email protected]> http://blog.udanax.org> > > > > -- > Best Regards, Edward J. Yoon @ NHN, corp. > edwardy > [email protected]http://blog.udanax.org > > >
