Re: Re: Re: Re: Re: Re: Re: Re: Doubt in hama

Tue, 03 Nov 2009 22:59:53 -0800 

Hey,
    In my program I am creating 2 matrices, putting values in them and then I
call the function  

DenseMatrix resultMatrix= covMatrix.mult(retMatrix);

I am not setting the number of maps/ reduces to be done, the block thing. Is
there any problem because of that ?Also the full error I am getting is as under.
I noticed these 2 sentences mapred.MapTask: io.sort.mb = 100 and
mapred.LocalJobRunner: job_local_0001. Can you tell what is the significance of
these 2 sentences ?

03/04/27 08:02:56 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
03/04/27 08:02:56 WARN mapred.JobClient: Use GenericOptionsParser for parsing 
the
arguments. Applications should implement Tool for the same.
03/04/27 08:02:56 WARN mapred.JobClient: No job jar file set.  User classes may
not be found. See JobConf(Class) or JobConf#setJar(String).
03/04/27 08:02:56 INFO mapred.HTableInputFormatBase: split: 0->aphrodite:,
03/04/27 08:02:56 INFO mapred.JobClient: Running job: job_local_0001
03/04/27 08:02:57 INFO mapred.HTableInputFormatBase: split: 0->aphrodite:,
03/04/27 08:02:57 INFO mapred.MapTask: numReduceTasks: 1
03/04/27 08:02:57 INFO mapred.MapTask: io.sort.mb = 100
03/04/27 08:02:57 WARN mapred.LocalJobRunner: job_local_0001
java.lang.OutOfMemoryError: Java heap space
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:498)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
        at 
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)
Exception in thread "Thread-14" java.lang.NoClassDefFoundError:
org/apache/commons/httpclient/HttpMethod
        at 
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:196)
Caused by: java.lang.ClassNotFoundException: 
org.apache.commons.httpclient.HttpMethod
        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
        ... 1 more
Exception in thread "main" java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
        at org.apache.hama.util.JobManager.execute(JobManager.java:52)
        at org.apache.hama.DenseMatrix.mult(DenseMatrix.java:554)
        at Markowitz.MarkowitzMain.main(MarkowitzMain.java:33)
had...@zeus:~/HAMA$

Thank You

Abhishek Agrawal

SUNY- Buffalo
(716-435-7122)

On Wed 11/04/09 12:19 AM , "Edward J. Yoon" [email protected] sent:
> I don't see any error during perform matrix multiplication of a matrix
> with itself.
> 
> How large your matrix is? (and, let me know your cluster information)
> 
> On Wed, Nov 4, 2009 at 11:40 AM, Edward J. Yoon  [email protected]> wrote:> OK, 
> That error appears *only when* perform the
> matrix multiplication> of a matrix with itself?
> >
> > I'll check it.
> >
> > On Wed, Nov 4, 2009 at 5:38 AM,
>   lo.edu> wrote:>> Hi,
> >>   As I was telling you yesterday
> I am getting an error when I call the matrix>> multiplication function from my
program. The
> exact error I am getting is as>> under. The call to the function
> is>>
> >>      
>        
>  DenseMatrix resultMatrix=
> covMatrix.mult(covMatrix);>>
> >> I am multiplying the matrix covMatrix by
> itself just to check if the>> configuration is done correctly. But then I
> am getting an exception. I tried to>> run the default matrix multiplication
> program bin/hama examples mult matA matA>> but that worked perfectly well. SO 
> I
think
> its not that the heap is full rather I>> have not done some setting
> properly.>>
> >> had...@zeus:~
> /HAMA$ java Markowitz.MarkowitzMain>> In matrix Multiplication
> >> 03/04/26 20:29:05 INFO jvm.JvmMetrics:
> Initializing JVM Metrics with>> processName=JobTracker,
> sessionId=>> 03/04/26 20:29:05 WARN mapred.JobClient: Use
> GenericOptionsParser for parsing the>> arguments. Applications should 
> implement
> Tool for the same.>> 03/04/26 20:29:05 WARN mapred.JobClient: No
> job jar file set.  User classes may>> not be found. See JobConf(Class) or
> JobConf#setJar(String).>> 03/04/26 20:29:05 INFO
> mapred.HTableInputFormatBase: split: 0->aphrodite:,>> 03/04/26 20:29:05 INFO
mapred.JobClient:
> Running job: job_local_0001>> 03/04/26 20:29:06 INFO
> mapred.HTableInputFormatBase: split: 0->aphrodite:,>> 03/04/26 20:29:06 INFO
mapred.MapTask:
> numReduceTasks: 1>> 03/04/26 20:29:06 INFO mapred.MapTask:
> io.sort.mb = 100>> 03/04/26 20:29:06 WARN
> mapred.LocalJobRunner: job_local_0001>> java.lang.OutOfMemoryError: Java heap
> space>>      
>  at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:498)>>      
>  at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)>>      
>  at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)>>
Exception in thread "Thread-14"
> java.lang.NoClassDefFoundError:>>
> org/apache/commons/httpclient/HttpMethod>>      
>  at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:196)>>
Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.httpclient.HttpMethod>>      
>  at
> java.net.URLClassLoader$1.run(URLClassLoader.java:200)>>      
>  at java.security.AccessController.doPrivileged(Native
> Method)>>      
>  at
> java.net.URLClassLoader.findClass(URLClassLoader.java:188)>>      
>  at
> java.lang.ClassLoader.loadClass(ClassLoader.java:306)>>      
>  at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)>>      
>  at
> java.lang.ClassLoader.loadClass(ClassLoader.java:251)>>      
>  at
> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)>>      
>  ... 1 more>> Exception in thread "main"
> java.io.IOException: Job failed!>>      
>  at
> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)>>      
>  at
> org.apache.hama.util.JobManager.execute(JobManager.java:52)>>      
>  at
> org.apache.hama.DenseMatrix.mult(DenseMatrix.java:554)>>      
>  at
> Markowitz.MarkowitzMain.main(MarkowitzMain.java:32)>> had...@zeus:~
> /HAMA$>>
> >> Thank You
> >>
> >>
> >> Abhishek Agrawal
> >>
> >> SUNY- Buffalo
> >> (716-435-7122)
> >>
> >> On Tue 11/03/09  2:58 AM ,
> "Edward J. Yoon" edwardy
> [email protected] sent:>>> Hi,
> >>>
> >>> # The maximum amount of heap to use, in
> MB. Default is 1000.>>> # export HBASE_HEAPSIZE=1000
> >>>
> >>> Check above option and regarding
> temporary tables, there is no method>>> yet. But, You can delete them using
> HbaseShell's disable and delete>>> command If you have no important data in
> HDFS, re-format HDFS is good>>> idea. It's just simple tip.
> >>>
> >>> On Tue, Nov 3, 2009 at 4:39 PM,
>   lo.edu> wrote:> Hi,>>> > �  I have
> another doubt. I am running>>> a program of mine which results in>
> creation of quite a few tables in Hbase(397 to>>> be exact). Now when I am
running a>
> matrix-matrix multiplication job I am getting a>>> heap size not enough
exception.> What
> should I do to delete these 397 table>>> easily ?>
> >>> > Thank You
> >>> >
> >>> > Abhishek Agrawal
> >>> >
> >>> > SUNY- Buffalo
> >>> > (716-435-7122)
> >>> >
> >>> > On Sat 10/31/09
> � 3:25 AM ,>>> "Edward J. Yoon"
> edwardy>>> [email protected]
>  sent:>> Sounds good. Thank you, I'll update>>> wiki.>>
> >>> >> (you can also change the wiki
> pages, please>>> feel free to
> contribute!)>>>>> >> On Sat, Oct 31, 2009 at 2:28
> PM,>>> � > lo.edu>
> wrote:> Hey,>>> >> >
> ���>>>
> ��� I just ran a test.
> I>> put a println statement and it worked.>>> Thanks a lot for> your help. 
> From
> my> experience, I feel addition>>> >> of the following statements to
> the> wiki>>> page would make it easier for new users
> to>> install/ get started with>>> hama.>>> > 1. Absolute paths
> have to be given in>>> the config>> file> 2. To see
> the O/P, changes have to>>> be made to>> the source code, the
> code has to be>>>> compiled again and the necessary
> files> copied to>>> >> $HAMA_HOME>
> >>> >> > Thank You
> >>> >> >
> >>> >> > Abhishek
> Agrawal>>> >> >
> >>> >> > SUNY- Buffalo
> >>> >> > (716-435-7122)
> >>> >> >
> >>> >>
> >>> >> --
> >>> >> Best Regards, Edward J. Yoon @
> NHN,>>> corp.>> edwardy
> >>> >> [email protected]http://blog.udanax.org>>>>> >>
> >>> >>
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards, Edward J. Yoon @ NHN,
> corp.>>> edwardy
> >>> [email protected]http://blog.udanax.org>>>
> >>>
> >>>
> >>
> >>
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon @ NHN,
> corp.> edwardy
> [email protected]> http://blog.udanax.org>
> 
> 
> 
> -- 
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardy
> [email protected]http://blog.udanax.org
> 
> 
>

Reply via email to