Hi Edward,
The class not found exception is gone. I placed my program in lib.
Secondly I noticed another thing. When my program gives the command for matrix
multiplication. I get this as O/P
03/04/28 21:46:33 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
03/04/28 21:46:33 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the
arguments. Applications should implement Tool for the same.
03/04/28 21:46:33 INFO mapred.HTableInputFormatBase: split: 0->aphrodite:,
03/04/28 21:46:34 INFO mapred.JobClient: Running job: job_local_0001
03/04/28 21:46:34 INFO mapred.HTableInputFormatBase: split: 0->aphrodite:,
03/04/28 21:46:34 INFO mapred.MapTask: numReduceTasks: 1
03/04/28 21:46:34 INFO mapred.MapTask: io.sort.mb = 100
03/04/28 21:46:34 WARN mapred.LocalJobRunner: job_local_0001
Is my job running locally ? Why is it running locally ?
Thank You
Abhishek Agrawal
SUNY- Buffalo
(716-435-7122)
On Wed 11/04/09 3:17 AM , "Edward J. Yoon" [email protected] sent:
> >> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.httpclient.HttpMethod
> Firstly, you have a ClassNotFoundException. Please add dependencies to
> your project. I would recommend to add all of {$HAMA_HOME/lib}.
>
> > 03/04/27 08:02:57 INFO mapred.MapTask:
> io.sort.mb = 100> 03/04/27 08:02:57 WARN mapred.LocalJobRunner:
> job_local_0001> java.lang.OutOfMemoryError: Java heap
> space
> Regarding OutOfMemoryError, It's hard to tell. Do you run the program
> via eclipse?
>
> On Wed, Nov 4, 2009 at 3:59 PM, lo.edu> wrote:> Hey,
> > Â Â In my program I am
> creating 2 matrices, putting values in them and then I> call the function
> >
> > DenseMatrix resultMatrix=
> covMatrix.mult(retMatrix);>
> > I am not setting the number of maps/ reduces to
> be done, the block thing. Is> there any problem because of that ?Also the full
> error I am getting is as under.> I noticed these 2 sentences mapred.MapTask:
> io.sort.mb = 100 and> mapred.LocalJobRunner: job_local_0001. Can you
> tell what is the significance of> these 2 sentences ?
> >
> > 03/04/27 08:02:56 INFO jvm.JvmMetrics:
> Initializing JVM Metrics with> processName=JobTracker, sessionId=
> > 03/04/27 08:02:56 WARN mapred.JobClient: Use
> GenericOptionsParser for parsing the> arguments. Applications should
> implement Tool
> for the same.> 03/04/27 08:02:56 WARN mapred.JobClient: No job
> jar file set. Â User classes may> not be found. See JobConf(Class) or
> JobConf#setJar(String).> 03/04/27 08:02:56 INFO
> mapred.HTableInputFormatBase: split: 0->aphrodite:,> 03/04/27 08:02:56 INFO
mapred.JobClient: Running
> job: job_local_0001> 03/04/27 08:02:57 INFO
> mapred.HTableInputFormatBase: split: 0->aphrodite:,> 03/04/27 08:02:57 INFO
mapred.MapTask:
> numReduceTasks: 1> 03/04/27 08:02:57 INFO mapred.MapTask:
> io.sort.mb = 100> 03/04/27 08:02:57 WARN mapred.LocalJobRunner:
> job_local_0001> java.lang.OutOfMemoryError: Java heap
> space> Â Â Â
> Â at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:498)> Â Â Â
> Â at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)> Â Â Â
> Â at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)>
Exception in thread "Thread-14"
> java.lang.NoClassDefFoundError:>
> org/apache/commons/httpclient/HttpMethod> Â Â Â
> Â at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:196)>
Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.httpclient.HttpMethod> Â Â Â
> Â at
> java.net.URLClassLoader$1.run(URLClassLoader.java:200)> Â Â Â
> Â at java.security.AccessController.doPrivileged(Native
> Method)> Â Â Â
> Â at
> java.net.URLClassLoader.findClass(URLClassLoader.java:188)> Â Â Â
> Â at
> java.lang.ClassLoader.loadClass(ClassLoader.java:306)> Â Â Â
> Â at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)> Â Â Â
> Â at
> java.lang.ClassLoader.loadClass(ClassLoader.java:251)> Â Â Â
> Â at
> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)> Â Â Â
> Â ... 1 more> Exception in thread "main"
> java.io.IOException: Job failed!> Â Â Â
> Â at
> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)> Â Â Â
> Â at
> org.apache.hama.util.JobManager.execute(JobManager.java:52)> Â Â Â
> Â at
> org.apache.hama.DenseMatrix.mult(DenseMatrix.java:554)> Â Â Â
> Â at
> Markowitz.MarkowitzMain.main(MarkowitzMain.java:33)> had...@zeus:~
> /HAMA$>
> > Thank You
> >
> > Abhishek Agrawal
> >
> > SUNY- Buffalo
> > (716-435-7122)
> >
> > On Wed 11/04/09 12:19 AM , "Edward J.
> Yoon" edwardy
> [email protected] sent:>> I don't see any error during perform matrix
> multiplication of a matrix>> with itself.
> >>
> >> How large your matrix is? (and, let me know
> your cluster information)>>
> >> On Wed, Nov 4, 2009 at 11:40 AM, Edward J.
> Yoon  [email protected]
> > wrote:> OK, That error appears *only when* perform
> the>> matrix multiplication> of a matrix with
> itself?>> >
> >> > I'll check it.
> >> >
> >> > On Wed, Nov 4, 2009 at 5:38
> AM,>> � lo.edu>
> wrote:>> Hi,>> >> � As I was
> telling you yesterday>> I am getting an error when I call the
> matrix>> multiplication function from my> program. The
> >> exact error I am getting is as>>
> under. The call to the function>> is>>
> >> >> Ã�Â
> Ã� Ã�>> Ã�Â
> � � �>> � DenseMatrix
> resultMatrix=>>
> covMatrix.mult(covMatrix);>>>> >> I am multiplying the matrix
> covMatrix by>> itself just to check if the>>
> configuration is done correctly. But then I>> am getting an exception. I
> tried to>>
> run the default matrix multiplication>> program bin/hama examples mult matA
> matA>> but that worked perfectly well. SO I> think
> >> its not that the heap is full rather
> I>> have not done some setting>> properly.>>
> >> >> had...@zeus:~
> >> /HAMA$ java Markowitz.MarkowitzMain>>
> In matrix Multiplication>> >> 03/04/26 20:29:05 INFO
> jvm.JvmMetrics:>> Initializing JVM Metrics with>>
> processName=JobTracker,>> sessionId=>> 03/04/26 20:29:05 WARN
> mapred.JobClient: Use>> GenericOptionsParser for parsing the>>
> arguments. Applications should implement>> Tool for the same.>> 03/04/26
> 20:29:05
> WARN mapred.JobClient: No>> job jar file set. � User
> classes may>> not be found. See JobConf(Class) or>> JobConf#setJar(String).>>
03/04/26
> 20:29:05 INFO>> mapred.HTableInputFormatBase: split:
> 0->aphrodite:,>> 03/04/26 20:29:05 INFO> mapred.JobClient:
> >> Running job: job_local_0001>> 03/04/26
> 20:29:06 INFO>> mapred.HTableInputFormatBase: split:
> 0->aphrodite:,>> 03/04/26 20:29:06 INFO> mapred.MapTask:
> >> numReduceTasks: 1>> 03/04/26 20:29:06
> INFO mapred.MapTask:>> io.sort.mb = 100>> 03/04/26 20:29:06
> WARN>> mapred.LocalJobRunner:
> job_local_0001>> java.lang.OutOfMemoryError: Java heap>> space>> Ã�Â
> � �>> � at
> >>
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:498)>>
> � � �>> � at
> >>
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)>>
> � � �>> � at
> >>
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:138)>
> ;>> Exception in thread
> "Thread-14">>
> java.lang.NoClassDefFoundError:>>>>
> org/apache/commons/httpclient/HttpMethod>> Ã�Â
> � �>> � at
> >>
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:196)>
> ;>> Caused by:
> java.lang.ClassNotFoundException:>>
> org.apache.commons.httpclient.HttpMethod>> Ã�Â
> � �>> � at
> >>
> java.net.URLClassLoader$1.run(URLClassLoader.java:200)>>
> � � �>> � at
> java.security.AccessController.doPrivileged(Native>> Method)>> Ã�Â
> � �>> � at
> >>
> java.net.URLClassLoader.findClass(URLClassLoader.java:188)>>
> � � �>> � at
> >>
> java.lang.ClassLoader.loadClass(ClassLoader.java:306)>>
> � � �>> � at
> >>
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)>>
> � � �>> � at
> >>
> java.lang.ClassLoader.loadClass(ClassLoader.java:251)>>
> � � �>> � at
> >>
> java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)>>
> � � �>> � ... 1 more>>
> Exception in thread "main">> java.io.IOException: Job failed!>>
> � � �>> � at
> >>
> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)>>
> � � �>> � at
> >>
> org.apache.hama.util.JobManager.execute(JobManager.java:52)>>
> � � �>> � at
> >>
> org.apache.hama.DenseMatrix.mult(DenseMatrix.java:554)>>
> � � �>> � at
> >>
> Markowitz.MarkowitzMain.main(MarkowitzMain.java:32)>> had...@zeus:~
> >> /HAMA$>>
> >> >> Thank You
> >> >>
> >> >>
> >> >> Abhishek Agrawal
> >> >>
> >> >> SUNY- Buffalo
> >> >> (716-435-7122)
> >> >>
> >> >> On Tue 11/03/09
> � 2:58 AM ,>> "Edward J. Yoon"
> edwardy>> [email protected]
> sent:>>> Hi,>> >>>
> >> >>> # The maximum amount of heap to
> use, in>> MB. Default is 1000.>>> # export
> HBASE_HEAPSIZE=1000>> >>>
> >> >>> Check above option and
> regarding>> temporary tables, there is no
> method>>> yet. But, You can delete them using>> HbaseShell's disable and
> delete>>>
> command If you have no important data in>> HDFS, re-format HDFS is good>>>
> idea. It's just simple tip.>> >>>
> >> >>> On Tue, Nov 3, 2009 at 4:39
> PM,>> � lo.edu> wrote:>
> Hi,>>> > Ã��Ã�Â
> I have>> another doubt. I am running>>> a
> program of mine which results in>>> creation of quite a few tables in
> Hbase(397
> to>>> be exact). Now when I am> running a>
> >> matrix-matrix multiplication job I am
> getting a>>> heap size not enough> exception.> What
> >> should I do to delete these 397
> table>>> easily ?>>> >>> > Thank You
> >> >>> >
> >> >>> > Abhishek
> Agrawal>> >>> >
> >> >>> > SUNY- Buffalo
> >> >>> > (716-435-7122)
> >> >>> >
> >> >>> > On Sat 10/31/09
> >>
> ��� 3:25 AM ,>>>
> "Edward J. Yoon">> edwardy>>> [email protected]
> >> Â sent:>> Sounds good.
> Thank you, I'll update>>> wiki.>>>> >>> >> (you can also change
> the wiki>> pages, please>>> feel free
> to>> contribute!)>>>>> >> On
> Sat, Oct 31, 2009 at 2:28>> PM,>>>
> ��� >
> lo.edu>>> wrote:> Hey,>>> >>
> >>>
> ���
> ��>>>>>
> ���
> ��� I just ran a
> test.>> I>> put a println statement and it
> worked.>>> Thanks a lot for> your help. From>> my> experience, I feel
> addition>>> >> of the following statements to>> the> wiki>>> page would make
> it
> easier for new users>> to>> install/ get started
> with>>> hama.>>> > 1. Absolute paths>> have to be given in>>> the
> config>> file> 2. To see>> the O/P, changes have to>>> be made
> to>> the source code, the>> code has to be>>>> compiled
> again and the necessary>> files> copied to>>> >>
> $HAMA_HOME>>> >>> >> > Thank
> You>> >>> >> >
> >> >>> >> >
> Abhishek>> Agrawal>>> >> >
> >> >>> >> > SUNY-
> Buffalo>> >>> >> >
> (716-435-7122)>> >>> >> >
> >> >>> >>
> >> >>> >> --
> >> >>> >> Best Regards, Edward
> J. Yoon @>> NHN,>>> corp.>>
> edwardy>> >>> >> [email protected]http://blog.udanax.org>>>>>
> >>>> >>> >>
> >> >>> >
> >> >>> >
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Best Regards, Edward J. Yoon @
> NHN,>> corp.>>> edwardy
> >> >>> [email protected]http://blog.udanax.org>>>>> >>>
> >> >>>
> >> >>
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards, Edward J. Yoon @
> NHN,>> corp.> edwardy
> >> [email protected]
> > http://blog.udanax.org>>>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon @ NHN,
> corp.>> edwardy
> >> [email protected]http://blog.udanax.org>>
> >>
> >>
> >
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardy
> [email protected]http://blog.udanax.org
>
>
>
