Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Hi Richard, Not sure how NPE happened on you command line, but I'd like to clarify something here: 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application". 2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2. 3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers. - Zhijie On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard wrote: > Dear group, > > > > I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop > job -list or hadoop job -list all, it throws a NPE like this: > > 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is > deprecated. Instead, use dfs.metrics.session-id > > 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with > processName=JobTracker, sessionId= > > Exception in thread "main" java.lang.NullPointerException > > at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504) > > at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > > at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237) > > and on hadoop webapp like jobhistory ( I turn on the jobhistory server). > It shows no job was running and no job finishing although I was running > jobs. > > Please help me to solve this problem. > > Thanks!! > > > > Richard Chen > > > -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future. Please see my reply inline to your questions below. Richard On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen wrote: > Hi Richard, > > Not sure how NPE happened on you command line, but I'd like to clarify > something here: > 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop > job" is deprecated. If you want to see all kinds of applications run by > your YARN cluster, please use "yarn application". > > Thanks for the suggestion. I do notice that.But the result is the same though. > 2. Job history server only shows the finished mapreduce jobs. There will > be another application history server that shows all completed applications > run by YARN, but it's not available on 2.2. > > 3. ResourceManager webUI is not the job history web UI. You should check > your yarn-site.xml to see what's the address of the RM webUI. It will list > all the applications that RM remembers. > RC: RM website address inside yarn-site.xml was corrected put there. > - Zhijie > > > On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard wrote: > >> Dear group, >> >> >> >> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop >> job -list or hadoop job -list all, it throws a NPE like this: >> >> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is >> deprecated. Instead, use dfs.metrics.session-id >> >> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with >> processName=JobTracker, sessionId= >> >> Exception in thread "main" java.lang.NullPointerException >> >> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504) >> >> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312) >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >> >> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237) >> >> and on hadoop webapp like jobhistory ( I turn on the jobhistory server). >> It shows no job was running and no job finishing although I was running >> jobs. >> >> Please help me to solve this problem. >> >> Thanks!! >> >> >> >> Richard Chen >> >> >> > > > > -- > Zhijie Shen > Hortonworks Inc. > http://hortonworks.com/ > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. -- Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future. Please see my reply inline to your questions below. Richard On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen wrote: > Hi Richard, > > Not sure how NPE happened on you command line, but I'd like to clarify > something here: > 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop > job" is deprecated. If you want to see all kinds of applications run by > your YARN cluster, please use "yarn application". > > Thanks for the suggestion. I do notice that.But the result is the same though. > 2. Job history server only shows the finished mapreduce jobs. There will > be another application history server that shows all completed applications > run by YARN, but it's not available on 2.2. > > 3. ResourceManager webUI is not the job history web UI. You should check > your yarn-site.xml to see what's the address of the RM webUI. It will list > all the applications that RM remembers. > > - Zhijie > > > On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard wrote: > >> Dear group, >> >> >> >> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop >> job -list or hadoop job -list all, it throws a NPE like this: >> >> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is >> deprecated. Instead, use dfs.metrics.session-id >> >> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with >> processName=JobTracker, sessionId= >> >> Exception in thread "main" java.lang.NullPointerException >> >> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504) >> >> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312) >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >> >> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237) >> >> and on hadoop webapp like jobhistory ( I turn on the jobhistory server). >> It shows no job was running and no job finishing although I was running >> jobs. >> >> Please help me to solve this problem. >> >> Thanks!! >> >> >> >> Richard Chen >> >> >> > > > > -- > Zhijie Shen > Hortonworks Inc. > http://hortonworks.com/ > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. -- Dislike trivia, appreciate no-nonsense!
Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster
found the problem was due to missing mapred-site.xml. After creating it, the problem was gone. thanks for all the answers. On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen wrote: > Thanks for the reply. Due to company mail filtering issue, I decided to > use my personal mail account so that I can attach files freely in the > future. > > Please see my reply inline to your questions below. > > Richard > > > On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen wrote: > >> Hi Richard, >> >> Not sure how NPE happened on you command line, but I'd like to clarify >> something here: >> 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop >> job" is deprecated. If you want to see all kinds of applications run by >> your YARN cluster, please use "yarn application". >> >> Thanks for the suggestion. I do notice that.But the result is the same > though. > > >> 2. Job history server only shows the finished mapreduce jobs. There will >> be another application history server that shows all completed applications >> run by YARN, but it's not available on 2.2. >> >> 3. ResourceManager webUI is not the job history web UI. You should check >> your yarn-site.xml to see what's the address of the RM webUI. It will list >> all the applications that RM remembers. >> > > > >> - Zhijie >> >> >> On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard wrote: >> >>> Dear group, >>> >>> >>> >>> I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop >>> job -list or hadoop job -list all, it throws a NPE like this: >>> >>> 14/01/28 17:18:39 INFO Configuration.deprecation: session.id is >>> deprecated. Instead, use dfs.metrics.session-id >>> >>> 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with >>> processName=JobTracker, sessionId= >>> >>> Exception in thread "main" java.lang.NullPointerException >>> >>> at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504) >>> >>> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312) >>> >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>> >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>> >>> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237) >>> >>> and on hadoop webapp like jobhistory ( I turn on the jobhistory server). >>> It shows no job was running and no job finishing although I was running >>> jobs. >>> >>> Please help me to solve this problem. >>> >>> Thanks!! >>> >>> >>> >>> Richard Chen >>> >>> >>> >> >> >> >> -- >> Zhijie Shen >> Hortonworks Inc. >> http://hortonworks.com/ >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. > > > > > -- > Dislike trivia, appreciate no-nonsense! > -- Dislike trivia, appreciate no-nonsense!
RE: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Unsubscribe me from this list Regards, Suresh +91-9941188684 From: Richard Chen [mailto:cxd3...@gmail.com] Sent: Monday, February 24, 2014 11:38 AM To: user@hadoop.apache.org Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster found the problem was due to missing mapred-site.xml. After creating it, the problem was gone. thanks for all the answers. On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen mailto:cxd3...@gmail.com>> wrote: Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future. Please see my reply inline to your questions below. Richard On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen mailto:zs...@hortonworks.com>> wrote: Hi Richard, Not sure how NPE happened on you command line, but I'd like to clarify something here: 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application". Thanks for the suggestion. I do notice that.But the result is the same though. 2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2. 3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers. - Zhijie On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard mailto:richard.c...@intel.com>> wrote: Dear group, I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this: 14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504) at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237) and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs. Please help me to solve this problem. Thanks!! Richard Chen -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Dislike trivia, appreciate no-nonsense! -- Dislike trivia, appreciate no-nonsense! Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please notify us immediately at mailmas...@mphasis.com and delete this mail from your records.
RE: No job shown in Hadoop resource manager web UI when running jobs in the cluster
Send a simple mail to user-unsubscr...@hadoop.apache.org<mailto:user-unsubscr...@hadoop.apache.org> FYI, http://hadoop.apache.org/mailing_lists.html From: Suresh M03 [mailto:suresh@mphasis.com] Sent: 24 February 2014 11:40 To: user@hadoop.apache.org Subject: RE: No job shown in Hadoop resource manager web UI when running jobs in the cluster Unsubscribe me from this list Regards, Suresh +91-9941188684 From: Richard Chen [mailto:cxd3...@gmail.com] Sent: Monday, February 24, 2014 11:38 AM To: user@hadoop.apache.org<mailto:user@hadoop.apache.org> Subject: Re: No job shown in Hadoop resource manager web UI when running jobs in the cluster found the problem was due to missing mapred-site.xml. After creating it, the problem was gone. thanks for all the answers. On Sun, Feb 23, 2014 at 9:05 AM, Richard Chen mailto:cxd3...@gmail.com>> wrote: Thanks for the reply. Due to company mail filtering issue, I decided to use my personal mail account so that I can attach files freely in the future. Please see my reply inline to your questions below. Richard On Sat, Feb 22, 2014 at 3:55 AM, Zhijie Shen mailto:zs...@hortonworks.com>> wrote: Hi Richard, Not sure how NPE happened on you command line, but I'd like to clarify something here: 1. If you want to see mapreduce jobs, please use "mapred job". "hadoop job" is deprecated. If you want to see all kinds of applications run by your YARN cluster, please use "yarn application". Thanks for the suggestion. I do notice that.But the result is the same though. 2. Job history server only shows the finished mapreduce jobs. There will be another application history server that shows all completed applications run by YARN, but it's not available on 2.2. 3. ResourceManager webUI is not the job history web UI. You should check your yarn-site.xml to see what's the address of the RM webUI. It will list all the applications that RM remembers. - Zhijie On Thu, Feb 20, 2014 at 7:04 PM, Chen, Richard mailto:richard.c...@intel.com>> wrote: Dear group, I compiled hadoop 2.2.0 x64 and running it on a cluster. When I do hadoop job -list or hadoop job -list all, it throws a NPE like this: 14/01/28 17:18:39 INFO Configuration.deprecation: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id 14/01/28 17:18:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.mapreduce.tools.CLI.listJobs(CLI.java:504) at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:312) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237) and on hadoop webapp like jobhistory ( I turn on the jobhistory server). It shows no job was running and no job finishing although I was running jobs. Please help me to solve this problem. Thanks!! Richard Chen -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Dislike trivia, appreciate no-nonsense! -- Dislike trivia, appreciate no-nonsense! Information transmitted by this e-mail is proprietary to MphasiS, its associated companies and/ or its customers and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please notify us immediately at mailmas...@mphasis.com<mailto:mailmas...@mphasis.com> and delete this mail from your records.
