Actually I started to play with the latest release 0.23.0 on two nodes
yesterday. And it was easy to start the hdfs. However it took me a while to
configure the yarn. I set the variables HADOOP_COMMON_HOME to where you
extracted the tarball and HADOOP_HDFS_HOME to the local dir where I pointed the
hdfs to. After that I could bring up yarn and run the benchmark. But I am
facing a problem that I could not see the jobs in the UI. And also when I
started the historyserver, I got the following error.
11/11/30 20:53:19 FATAL hs.JobHistoryServer: Error starting JobHistoryServer
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.fs.Hdfs
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1179)
at
org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:142)
at
org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:233)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:315)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:313)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1152)
at
org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:313)
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:426)
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:448)
at
org.apache.hadoop.mapreduce.v2.hs.JobHistory.init(JobHistory.java:183)
at
org.apache.hadoop.yarn.service.CompositeService.init(CompositeService.java:58)
at
org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.init(JobHistoryServer.java:62)
at
org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:77)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.Hdfs
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1125)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1177)
... 14 more
Any clue?
Hailong
***
* Hailong Yang, PhD. Candidate
* Sino-German Joint Software Institute,
* School of Computer ScienceEngineering, Beihang University
* Phone: (86-010)82315908
* Email: hailong.yang1...@gmail.com
* Address: G413, New Main Building in Beihang University,
* No.37 XueYuan Road,HaiDian District,
* Beijing,P.R.China,100191
***
发件人： cat fa
发送时间： 2011-11-30 10:28
收件人： common-user
主题： Re: Re: [help]how to stop HDFS
In fact it's me to say sorry. I used the word install which was misleading.
In fact I downloaded a tar file and extracted it to ／usr／bin／hadoop
Could you please tell me where to point those variables？
2011/11/30, Prashant Sharma prashant.ii...@gmail.com:
I am sorry, I had no idea you have done a rpm install, my suggestion was
based on the assumption that you have done a tar extract install where all
three distribution have to extracted and then export variables.
Also I have no experience with rpm based installs - so no comments about
what went wrong in your case.
Basically from the error i can say that it is not able to find the jars
needed on classpath which is referred by scripts through
HADOOP_COMMON_HOME. I would say check with the access permission as in
which user was it installed with and which user is it running with ?
On Tue, Nov 29, 2011 at 10:48 PM, cat fa boost.subscrib...@gmail.comwrote:
Thank you for your help, but I'm still a little confused.
Suppose I installed hadoop in /usr/bin/hadoop/ .Should I
point HADOOP_COMMON_HOME to /usr/bin/hadoop ? Where should I
point HADOOP_HDFS_HOME? Also to /usr/bin/hadoop/ ?
2011/11/30 Prashant Sharma prashant.ii...@gmail.com
I mean, you have to export the variables
export HADOOP_CONF_DIR=/path/to/your/configdirectory.
also export HADOOP_HDFS_HOME ,HADOOP_COMMON_HOME. before your run your
command. I suppose this should fix the problem.
-P
On Tue, Nov 29, 2011 at 6:23 PM, cat fa boost.subscrib...@gmail.com
wrote:
it didn't work. It gave me the Usage information.
2011/11/29 hailong.yang1115 hailong.yang1...@gmail.com
Try $HADOOP_PREFIX_HOME/bin/hdfs namenode stop --config
$HADOOP_CONF_DIR
and
