Re: Re[2]: Error: Could not find or load main class when Running Hadoop
The problem was with the compilation folder I got using maven command : mvn clean package -Pdist -Psrc -DskipTests -Dtar The right one is: mvn clean install -Pdist -Dtar -DskipTests P.S. even after I change the classpath of extracted folder from compiled source folder to the class path of the respected taraball the error stayed. On 2018/10/14 18:24:56, Arpit Agarwal wrote: > You don't need to configure the classpath if you just extracted the Hadoop > tarball under /usr/local/. > > The only required setting is JAVA_HOME, everything else should be inferred. > Is it possible some directories got moved around after unpacking? > > > On 2018/10/14, 9:32 AM, "razo@" wrote: > > So how should I configure it? > I looked over here > https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop > to Siva's answer and the output of ($HADOOP+HOME/bin/hadoop classpath) is : > > hadoop2@master:~$ $HADOOP_HOME/bin/hadoop classpath > > /usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/local/hadoop-2.9.1/contrib/capacity-scheduler/*.jar > > But the problem stayed the same. > How should I configure the classpath to be what you suggested? > > P.S > hadoop2@master:~$ $HADOOP_HOME/bin/hadoop jar > $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar > org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+' > Error: Could not find or load main class org.apache.hadoop.util.RunJar > > > On 2018/10/14 15:55:06, ITD wrote: > > Try and check your classpath, it looks a bit strange: > > > > /usp/hdfs: what's this? > > > > Working classpath is: > > /usr/local/hadoop/etc/hadoop:\ > > /usr/local/hadoop/share/hadoop/common/lib/*:\ > > /usr/local/hadoop/share/hadoop/common/*:\ > > /usr/local/hadoop/share/hadoop/hdfs:\ > > /usr/local/hadoop/share/hadoop/hdfs/lib/*:\ > > /usr/local/hadoop/share/hadoop/hdfs/*:\ > > /usr/local/hadoop/share/hadoop/mapreduce/lib/*:\ > > /usr/local/hadoop/share/hadoop/mapreduce/*:\ > > /usr/local/hadoop/share/hadoop/yarn:\ > > /usr/local/hadoop/share/hadoop/yarn/lib/*:\ > > /usr/local/hadoop/share/hadoop/yarn/* > > > > > > > > >Воскресенье, 14 октября 2018, 18:26 +03:00 от r...@post.bgu.ac.il > : > > > > > >I tried, I still get the same error. > > > > > >On 2018/10/14 15:18:57, ITD < itdir...@mail.ru.INVALID > wrote: > > >> Class names are case-sensitive in Java, so try > > >> > > >> $HADOOP_HOME/bin/hadoop jar > $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep > input output 'dfs[a-z.]+' > > >> > > >> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz < > r...@post.bgu.ac.il >: > > >> > > > >> >I am using Hadoop 2.9.1 standalone (the folder I am using is after > successful compilation of the source code) and whenever I run a Hadoop > command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of > Hadoop) > > >> >$HADOOP_HOME/bin/hadoop jar > $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep > input output 'dfs[a-z.]+' > > >> >I get this error : > > >> >>Error: Could not find or load main class > org.apache.hadoop.util.RunJar > > >> >At first, I was thinking that I am missing some environment > variables but are they necessary for the standalone case? Why did I get this > error? (I even replaced with another example, I believe it is a classpath > problem) > > >> > > > >> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath > > >> > >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar > > >> > > >> > > >> > > >> > > > > > >- > > >To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org > > >For additional commands, e-mail: user-h...@hadoop.apache.org > > > > > > > > > > > > > > > - > To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org > For additional commands, e-mail: user-h...@hadoop.apache.org > > > >
Re: Re[2]: Error: Could not find or load main class when Running Hadoop
You don't need to configure the classpath if you just extracted the Hadoop tarball under /usr/local/. The only required setting is JAVA_HOME, everything else should be inferred. Is it possible some directories got moved around after unpacking? On 2018/10/14, 9:32 AM, "razo@" wrote: So how should I configure it? I looked over here https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop to Siva's answer and the output of ($HADOOP+HOME/bin/hadoop classpath) is : hadoop2@master:~$ $HADOOP_HOME/bin/hadoop classpath /usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/local/hadoop-2.9.1/contrib/capacity-scheduler/*.jar But the problem stayed the same. How should I configure the classpath to be what you suggested? P.S hadoop2@master:~$ $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+' Error: Could not find or load main class org.apache.hadoop.util.RunJar On 2018/10/14 15:55:06, ITD wrote: > Try and check your classpath, it looks a bit strange: > > /usp/hdfs: what's this? > > Working classpath is: > /usr/local/hadoop/etc/hadoop:\ > /usr/local/hadoop/share/hadoop/common/lib/*:\ > /usr/local/hadoop/share/hadoop/common/*:\ > /usr/local/hadoop/share/hadoop/hdfs:\ > /usr/local/hadoop/share/hadoop/hdfs/lib/*:\ > /usr/local/hadoop/share/hadoop/hdfs/*:\ > /usr/local/hadoop/share/hadoop/mapreduce/lib/*:\ > /usr/local/hadoop/share/hadoop/mapreduce/*:\ > /usr/local/hadoop/share/hadoop/yarn:\ > /usr/local/hadoop/share/hadoop/yarn/lib/*:\ > /usr/local/hadoop/share/hadoop/yarn/* > > > > >Воскресенье, 14 октября 2018, 18:26 +03:00 от r...@post.bgu.ac.il : > > > >I tried, I still get the same error. > > > >On 2018/10/14 15:18:57, ITD < itdir...@mail.ru.INVALID > wrote: > >> Class names are case-sensitive in Java, so try > >> > >> $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar Grep input output 'dfs[a-z.]+' > >> > >> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz < r...@post.bgu.ac.il >: > >> > > >> >I am using Hadoop 2.9.1 standalone (the folder I am using is after successful compilation of the source code) and whenever I run a Hadoop command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of Hadoop) > >> >$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep input output 'dfs[a-z.]+' > >> >I get this error : > >> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar > >> >At first, I was thinking that I am missing some environment variables but are they necessary for the standalone case? Why did I get this error? (I even replaced with another example, I believe it is a classpath problem) > >> > > >> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath > >> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar > >> > >> > >> > >> > > > >- > >To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org > >For additional commands, e-mail: user-h...@hadoop.apache.org > > > > > > > - To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org For additional commands, e-mail: user-h...@hadoop.apache.org
Re: Re[2]: Error: Could not find or load main class when Running Hadoop
So how should I configure it? I looked over here https://stackoverflow.com/questions/28260653/where-is-the-classpath-set-for-hadoop to Siva's answer and the output of ($HADOOP+HOME/bin/hadoop classpath) is : hadoop2@master:~$ $HADOOP_HOME/bin/hadoop classpath /usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/local/hadoop-2.9.1/contrib/capacity-scheduler/*.jar But the problem stayed the same. How should I configure the classpath to be what you suggested? P.S hadoop2@master:~$ $HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar org.apache.hadoop.examples.Grep input output 'dfs[a-z.]+' Error: Could not find or load main class org.apache.hadoop.util.RunJar On 2018/10/14 15:55:06, ITD wrote: > Try and check your classpath, it looks a bit strange: > > /usp/hdfs: what's this? > > Working classpath is: > /usr/local/hadoop/etc/hadoop:\ > /usr/local/hadoop/share/hadoop/common/lib/*:\ > /usr/local/hadoop/share/hadoop/common/*:\ > /usr/local/hadoop/share/hadoop/hdfs:\ > /usr/local/hadoop/share/hadoop/hdfs/lib/*:\ > /usr/local/hadoop/share/hadoop/hdfs/*:\ > /usr/local/hadoop/share/hadoop/mapreduce/lib/*:\ > /usr/local/hadoop/share/hadoop/mapreduce/*:\ > /usr/local/hadoop/share/hadoop/yarn:\ > /usr/local/hadoop/share/hadoop/yarn/lib/*:\ > /usr/local/hadoop/share/hadoop/yarn/* > > > > >Воскресенье, 14 октября 2018, 18:26 +03:00 от r...@post.bgu.ac.il > >: > > > >I tried, I still get the same error. > > > >On 2018/10/14 15:18:57, ITD < itdir...@mail.ru.INVALID > wrote: > >> Class names are case-sensitive in Java, so try > >> > >> $HADOOP_HOME/bin/hadoop jar > >> $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar > >> Grep input output 'dfs[a-z.]+' > >> > >> >Воскресенье, 14 октября 2018, 18:04 +03:00 от Or Raz < > >> >r...@post.bgu.ac.il >: > >> > > >> >I am using Hadoop 2.9.1 standalone (the folder I am using is after > >> >successful compilation of the source code) and whenever I run a Hadoop > >> >command such as (where$HADOOP_HOME= /usr/local/hadoop, the directory of > >> >Hadoop) > >> >$HADOOP_HOME/bin/hadoop jar > >> >$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar > >> >grep input output 'dfs[a-z.]+' > >> >I get this error : > >> >>Error: Could not find or load main class org.apache.hadoop.util.RunJar > >> >At first, I was thinking that I am missing some environment variables but > >> >are they necessary for the standalone case? Why did I get this error? (I > >> >even replaced with another example, I believe it is a classpath problem) > >> > > >> >hadoop2@master:/usr/local/hadoop-2.9.1$ bin/hadoop classpath > >> >/usr/local/hadoop-2.9.1/etc/hadoop:/usr/local/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/common/*:/usr/local/hadoop-2.9.1/share/hadoop/hdfs:/usp/hdfs/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/local/hadoop-2.9.1/share/hadoop/yarn/*:/usr/local/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop-2.9.jar > >> > >> > >> > >> > > > >- > >To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org > >For additional commands, e-mail: user-h...@hadoop.apache.org > > > > > > > - To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org For additional commands, e-mail: user-h...@hadoop.apache.org
