subject:"RE\: Yarn not picking up the Resource manager configurations"

Re: Yarn not picking up the Resource manager configurations

2017-05-29 Thread Gurmukh Singh


it is  ".bashrc" not "*.barshrc"*

is port 8030 listening ?

netstat -tlpn | grep 8030

Can you give the output of:

$ hdfs getconf -confkey yarn.resourcemanager.scheduler.address

$ hdfs getconf -confkey fs.defaultFS

$ nslookup nn1.cluster1.com (if using DNS), else

 $ getent hosts nn1.cluster1.com

$ ifconfig from the host where resourcemanager is running

Refer my github for configs: https://github.com/netxillon/hadoop*
*

*
*

*
*

*
*


On 19/5/17 6:39 pm, Dhanushka Parakrama wrote:

Hi All

I have setup the multi node hadoop cluster . but when is run the 
simple map reduce job it gives me the following error the in  the data 
nodes container log .



Im running the hadoop 2.7.3 version

In namenode which contain resource manager as well . i ran below job

hadoop jar hadoop-mapreduce-examples-2.7.3.jar wordcount /words  /output

Error


tryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 
MILLISECONDS)
2017-05-19 07:57:14,340 INFO [main] org.apache.hadoop.ipc.Client: 
Retrying connect to server: 0.0.0.0/0.0.0.0:8030 
. Already tried 8 time(s); retry policy 
is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 
MILLISECONDS)
2017-05-19 07:57:15,341 INFO [main] org.apache.hadoop.ipc.Client: 
Retrying connect to server: 0.0.0.0/0.0.0.0:8030 
. Already tried 9 time(s); retry policy 
is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 
MILLISECONDS)



Configurations
===
*
*
*vi .barshrc file*

export JAVA_HOME=/home/ubuntu/jdk1.8.0_45
export HADOOP_HOME=/usr/local/hadoop/current
#export PATH=$JAVA_HOME/bin:$PATH
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
export ZOOKEEPER_HOME=/usr/local/zookeeper/current

export 
PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$ZOOKEEPER_HOME/bin



*vi .yarn-site.xml file*







   yarn.nodemanager.aux-services
   mapreduce_shuffle
 


yarn.nodemanager.aux-services.mapreduce.shuffle.class
org.apache.hadoop.mapred.ShuffleHandler



yarn.resourcemanager.hostname
nn1.cluster.com 


 
yarn.resourcemanager.scheduler.address
  nn1.cluster.com:8030 
 






Can any expert help me with this issue , im stuck at this point

Thank You

RE: Yarn not picking up the Resource manager configurations

2017-05-20 Thread Brahma Reddy Battula


Looks configurations(mainly yarn-site.xml here which you pointed) are not 
loaded in classpath. Can you check the output of “hadoop classpath”..?



Regards
Brahma Reddy Battula

From: Dhanushka Parakrama [mailto:parakrama1...@gmail.com]
Sent: 19 May 2017 16:40
To: user@hadoop.apache.org
Subject: Yarn not picking up the Resource manager configurations

Hi All
I have setup the multi node hadoop cluster . but when is run the simple map 
reduce job it gives me the following error the in  the data nodes container log 
.

Im running the hadoop 2.7.3 version

In namenode which contain resource manager as well . i ran below job

hadoop jar hadoop-mapreduce-examples-2.7.3.jar wordcount /words  /output

Error


tryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-05-19 07:57:14,340 INFO [main] org.apache.hadoop.ipc.Client: Retrying 
connect to server: 0.0.0.0/0.0.0.0:8030. Already 
tried 8 time(s); retry policy is 
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-05-19 07:57:15,341 INFO [main] org.apache.hadoop.ipc.Client: Retrying 
connect to server: 0.0.0.0/0.0.0.0:8030. Already 
tried 9 time(s); retry policy is 
RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

Configurations
===
vi .barshrc file

export JAVA_HOME=/home/ubuntu/jdk1.8.0_45
export HADOOP_HOME=/usr/local/hadoop/current
#export PATH=$JAVA_HOME/bin:$PATH
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
export ZOOKEEPER_HOME=/usr/local/zookeeper/current

export 
PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$ZOOKEEPER_HOME/bin

vi .yarn-site.xml file







   yarn.nodemanager.aux-services
   mapreduce_shuffle
 


   yarn.nodemanager.aux-services.mapreduce.shuffle.class
   org.apache.hadoop.mapred.ShuffleHandler



yarn.resourcemanager.hostname
nn1.cluster.com


 
  yarn.resourcemanager.scheduler.address
  nn1.cluster.com:8030
 





Can any expert help me with this issue , im stuck at this point
Thank You

Re: Yarn not picking up the Resource manager configurations

2017-05-19 Thread Miklos Szegedi

Hi,

I usually get this error when the yarn-site.xml is not on the class path
and it picks up a default 0.0.0.0. I would try:
1. bin/yarn jar
./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar pi 1 1000
2. explicitly adding $HADOOP_HOME/etc/hadoop to your client classpath
3. make sure $HADOOP_HOME/etc/hadoop contains your yarn-site.xml and it is
not somewhere else
4. nn1.cluster.com: you actually need to attach to the resource manager
host name not the name node, if they are running on different hosts.

I hope this helps.

Thanks,
Miklos


On Fri, May 19, 2017 at 1:39 AM, Dhanushka Parakrama <
parakrama1...@gmail.com> wrote:

> Hi All
>
> I have setup the multi node hadoop cluster . but when is run the simple
> map reduce job it gives me the following error the in  the data nodes
> container log .
>
>
> Im running the hadoop 2.7.3 version
>
> In namenode which contain resource manager as well . i ran below job
>
> hadoop jar hadoop-mapreduce-examples-2.7.3.jar wordcount /words  /output
>
> Error
> 
>
> tryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000
> MILLISECONDS)
> 2017-05-19 07:57:14,340 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry
> policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS)
> 2017-05-19 07:57:15,341 INFO [main] org.apache.hadoop.ipc.Client: Retrying
> connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry
> policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10,
> sleepTime=1000 MILLISECONDS)
>
>
> Configurations
> ===
>
> *vi .barshrc file*
>
> export JAVA_HOME=/home/ubuntu/jdk1.8.0_45
> export HADOOP_HOME=/usr/local/hadoop/current
> #export PATH=$JAVA_HOME/bin:$PATH
> export HADOOP_MAPRED_HOME=$HADOOP_HOME
> export HADOOP_COMMON_HOME=$HADOOP_HOME
> export HADOOP_HDFS_HOME=$HADOOP_HOME
> export YARN_HOME=$HADOOP_HOME
> export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
> export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
> export ZOOKEEPER_HOME=/usr/local/zookeeper/current
>
> export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/
> sbin:$ZOOKEEPER_HOME/bin
>
>
> *vi .yarn-site.xml file*
>
>
> 
>
> 
>
> 
>yarn.nodemanager.aux-services
>mapreduce_shuffle
>  
>
> 
>yarn.nodemanager.aux-services.mapreduce.shuffle.class
>org.apache.hadoop.mapred.ShuffleHandler
> 
>
> 
> yarn.resourcemanager.hostname
> nn1.cluster.com
> 
>
>  
>   yarn.resourcemanager.scheduler.address
>   nn1.cluster.com:8030
>  
>
>
> 
>
>
>
> Can any expert help me with this issue , im stuck at this point
>
> Thank You
>
>
>
>
>

Re: Yarn not picking up the Resource manager configurations

RE: Yarn not picking up the Resource manager configurations

Re: Yarn not picking up the Resource manager configurations

3 matches

Site Navigation

Mail list logo

Footer information