Running Distributed shell in hadoop0.23

2011-12-14 Thread sri ram 
Hi,
 Can anyone give the procedure about how to run Distibuted shell
example in hadoop yarn.So that i try to understand how applicatin master
really works.

Running Distributed shell in hadoop0.23

2011-12-14 Thread sri ram 
Hi,
 Can anyone give the procedure about how to run Distibuted shell
example in hadoop yarn.So that i try to understand how applicatin master
really works.

Error while starting datanode in hadoop 0.23 in secure mode

2011-12-13 Thread sri ram 
Hi,
 I receive the following error while starting datanode in secure
mode of hadoop 0.23

2011-12-14 14:35:48,468 INFO  http.HttpServer
(HttpServer.java:addGlobalFilter(476)) - Added global filter 'safety'
(class=org.apache.hadoop.http.HttpServer$
2011-12-14 14:35:48,471 WARN  lib.StaticUserWebFilter
(StaticUserWebFilter.java:getUsernameFromConf(141)) - dfs.web.ugi should
not be used. Instead, use had$
2011-12-14 14:35:48,472 INFO  http.HttpServer
(HttpServer.java:addFilter(454)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
2011-12-14 14:35:48,472 INFO  http.HttpServer
(HttpServer.java:addFilter(461)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
2011-12-14 14:35:48,473 INFO  http.HttpServer
(HttpServer.java:addFilter(461)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
jetty.ssl.password : jetty.ssl.keypassword : 2011-12-14 14:35:53,553 INFO
mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26
2011-12-14 14:35:54,044 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Started SelectChannelConnector@master:1006
2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
java.lang.NullPointerException
2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) - failed
Krb5AndCertsSslSocketConnector@0.0.0.0:1005: java.io.IOException:
!JsseListener:$
2011-12-14 14:35:54,048 WARN  mortbay.log (Slf4jLog.java:warn(76)) - failed
Server@1867df9: java.io.IOException: !JsseListener:
java.lang.NullPointerExcepti$
2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped Krb5AndCertsSslSocketConnector@0.0.0.0:1005
2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped SelectChannelConnector@master:1006
2011-12-14 14:35:54,189 INFO  datanode.DataNode
(DataNode.java:shutdown(1741)) - Waiting for threadgroup to exit, active
threads is 0
2011-12-14 14:35:54,190 ERROR datanode.DataNode
(DataNode.java:secureMain(2371)) - Exception in secureMain
java.io.IOException: !JsseListener: java.lang.NullPointerException
at
org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)
at
org.apache.hadoop.security.Krb5AndCertsSslSocketConnector.newServerSocket(Krb5AndCertsSslSocketConnector.java:123)
at
org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at
org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
at
org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)

Is there any way to resolve this???

Error while starting datanode in hadoop 0.23 in secure mode

2011-12-13 Thread sri ram 
Hi,
 I receive the following error while starting datanode in secure
mode of hadoop 0.23

2011-12-14 14:35:48,468 INFO  http.HttpServer
(HttpServer.java:addGlobalFilter(476)) - Added global filter 'safety'
(class=org.apache.hadoop.http.HttpServer$
2011-12-14 14:35:48,471 WARN  lib.StaticUserWebFilter
(StaticUserWebFilter.java:getUsernameFromConf(141)) - dfs.web.ugi should
not be used. Instead, use had$
2011-12-14 14:35:48,472 INFO  http.HttpServer
(HttpServer.java:addFilter(454)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
2011-12-14 14:35:48,472 INFO  http.HttpServer
(HttpServer.java:addFilter(461)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
2011-12-14 14:35:48,473 INFO  http.HttpServer
(HttpServer.java:addFilter(461)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
jetty.ssl.password : jetty.ssl.keypassword : 2011-12-14 14:35:53,553 INFO
mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26
2011-12-14 14:35:54,044 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Started SelectChannelConnector@master:1006
2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
java.lang.NullPointerException
2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) - failed
Krb5AndCertsSslSocketConnector@0.0.0.0:1005: java.io.IOException:
!JsseListener:$
2011-12-14 14:35:54,048 WARN  mortbay.log (Slf4jLog.java:warn(76)) - failed
Server@1867df9: java.io.IOException: !JsseListener:
java.lang.NullPointerExcepti$
2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped Krb5AndCertsSslSocketConnector@0.0.0.0:1005
2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped SelectChannelConnector@master:1006
2011-12-14 14:35:54,189 INFO  datanode.DataNode
(DataNode.java:shutdown(1741)) - Waiting for threadgroup to exit, active
threads is 0
2011-12-14 14:35:54,190 ERROR datanode.DataNode
(DataNode.java:secureMain(2371)) - Exception in secureMain
java.io.IOException: !JsseListener: java.lang.NullPointerException
at
org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)
at
org.apache.hadoop.security.Krb5AndCertsSslSocketConnector.newServerSocket(Krb5AndCertsSslSocketConnector.java:123)
at
org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at
org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
at
org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)

Is there any way to resolve this???

Re: Error while starting datanode in hadoop 0.23 in secure mode

2011-12-13 Thread sri ram 
Thanks for the reply,
   I have tried with the ip of the individual systems also.But
the same eroor reoccurs

On Tue, Dec 13, 2011 at 3:40 PM, alo alt wget.n...@googlemail.com wrote:

 Hi,

 2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
 failed Krb5AndCertsSslSocketConnector@0.0.0.0:1005:
 java.io.IOException: !JsseListener:$
 2011-12-14 14:35:54,048 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
 failed Server@1867df9: java.io.IOException: !JsseListener:
 java.lang.NullPointerExcepti$
 2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
 Stopped Krb5AndCertsSslSocketConnector@0.0.0.0:1005

 0.0.0.0 as Kerberos - IP can't work.

 - Alex

 On Tue, Dec 13, 2011 at 10:27 AM, sri ram rsriram...@gmail.com wrote:
  Hi,
   I receive the following error while starting datanode in secure
  mode of hadoop 0.23
 
  2011-12-14 14:35:48,468 INFO  http.HttpServer
  (HttpServer.java:addGlobalFilter(476)) - Added global filter 'safety'
  (class=org.apache.hadoop.http.HttpServer$
  2011-12-14 14:35:48,471 WARN  lib.StaticUserWebFilter
  (StaticUserWebFilter.java:getUsernameFromConf(141)) - dfs.web.ugi should
 not
  be used. Instead, use had$
  2011-12-14 14:35:48,472 INFO  http.HttpServer
  (HttpServer.java:addFilter(454)) - Added filter static_user_filter
  (class=org.apache.hadoop.http.lib.StaticUse$
  2011-12-14 14:35:48,472 INFO  http.HttpServer
  (HttpServer.java:addFilter(461)) - Added filter static_user_filter
  (class=org.apache.hadoop.http.lib.StaticUse$
  2011-12-14 14:35:48,473 INFO  http.HttpServer
  (HttpServer.java:addFilter(461)) - Added filter static_user_filter
  (class=org.apache.hadoop.http.lib.StaticUse$
  jetty.ssl.password : jetty.ssl.keypassword : 2011-12-14 14:35:53,553 INFO
  mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26
  2011-12-14 14:35:54,044 INFO  mortbay.log (Slf4jLog.java:info(67)) -
 Started
  SelectChannelConnector@master:1006
  2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
  java.lang.NullPointerException
  2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
 failed
  Krb5AndCertsSslSocketConnector@0.0.0.0:1005: java.io.IOException:
  !JsseListener:$
  2011-12-14 14:35:54,048 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
 failed
  Server@1867df9: java.io.IOException: !JsseListener:
  java.lang.NullPointerExcepti$
  2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
 Stopped
  Krb5AndCertsSslSocketConnector@0.0.0.0:1005
  2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
 Stopped
  SelectChannelConnector@master:1006
  2011-12-14 14:35:54,189 INFO  datanode.DataNode
  (DataNode.java:shutdown(1741)) - Waiting for threadgroup to exit, active
  threads is 0
  2011-12-14 14:35:54,190 ERROR datanode.DataNode
  (DataNode.java:secureMain(2371)) - Exception in secureMain
  java.io.IOException: !JsseListener: java.lang.NullPointerException
  at
 
 org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)
  at
 
 org.apache.hadoop.security.Krb5AndCertsSslSocketConnector.newServerSocket(Krb5AndCertsSslSocketConnector.java:123)
  at
  org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
  at
  org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
  at
  org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
 
  Is there any way to resolve this???
 



 --
 Alexander Lorenz
 http://mapredit.blogspot.com

 P Think of the environment: please don't print this email unless you
 really need to.

Re: Error while starting datanode in hadoop 0.23 in secure mode

2011-12-13 Thread sri ram 
telnet master ip says unable to connect to remote host.
This is my following property for datanode in secure mode
property
namedfs.datanode.https.address/name
valuemaster:1005/value
/property


On Tue, Dec 13, 2011 at 4:18 PM, alo alt wget.n...@googlemail.com wrote:

 Check if kerberos respond:

 telnet KERBEROS_IP 1005

 As I know use kerberos per default ports 88, AFS token 746 (), kx509
 9878

 - Alex


 On Tue, Dec 13, 2011 at 11:39 AM, sri ram rsriram...@gmail.com wrote:
  Thanks for the reply,
 I have tried with the ip of the individual systems
 also.But
  the same eroor reoccurs
 
 
  On Tue, Dec 13, 2011 at 3:40 PM, alo alt wget.n...@googlemail.com
 wrote:
 
  Hi,
 
  2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
  failed Krb5AndCertsSslSocketConnector@0.0.0.0:1005:
  java.io.IOException: !JsseListener:$
  2011-12-14 14:35:54,048 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
  failed Server@1867df9: java.io.IOException: !JsseListener:
  java.lang.NullPointerExcepti$
  2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
  Stopped Krb5AndCertsSslSocketConnector@0.0.0.0:1005
 
  0.0.0.0 as Kerberos - IP can't work.
 
  - Alex
 
  On Tue, Dec 13, 2011 at 10:27 AM, sri ram rsriram...@gmail.com wrote:
   Hi,
I receive the following error while starting datanode in
 secure
   mode of hadoop 0.23
  
   2011-12-14 14:35:48,468 INFO  http.HttpServer
   (HttpServer.java:addGlobalFilter(476)) - Added global filter 'safety'
   (class=org.apache.hadoop.http.HttpServer$
   2011-12-14 14:35:48,471 WARN  lib.StaticUserWebFilter
   (StaticUserWebFilter.java:getUsernameFromConf(141)) - dfs.web.ugi
 should
   not
   be used. Instead, use had$
   2011-12-14 14:35:48,472 INFO  http.HttpServer
   (HttpServer.java:addFilter(454)) - Added filter static_user_filter
   (class=org.apache.hadoop.http.lib.StaticUse$
   2011-12-14 14:35:48,472 INFO  http.HttpServer
   (HttpServer.java:addFilter(461)) - Added filter static_user_filter
   (class=org.apache.hadoop.http.lib.StaticUse$
   2011-12-14 14:35:48,473 INFO  http.HttpServer
   (HttpServer.java:addFilter(461)) - Added filter static_user_filter
   (class=org.apache.hadoop.http.lib.StaticUse$
   jetty.ssl.password : jetty.ssl.keypassword : 2011-12-14 14:35:53,553
   INFO
   mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26
   2011-12-14 14:35:54,044 INFO  mortbay.log (Slf4jLog.java:info(67)) -
   Started
   SelectChannelConnector@master:1006
   2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
   java.lang.NullPointerException
   2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
   failed
   Krb5AndCertsSslSocketConnector@0.0.0.0:1005: java.io.IOException:
   !JsseListener:$
   2011-12-14 14:35:54,048 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
   failed
   Server@1867df9: java.io.IOException: !JsseListener:
   java.lang.NullPointerExcepti$
   2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
   Stopped
   Krb5AndCertsSslSocketConnector@0.0.0.0:1005
   2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
   Stopped
   SelectChannelConnector@master:1006
   2011-12-14 14:35:54,189 INFO  datanode.DataNode
   (DataNode.java:shutdown(1741)) - Waiting for threadgroup to exit,
 active
   threads is 0
   2011-12-14 14:35:54,190 ERROR datanode.DataNode
   (DataNode.java:secureMain(2371)) - Exception in secureMain
   java.io.IOException: !JsseListener: java.lang.NullPointerException
   at
  
  
 org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)
   at
  
  
 org.apache.hadoop.security.Krb5AndCertsSslSocketConnector.newServerSocket(Krb5AndCertsSslSocketConnector.java:123)
   at
   org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
   at
  
 org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
   at
  
 org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
  
   Is there any way to resolve this???
  
 
 
 
  --
  Alexander Lorenz
  http://mapredit.blogspot.com
 
  P Think of the environment: please don't print this email unless you
  really need to.
 
 



 --
 Alexander Lorenz
 http://mapredit.blogspot.com

 P Think of the environment: please don't print this email unless you
 really need to.

Re: Error while starting datanode in hadoop 0.23 in secure mode

2011-12-13 Thread sri ram 
 16:19:28,101 WARN  mortbay.log (Slf4jLog.java:warn(76)) - failed
Server@179d854: java.io.IOException: !JsseListener:
java.lang.NullPointerExcepti$
2011-12-13 16:19:28,104 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped Krb5AndCertsSslSocketConnector@master:1005
2011-12-13 16:19:28,104 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped SelectChannelConnector@master:1006
2011-12-13 16:19:28,107 INFO  datanode.DataNode
(DataNode.java:shutdown(1741)) - Waiting for threadgroup to exit, active
threads is 0
2011-12-13 16:19:28,108 ERROR datanode.DataNode
(DataNode.java:secureMain(2371)) - Exception in secureMain
java.io.IOException: !JsseListener: java.lang.NullPointerException
at
org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)
at
org.apache.hadoop.security.Krb5AndCertsSslSocketConnector.newServerSocket(Krb5AndCertsSslSocketConnector.java:123)
at
org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at
org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
at
org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)

at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.Server.doStart(Server.java:235)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:639)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startInfoServer(DataNode.java:575)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1501)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.init(DataNode.java:457)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2263)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2196)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2219)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2367)
at
org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:188)
2011-12-13 16:19:28,111 INFO  datanode.DataNode (StringUtils.java:run(605))
- SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down DataNode at master.example.com/147.128.152.179


On Tue, Dec 13, 2011 at 4:23 PM, sri ram rsriram...@gmail.com wrote:

 telnet master ip says unable to connect to remote host.
 This is my following property for datanode in secure mode
 property
 namedfs.datanode.https.address/name
 valuemaster:1005/value
 /property



 On Tue, Dec 13, 2011 at 4:18 PM, alo alt wget.n...@googlemail.com wrote:

 Check if kerberos respond:

 telnet KERBEROS_IP 1005

 As I know use kerberos per default ports 88, AFS token 746 (), kx509
 9878

 - Alex


 On Tue, Dec 13, 2011 at 11:39 AM, sri ram rsriram...@gmail.com wrote:
  Thanks for the reply,
 I have tried with the ip of the individual systems
 also.But
  the same eroor reoccurs
 
 
  On Tue, Dec 13, 2011 at 3:40 PM, alo alt wget.n...@googlemail.com
 wrote:
 
  Hi,
 
  2011-12-14 14:35:54,047 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
  failed Krb5AndCertsSslSocketConnector@0.0.0.0:1005:
  java.io.IOException: !JsseListener:$
  2011-12-14 14:35:54,048 WARN  mortbay.log (Slf4jLog.java:warn(76)) -
  failed Server@1867df9: java.io.IOException: !JsseListener:
  java.lang.NullPointerExcepti$
  2011-12-14 14:35:54,085 INFO  mortbay.log (Slf4jLog.java:info(67)) -
  Stopped Krb5AndCertsSslSocketConnector@0.0.0.0:1005
 
  0.0.0.0 as Kerberos - IP can't work.
 
  - Alex
 
  On Tue, Dec 13, 2011 at 10:27 AM, sri ram rsriram...@gmail.com
 wrote:
   Hi,
I receive the following error while starting datanode in
 secure
   mode of hadoop 0.23
  
   2011-12-14 14:35:48,468 INFO  http.HttpServer
   (HttpServer.java:addGlobalFilter(476)) - Added global filter 'safety'
   (class=org.apache.hadoop.http.HttpServer$
   2011-12-14 14:35:48,471 WARN  lib.StaticUserWebFilter
   (StaticUserWebFilter.java:getUsernameFromConf(141)) - dfs.web.ugi
 should
   not
   be used. Instead, use had$
   2011-12-14 14:35:48,472 INFO  http.HttpServer
   (HttpServer.java:addFilter(454)) - Added filter static_user_filter
   (class=org.apache.hadoop.http.lib.StaticUse$
   2011-12-14 14:35:48,472 INFO  http.HttpServer
   (HttpServer.java:addFilter(461)) - Added filter static_user_filter
   (class=org.apache.hadoop.http.lib.StaticUse$
   2011-12-14

Re: Error while starting datanode in hadoop 0.23 in secure mode

2011-12-13 Thread sri ram 
Both master and master.example.com points to the current local address.
Namenode is running well.

On Tue, Dec 13, 2011 at 4:57 PM, alo alt wget.n...@googlemail.com wrote:

 Hi,

 master.example.com? I don't think that a NN is running there ;)
 And master are available in DNS?

 The config looks misconfigured, you have to setup a working environment.

 - Alex


 On Tue, Dec 13, 2011 at 12:02 PM, sri ram rsriram...@gmail.com wrote:
  The following is the content of mapred-site.xml
  ?xml version=1.0?
  ?xml-stylesheet href=configuration.xsl?
  configuration
  property
  namedfs.replication/name
  value1/value
  /property
  property
  namedfs.permissions/name
  valuefalse/value
  /property
  property
  namedfs.namenode.name.dir/name
  value/app/tmp/name/value
  /property
  property
  namedfs.datanode.data.dir/name
  value/app/tmp/data/value
  /property
  !--kerberos--
  !--NAMENODE CONF--
  property
  namedfs.block.access.token.enable/name
  valuetrue/value
  /property
  property
  namedfs.https.enable/name
  valuetrue/value
  /property
  property
  namedfs.namenode.https-address/name
  valuemaster.example.com:50470/value
  /property
  property
  namedfs.https.port/name
  value50470/value
  /property
  property
  namedfs.namenode.keytab.file/name
  value/etc/security/keytab/nn.service.keytab/value
  /property
  property
  namedfs.namenode.kerberos.principal/name
  valuenn/master.example@example.com/value
  /property
  property
  namedfs.namenode.kerberos.https.principal/name
  valuehost/master.example@example.com/value
  /property
  !--DATANODE CONF--
  property
  namedfs.datanode.data.dir.perm/name
  value700/value
  /property
  property
  namedfs.datanode.address/name
  valuemaster:1003/value
  /property
 
  property
  namedfs.datanode.https.address/name
  valuemaster:1005/value
  /property
  property
  namedfs.datanode.http.address/name
  valuemaster:1006/value
  /property
 
  property
  namedfs.datanode.keytab.file/name
  value/etc/security/keytab/dn.service.keytab/value
  /property
  property
  namedfs.datanode.kerberos.principal/name
  valuedn/master.example@example.com/value
  /property
  property
  namedfs.datanode.kerberos.https.principal/name
  valuehost/master.example@example.com/value
  /property
 
  /configuration
 
  The following is the error log while starting datanode as root
  2011-12-13 16:19:26,837 INFO  datanode.DataNode
  (StringUtils.java:startupShutdownMessage(589)) - STARTUP_MSG:
  /
  STARTUP_MSG: Starting DataNode
  STARTUP_MSG:   host = master.example.com/147.128.152.179
  STARTUP_MSG:   args = []
  STARTUP_MSG:   version = 0.23.0
  STARTUP_MSG:   classpath =
 
 /usr/local/hadoop/conf:/usr/local/hadoop/libexec/../share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop/libexec/../share$
  STARTUP_MSG:   build =
  git://
 devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common
  -r d4fee83ec1462ab9824add6449320617$
  /
  2011-12-13 16:19:26,921 WARN  common.Util (Util.java:stringAsURI(63)) -
 Path
  /app/tmp/data should be specified as a URI in configuration files. Please
  updat$
  2011-12-13 16:19:27,314 INFO  security.UserGroupInformation
  (UserGroupInformation.java:loginUserFromKeytab(633)) - Login successful
 for
  user dn/master.examp$
  2011-12-13 16:19:27,454 WARN  impl.MetricsConfig
  (MetricsConfig.java:loadFirst(125)) - Cannot locate configuration: tried
  hadoop-metrics2-datanode.propertie$
  2011-12-13 16:19:27,548 INFO  impl.MetricsSystemImpl
  (MetricsSystemImpl.java:startTimer(343)) - Scheduled snapshot period at
 10
  second(s).
  2011-12-13 16:19:27,548 INFO  impl.MetricsSystemImpl
  (MetricsSystemImpl.java:start(182)) - DataNode metrics system started
  2011-12-13 16:19:27,549 INFO  impl.MetricsSystemImpl
  (MetricsSystemImpl.java:registerSource(244)) - Registered source
 UgiMetrics
  2011-12-13 16:19:27,572 INFO  datanode.DataNode
  (DataNode.java:initDataXceiver(701)) - Opened info server at 1003
  2011-12-13 16:19:27,585 INFO  datanode.DataNode
  (DataXceiverServer.java:init(77)) - Balancing bandwith is 1048576
 bytes/s
  2011-12-13 16:19:27,617 INFO  mortbay.log (Slf4jLog.java:info(67)) -
 Logging
  to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
  org.mortbay.log.Slf4j$
  2011-12-13 16:19:27,703 INFO  http.HttpServer
  (HttpServer.java:addGlobalFilter(476)) - Added global filter 'safety'
  (class=org.apache.hadoop.http.HttpServer$
  2011-12-13 16:19:27,706 WARN  lib.StaticUserWebFilter
  (StaticUserWebFilter.java:getUsernameFromConf(141)) - dfs.web.ugi should
 not
  be used. Instead, use had$
  2011-12-13 16:19:27,707 INFO  http.HttpServer
  (HttpServer.java:addFilter(454)) - Added filter static_user_filter
  (class=org.apache.hadoop.http.lib.StaticUse$
  2011-12-13 16:19:27,707 INFO  http.HttpServer
  (HttpServer.java:addFilter(461)) - Added filter static_user_filter

Cannot start secure cluster without privileged resources.

2011-12-12 Thread sri ram 
HI,
   I TRIED INSTALLING HADOOP SECURE MODE IN 0.23 VERSION.I STUCK UP
WITH THE ERROR OF

java.lang.RuntimeException: Cannot start secure cluster without privileged
resources.
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1487)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.init(DataNode.java:457)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2263)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2196)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2219)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2367)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2384)
2011-12-12 13:24:04,419 INFO  datanode.DataNode (StringUtils.java:run(605))
- SHUTDOWN_MSG:

I tried this post
http://www.mail-archive.com/common-user@hadoop.apache.org/msg13679.html
But fail to configure.

hadoop 0.23 secure mode error

2011-12-12 Thread sri ram 
Hi,
  I am trying to form a hadoop cluster of 0.23 version in secure
mode.
   While starting nodemanager i get the following error
2011-12-12 15:37:26,874 INFO  ipc.HadoopYarnRPC
(HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc
proxy for protocol interface org.apac$
2011-12-12 15:37:26,953 INFO  nodemanager.NodeStatusUpdaterImpl
(NodeStatusUpdaterImpl.java:registerWithRM(155)) - Connected to
ResourceManager at master:80$
2011-12-12 15:37:38,784 WARN  ipc.Client (Client.java:run(526)) - Couldn't
setup connection for nm/ad...@master.example.com to rm/
ad...@master.example.com
2011-12-12 15:37:38,787 ERROR service.CompositeService
(CompositeService.java:start(72)) - Error starting services
org.apache.hadoop.yarn.server.nodemanager$
org.apache.avro.AvroRuntimeException:
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:132)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:231)
Caused by: java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:66)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:161)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:128)
... 3 more
Caused by: com.google.protobuf.ServiceException: java.io.IOException:
Failed on local exception: java.io.IOException: Couldn't setup connection
for nm/admin$
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:139)
at $Proxy14.registerNodeManager(Unknown Source)
at
org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:59)
... 5 more
Caused by: java.io.IOException: Failed on local exception:
java.io.IOException: Couldn't setup connection for nm/
ad...@master.example.com to rm/admin@MASTER$
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:655)
at org.apache.hadoop.ipc.Client.call(Client.java:1089)


Any help is appreciated

hadoop 0.23 secure mode error

2011-12-12 Thread sri ram 
Hi,
  I am trying to form a hadoop cluster of 0.23 version in secure
mode.
   While starting nodemanager i get the following error
2011-12-12 15:37:26,874 INFO  ipc.HadoopYarnRPC
(HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc
proxy for protocol interface org.apac$
2011-12-12 15:37:26,953 INFO  nodemanager.NodeStatusUpdaterImpl
(NodeStatusUpdaterImpl.java:registerWithRM(155)) - Connected to
ResourceManager at master:80$
2011-12-12 15:37:38,784 WARN  ipc.Client (Client.java:run(526)) - Couldn't
setup connection for nm/ad...@master.example.com to rm/
ad...@master.example.com
2011-12-12 15:37:38,787 ERROR service.CompositeService
(CompositeService.java:start(72)) - Error starting services
org.apache.hadoop.yarn.server.nodemanager$
org.apache.avro.AvroRuntimeException:
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:132)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:231)
Caused by: java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:66)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:161)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:128)
... 3 more
Caused by: com.google.protobuf.ServiceException: java.io.IOException:
Failed on local exception: java.io.IOException: Couldn't setup connection
for nm/admin$
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:139)
at $Proxy14.registerNodeManager(Unknown Source)
at
org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:59)
... 5 more
Caused by: java.io.IOException: Failed on local exception:
java.io.IOException: Couldn't setup connection for nm/
ad...@master.example.com to rm/admin@MASTER$
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:655)
at org.apache.hadoop.ipc.Client.call(Client.java:1089)


Any help is appreciated

Re: hadoop 0.23 secure mode error

2011-12-12 Thread sri ram 
Actually i am trying it locally just between 3 systems.I I have generated
the keytabs in kerberos and added these users in acl.Is there any other
configurations required???

On Mon, Dec 12, 2011 at 10:51 PM, Robert Evans ev...@yahoo-inc.com wrote:

  It looks like you do not have nm/ad...@master.example.com configured in
 your kerberos setup.  I wander how much traffic example.com gets on a
 daily basis.

 --Bobby Evans


 On 12/12/11 4:15 AM, sri ram rsriram...@gmail.com wrote:

 Hi,
   I am trying to form a hadoop cluster of 0.23 version in secure
 mode.
While starting nodemanager i get the following error
 2011-12-12 15:37:26,874 INFO  ipc.HadoopYarnRPC
 (HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc
 proxy for protocol interface org.apac$
 2011-12-12 15:37:26,953 INFO  nodemanager.NodeStatusUpdaterImpl
 (NodeStatusUpdaterImpl.java:registerWithRM(155)) - Connected to
 ResourceManager at master:80$
 2011-12-12 15:37:38,784 WARN  ipc.Client (Client.java:run(526)) - Couldn't
 setup connection for nm/ad...@master.example.com to
 rm/ad...@master.example.com
 2011-12-12 15:37:38,787 ERROR service.CompositeService
 (CompositeService.java:start(72)) - Error starting services
 org.apache.hadoop.yarn.server.nodemanager$
 org.apache.avro.AvroRuntimeException:
 java.lang.reflect.UndeclaredThrowableException
 at
 org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:132)
 at
 org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
 at
 org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:163)
 at
 org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:231)
 Caused by: java.lang.reflect.UndeclaredThrowableException
 at
 org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:66)
 at
 org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:161)
 at
 org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:128)
 ... 3 more
 Caused by: com.google.protobuf.ServiceException: java.io.IOException:
 Failed on local exception: java.io.IOException: Couldn't setup connection
 for nm/admin$
 at
 org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:139)
 at $Proxy14.registerNodeManager(Unknown Source)
 at
 org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:59)
 ... 5 more
 Caused by: java.io.IOException: Failed on local exception:
 java.io.IOException: Couldn't setup connection for
 nm/ad...@master.example.com to rm/admin@MASTER$
 at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:655)
 at org.apache.hadoop.ipc.Client.call(Client.java:1089)


 Any help is appreciated

Registration of Node Manger Failed

2011-11-25 Thread sri ram 
Hi,
   I try to install hadoop 0.23 and form a small cluster with 3
machines.
   Whenever i try to start nodemanager and resource manager.The
nodemanager fails to start by throwing the following error log.And the
nodemanager fails in both master and slaves.

2011-11-25 13:40:15,244 INFO  service.AbstractService
(AbstractService.java:start(61)) - Service:Dispatcher is started.
2011-11-25 13:40:15,244 INFO  ipc.YarnRPC (YarnRPC.java:create(47)) -
Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
2011-11-25 13:40:15,246 INFO  ipc.HadoopYarnRPC
(HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc
proxy for protocol interface
org.apache.hadoop.yarn.server.api.ResourceTracker
2011-11-25 13:40:15,289 INFO  nodemanager.NodeStatusUpdaterImpl
(NodeStatusUpdaterImpl.java:registerWithRM(155)) - Connected to
ResourceManager at master:8025
2011-11-25 13:40:15,407 ERROR service.CompositeService
(CompositeService.java:start(72)) - Error starting services
org.apache.hadoop.yarn.server.nodemanager.NodeManager
org.apache.avro.AvroRuntimeException: org.apache.hadoop.yarn.YarnException:
Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager
failed
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:132)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:231)
Caused by: org.apache.hadoop.yarn.YarnException: Recieved SHUTDOWN signal
from Resourcemanager ,Registration of NodeManager failed
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:165)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:128)
... 3 more
2011-11-25 13:40:15,408 INFO  event.AsyncDispatcher
(AsyncDispatcher.java:run(71)) - AsyncDispatcher thread interrupted
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2052)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:69)
at java.lang.Thread.run(Thread.java:636)
2011-11-25 13:40:15,410 INFO  service.AbstractService
(AbstractService.java:stop(75)) - Service:Dispatcher is stopped.
2011-11-25 13:40:15,470 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped SelectChannelConnector@0.0.0.0:
2011-11-25 13:40:15,588 INFO  service.AbstractService
(AbstractService.java:stop(75)) -
Service:org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer is
stopped.
2011-11-25 13:40:15,589 INFO  ipc.Server (Server.java:stop(1709)) -
Stopping server on 59072
2011-11-25 13:40:15,589 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 0 on 59072: exiting
2011-11-25 13:40:15,590 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 1 on 59072: exiting
2011-11-25 13:40:15,590 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 2 on 59072: exiting
2011-11-25 13:40:15,591 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 3 on 59072: exiting
2011-11-25 13:40:15,591 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 4 on 59072: exiting
2011-11-25 13:40:15,593 INFO  ipc.Server (Server.java:run(495)) - Stopping
IPC Server listener on 59072
2011-11-25 13:40:15,594 INFO  service.AbstractService
(AbstractService.java:stop(75)) -
Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler
is stopped.
2011-11-25 13:40:15,600 INFO  event.AsyncDispatcher
(AsyncDispatcher.java:run(71)) - AsyncDispatcher thread interrupted
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2052)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:69)
at java.lang.Thread.run(Thread.java:636)
2011-11-25 13:40:15,601 INFO  ipc.Server (Server.java:run(637)) - Stopping
IPC Server Responder
2011-11-25 13:40:15,601 INFO  service.AbstractService
(AbstractService.java:stop(75)) - Service:Dispatcher is stopped.
2011-11-25 13:40:15,602 WARN  monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(464)) -
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
is interrupted. Exiting.
2011-11-25 13:40:15,602 INFO

Registration of Node Manger Failed

2011-11-25 Thread sri ram 
Hi,
   I try to install hadoop 0.23 and form a small cluster with 3
machines.
   Whenever i try to start nodemanager and resource manager.The
nodemanager fails to start by throwing the following error log.And the
nodemanager fails in both master and slaves.

2011-11-25 13:40:15,244 INFO  service.AbstractService
(AbstractService.java:start(61)) - Service:Dispatcher is started.
2011-11-25 13:40:15,244 INFO  ipc.YarnRPC (YarnRPC.java:create(47)) -
Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
2011-11-25 13:40:15,246 INFO  ipc.HadoopYarnRPC
(HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc
proxy for protocol interface
org.apache.hadoop.yarn.server.api.ResourceTracker
2011-11-25 13:40:15,289 INFO  nodemanager.NodeStatusUpdaterImpl
(NodeStatusUpdaterImpl.java:registerWithRM(155)) - Connected to
ResourceManager at master:8025
2011-11-25 13:40:15,407 ERROR service.CompositeService
(CompositeService.java:start(72)) - Error starting services
org.apache.hadoop.yarn.server.nodemanager.NodeManager
org.apache.avro.AvroRuntimeException: org.apache.hadoop.yarn.YarnException:
Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager
failed
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:132)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:231)
Caused by: org.apache.hadoop.yarn.YarnException: Recieved SHUTDOWN signal
from Resourcemanager ,Registration of NodeManager failed
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:165)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:128)
... 3 more
2011-11-25 13:40:15,408 INFO  event.AsyncDispatcher
(AsyncDispatcher.java:run(71)) - AsyncDispatcher thread interrupted
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2052)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:69)
at java.lang.Thread.run(Thread.java:636)
2011-11-25 13:40:15,410 INFO  service.AbstractService
(AbstractService.java:stop(75)) - Service:Dispatcher is stopped.
2011-11-25 13:40:15,470 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped SelectChannelConnector@0.0.0.0:
2011-11-25 13:40:15,588 INFO  service.AbstractService
(AbstractService.java:stop(75)) -
Service:org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer is
stopped.
2011-11-25 13:40:15,589 INFO  ipc.Server (Server.java:stop(1709)) -
Stopping server on 59072
2011-11-25 13:40:15,589 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 0 on 59072: exiting
2011-11-25 13:40:15,590 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 1 on 59072: exiting
2011-11-25 13:40:15,590 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 2 on 59072: exiting
2011-11-25 13:40:15,591 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 3 on 59072: exiting
2011-11-25 13:40:15,591 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 4 on 59072: exiting
2011-11-25 13:40:15,593 INFO  ipc.Server (Server.java:run(495)) - Stopping
IPC Server listener on 59072
2011-11-25 13:40:15,594 INFO  service.AbstractService
(AbstractService.java:stop(75)) -
Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler
is stopped.
2011-11-25 13:40:15,600 INFO  event.AsyncDispatcher
(AsyncDispatcher.java:run(71)) - AsyncDispatcher thread interrupted
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2052)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:69)
at java.lang.Thread.run(Thread.java:636)
2011-11-25 13:40:15,601 INFO  ipc.Server (Server.java:run(637)) - Stopping
IPC Server Responder
2011-11-25 13:40:15,601 INFO  service.AbstractService
(AbstractService.java:stop(75)) - Service:Dispatcher is stopped.
2011-11-25 13:40:15,602 WARN  monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(464)) -
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
is interrupted. Exiting.
2011-11-25 13:40:15,602 INFO

Registration of Node Manger Failed

2011-11-25 Thread sri ram 
Hi,
   I try to install hadoop 0.23 and form a small cluster with 3
machines.
   Whenever i try to start nodemanager and resource manager.The
nodemanager fails to start by throwing the following error log.And the
nodemanager fails in both master and slaves.

2011-11-25 13:40:15,244 INFO  service.AbstractService
(AbstractService.java:start(61)) - Service:Dispatcher is started.
2011-11-25 13:40:15,244 INFO  ipc.YarnRPC (YarnRPC.java:create(47)) -
Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
2011-11-25 13:40:15,246 INFO  ipc.HadoopYarnRPC
(HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc
proxy for protocol interface
org.apache.hadoop.yarn.server.api.ResourceTracker
2011-11-25 13:40:15,289 INFO  nodemanager.NodeStatusUpdaterImpl
(NodeStatusUpdaterImpl.java:registerWithRM(155)) - Connected to
ResourceManager at master:8025
2011-11-25 13:40:15,407 ERROR service.CompositeService
(CompositeService.java:start(72)) - Error starting services
org.apache.hadoop.yarn.server.nodemanager.NodeManager
org.apache.avro.AvroRuntimeException: org.apache.hadoop.yarn.YarnException:
Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager
failed
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:132)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:163)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:231)
Caused by: org.apache.hadoop.yarn.YarnException: Recieved SHUTDOWN signal
from Resourcemanager ,Registration of NodeManager failed
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:165)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:128)
... 3 more
2011-11-25 13:40:15,408 INFO  event.AsyncDispatcher
(AsyncDispatcher.java:run(71)) - AsyncDispatcher thread interrupted
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2052)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:69)
at java.lang.Thread.run(Thread.java:636)
2011-11-25 13:40:15,410 INFO  service.AbstractService
(AbstractService.java:stop(75)) - Service:Dispatcher is stopped.
2011-11-25 13:40:15,470 INFO  mortbay.log (Slf4jLog.java:info(67)) -
Stopped SelectChannelConnector@0.0.0.0:
2011-11-25 13:40:15,588 INFO  service.AbstractService
(AbstractService.java:stop(75)) -
Service:org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer is
stopped.
2011-11-25 13:40:15,589 INFO  ipc.Server (Server.java:stop(1709)) -
Stopping server on 59072
2011-11-25 13:40:15,589 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 0 on 59072: exiting
2011-11-25 13:40:15,590 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 1 on 59072: exiting
2011-11-25 13:40:15,590 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 2 on 59072: exiting
2011-11-25 13:40:15,591 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 3 on 59072: exiting
2011-11-25 13:40:15,591 INFO  ipc.Server (Server.java:run(1533)) - IPC
Server handler 4 on 59072: exiting
2011-11-25 13:40:15,593 INFO  ipc.Server (Server.java:run(495)) - Stopping
IPC Server listener on 59072
2011-11-25 13:40:15,594 INFO  service.AbstractService
(AbstractService.java:stop(75)) -
Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler
is stopped.
2011-11-25 13:40:15,600 INFO  event.AsyncDispatcher
(AsyncDispatcher.java:run(71)) - AsyncDispatcher thread interrupted
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2052)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:69)
at java.lang.Thread.run(Thread.java:636)
2011-11-25 13:40:15,601 INFO  ipc.Server (Server.java:run(637)) - Stopping
IPC Server Responder
2011-11-25 13:40:15,601 INFO  service.AbstractService
(AbstractService.java:stop(75)) - Service:Dispatcher is stopped.
2011-11-25 13:40:15,602 WARN  monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:run(464)) -
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
is interrupted. Exiting.
2011-11-25 13:40:15,602 INFO