The following is the content of mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet href="configuration.xsl"?>
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/app/tmp/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/app/tmp/data</value>
</property>
<!--kerberos-->
<!--NAMENODE CONF-->
<property>
<name>dfs.block.access.token.enable</name>
<value>true</value>
</property>
<property>
<name>dfs.https.enable</name>
<value>true</value>
</property>
<property>
<name>dfs.namenode.https-address</name>
<value>master.example.com:50470</value>
</property>
<property>
<name>dfs.https.port</name>
<value>50470</value>
</property>
<property>
<name>dfs.namenode.keytab.file</name>
<value>/etc/security/keytab/nn.service.keytab</value>
</property>
<property>
<name>dfs.namenode.kerberos.principal</name>
<value>nn/master.example....@example.com</value>
</property>
<property>
<name>dfs.namenode.kerberos.https.principal</name>
<value>host/master.example....@example.com</value>
</property>
<!--DATANODE CONF-->
<property>
<name>dfs.datanode.data.dir.perm</name>
<value>700</value>
</property>
<property>
<name>dfs.datanode.address</name>
<value>master:1003</value>
</property>
<property>
<name>dfs.datanode.https.address</name>
<value>master:1005</value>
</property>
<property>
<name>dfs.datanode.http.address</name>
<value>master:1006</value>
</property>
<property>
<name>dfs.datanode.keytab.file</name>
<value>/etc/security/keytab/dn.service.keytab</value>
</property>
<property>
<name>dfs.datanode.kerberos.principal</name>
<value>dn/master.example....@example.com</value>
</property>
<property>
<name>dfs.datanode.kerberos.https.principal</name>
<value>host/master.example....@example.com</value>
</property>
</configuration>
The following is the error log while starting datanode as root
2011-12-13 16:19:26,837 INFO datanode.DataNode
(StringUtils.java:startupShutdownMessage(589)) - STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG: host = master.example.com/147.128.152.179
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.23.0
STARTUP_MSG: classpath =
/usr/local/hadoop/conf:/usr/local/hadoop/libexec/../share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop/libexec/../share$
STARTUP_MSG: build = git://
devadm900.cc1.ygridcore.net/grid/0/dev/acm/hadoop-trunk/hadoop-common-project/hadoop-common-r
d4fee83ec1462ab9824add6449320617$
************************************************************/
2011-12-13 16:19:26,921 WARN common.Util (Util.java:stringAsURI(63)) -
Path /app/tmp/data should be specified as a URI in configuration files.
Please updat$
2011-12-13 16:19:27,314 INFO security.UserGroupInformation
(UserGroupInformation.java:loginUserFromKeytab(633)) - Login successful for
user dn/master.examp$
2011-12-13 16:19:27,454 WARN impl.MetricsConfig
(MetricsConfig.java:loadFirst(125)) - Cannot locate configuration: tried
hadoop-metrics2-datanode.propertie$
2011-12-13 16:19:27,548 INFO impl.MetricsSystemImpl
(MetricsSystemImpl.java:startTimer(343)) - Scheduled snapshot period at 10
second(s).
2011-12-13 16:19:27,548 INFO impl.MetricsSystemImpl
(MetricsSystemImpl.java:start(182)) - DataNode metrics system started
2011-12-13 16:19:27,549 INFO impl.MetricsSystemImpl
(MetricsSystemImpl.java:registerSource(244)) - Registered source UgiMetrics
2011-12-13 16:19:27,572 INFO datanode.DataNode
(DataNode.java:initDataXceiver(701)) - Opened info server at 1003
2011-12-13 16:19:27,585 INFO datanode.DataNode
(DataXceiverServer.java:<init>(77)) - Balancing bandwith is 1048576 bytes/s
2011-12-13 16:19:27,617 INFO mortbay.log (Slf4jLog.java:info(67)) -
Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4j$
2011-12-13 16:19:27,703 INFO http.HttpServer
(HttpServer.java:addGlobalFilter(476)) - Added global filter 'safety'
(class=org.apache.hadoop.http.HttpServer$
2011-12-13 16:19:27,706 WARN lib.StaticUserWebFilter
(StaticUserWebFilter.java:getUsernameFromConf(141)) - dfs.web.ugi should
not be used. Instead, use had$
2011-12-13 16:19:27,707 INFO http.HttpServer
(HttpServer.java:addFilter(454)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
2011-12-13 16:19:27,707 INFO http.HttpServer
(HttpServer.java:addFilter(461)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
2011-12-13 16:19:27,708 INFO http.HttpServer
(HttpServer.java:addFilter(461)) - Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUse$
jetty.ssl.password : jetty.ssl.keypassword : 2011-12-13 16:19:27,732 INFO
mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26
2011-12-13 16:19:28,096 INFO mortbay.log (Slf4jLog.java:info(67)) -
Started SelectChannelConnector@master:1006
2011-12-13 16:19:28,100 WARN mortbay.log (Slf4jLog.java:warn(76)) -
java.lang.NullPointerException
2011-12-13 16:19:28,101 WARN mortbay.log (Slf4jLog.java:warn(76)) - failed
Krb5AndCertsSslSocketConnector@master:1005: java.io.IOException:
!JsseListener: $
2011-12-13 16:19:28,101 WARN mortbay.log (Slf4jLog.java:warn(76)) - failed
Server@179d854: java.io.IOException: !JsseListener:
java.lang.NullPointerExcepti$
2011-12-13 16:19:28,104 INFO mortbay.log (Slf4jLog.java:info(67)) -
Stopped Krb5AndCertsSslSocketConnector@master:1005
2011-12-13 16:19:28,104 INFO mortbay.log (Slf4jLog.java:info(67)) -
Stopped SelectChannelConnector@master:1006
2011-12-13 16:19:28,107 INFO datanode.DataNode
(DataNode.java:shutdown(1741)) - Waiting for threadgroup to exit, active
threads is 0
2011-12-13 16:19:28,108 ERROR datanode.DataNode
(DataNode.java:secureMain(2371)) - Exception in secureMain
java.io.IOException: !JsseListener: java.lang.NullPointerException
at
org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)
at
org.apache.hadoop.security.Krb5AndCertsSslSocketConnector.newServerSocket(Krb5AndCertsSslSocketConnector.java:123)
at
org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at
org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
at
org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.Server.doStart(Server.java:235)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:639)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startInfoServer(DataNode.java:575)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1501)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:457)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2263)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2196)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2219)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2367)
at
org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter.start(SecureDataNodeStarter.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:188)
2011-12-13 16:19:28,111 INFO datanode.DataNode (StringUtils.java:run(605))
- SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at master.example.com/147.128.152.179
On Tue, Dec 13, 2011 at 4:23 PM, sri ram <rsriram...@gmail.com> wrote:
> telnet master ip says "unable to connect to remote host.
> This is my following property for datanode in secure mode
> <property>
> <name>dfs.datanode.https.address</name>
> <value>master:1005</value>
> </property>
>
>
>
> On Tue, Dec 13, 2011 at 4:18 PM, alo alt <wget.n...@googlemail.com> wrote:
>
>> Check if kerberos respond:
>>
>> telnet KERBEROS_IP 1005
>>
>> As I know use kerberos per default ports 88, AFS token 746 (4444), kx509
>> 9878
>>
>> - Alex
>>
>>
>> On Tue, Dec 13, 2011 at 11:39 AM, sri ram <rsriram...@gmail.com> wrote:
>> > Thanks for the reply,
>> > I have tried with the ip of the individual systems
>> also.But
>> > the same eroor reoccurs
>> >
>> >
>> > On Tue, Dec 13, 2011 at 3:40 PM, alo alt <wget.n...@googlemail.com>
>> wrote:
>> >>
>> >> Hi,
>> >>
>> >> 2011-12-14 14:35:54,047 WARN mortbay.log (Slf4jLog.java:warn(76)) -
>> >> failed Krb5AndCertsSslSocketConnector@0.0.0.0:1005:
>> >> java.io.IOException: !JsseListener:$
>> >> 2011-12-14 14:35:54,048 WARN mortbay.log (Slf4jLog.java:warn(76)) -
>> >> failed Server@1867df9: java.io.IOException: !JsseListener:
>> >> java.lang.NullPointerExcepti$
>> >> 2011-12-14 14:35:54,085 INFO mortbay.log (Slf4jLog.java:info(67)) -
>> >> Stopped Krb5AndCertsSslSocketConnector@0.0.0.0:1005
>> >>
>> >> 0.0.0.0 as Kerberos - IP can't work.
>> >>
>> >> - Alex
>> >>
>> >> On Tue, Dec 13, 2011 at 10:27 AM, sri ram <rsriram...@gmail.com>
>> wrote:
>> >> > Hi,
>> >> > I receive the following error while starting datanode in
>> secure
>> >> > mode of hadoop 0.23
>> >> >
>> >> > 2011-12-14 14:35:48,468 INFO http.HttpServer
>> >> > (HttpServer.java:addGlobalFilter(476)) - Added global filter 'safety'
>> >> > (class=org.apache.hadoop.http.HttpServer$
>> >> > 2011-12-14 14:35:48,471 WARN lib.StaticUserWebFilter
>> >> > (StaticUserWebFilter.java:getUsernameFromConf(141)) - dfs.web.ugi
>> should
>> >> > not
>> >> > be used. Instead, use had$
>> >> > 2011-12-14 14:35:48,472 INFO http.HttpServer
>> >> > (HttpServer.java:addFilter(454)) - Added filter static_user_filter
>> >> > (class=org.apache.hadoop.http.lib.StaticUse$
>> >> > 2011-12-14 14:35:48,472 INFO http.HttpServer
>> >> > (HttpServer.java:addFilter(461)) - Added filter static_user_filter
>> >> > (class=org.apache.hadoop.http.lib.StaticUse$
>> >> > 2011-12-14 14:35:48,473 INFO http.HttpServer
>> >> > (HttpServer.java:addFilter(461)) - Added filter static_user_filter
>> >> > (class=org.apache.hadoop.http.lib.StaticUse$
>> >> > jetty.ssl.password : jetty.ssl.keypassword : 2011-12-14 14:35:53,553
>> >> > INFO
>> >> > mortbay.log (Slf4jLog.java:info(67)) - jetty-6.1.26
>> >> > 2011-12-14 14:35:54,044 INFO mortbay.log (Slf4jLog.java:info(67)) -
>> >> > Started
>> >> > SelectChannelConnector@master:1006
>> >> > 2011-12-14 14:35:54,047 WARN mortbay.log (Slf4jLog.java:warn(76)) -
>> >> > java.lang.NullPointerException
>> >> > 2011-12-14 14:35:54,047 WARN mortbay.log (Slf4jLog.java:warn(76)) -
>> >> > failed
>> >> > Krb5AndCertsSslSocketConnector@0.0.0.0:1005: java.io.IOException:
>> >> > !JsseListener:$
>> >> > 2011-12-14 14:35:54,048 WARN mortbay.log (Slf4jLog.java:warn(76)) -
>> >> > failed
>> >> > Server@1867df9: java.io.IOException: !JsseListener:
>> >> > java.lang.NullPointerExcepti$
>> >> > 2011-12-14 14:35:54,085 INFO mortbay.log (Slf4jLog.java:info(67)) -
>> >> > Stopped
>> >> > Krb5AndCertsSslSocketConnector@0.0.0.0:1005
>> >> > 2011-12-14 14:35:54,085 INFO mortbay.log (Slf4jLog.java:info(67)) -
>> >> > Stopped
>> >> > SelectChannelConnector@master:1006
>> >> > 2011-12-14 14:35:54,189 INFO datanode.DataNode
>> >> > (DataNode.java:shutdown(1741)) - Waiting for threadgroup to exit,
>> active
>> >> > threads is 0
>> >> > 2011-12-14 14:35:54,190 ERROR datanode.DataNode
>> >> > (DataNode.java:secureMain(2371)) - Exception in secureMain
>> >> > java.io.IOException: !JsseListener: java.lang.NullPointerException
>> >> > at
>> >> >
>> >> >
>> org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:516)
>> >> > at
>> >> >
>> >> >
>> org.apache.hadoop.security.Krb5AndCertsSslSocketConnector.newServerSocket(Krb5AndCertsSslSocketConnector.java:123)
>> >> > at
>> >> > org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
>> >> > at
>> >> >
>> org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
>> >> > at
>> >> >
>> org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
>> >> >
>> >> > Is there any way to resolve this???
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Alexander Lorenz
>> >> http://mapredit.blogspot.com
>> >>
>> >> P Think of the environment: please don't print this email unless you
>> >> really need to.
>> >
>> >
>>
>>
>>
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>>
>> P Think of the environment: please don't print this email unless you
>> really need to.
>>
>
>
