Re: Encrypt a directory using some key (JAVA)

[Aneela Saleem]

Thanks Wellington,

I already looked into that. But those KMS HTTP REST API are only for key
management like create key, rollover key, delete key etc. I didn't see any
API for encrypting a zone. If there exist any, then do tell me please!

Thanks Wei-Chiu,

I looked into that. I'm able to create and encrypt the zone by using the
following code from CryptoAdmin class:

Path deepZone = new Path("/d/e/e/p/zone");
fsWrapper.mkdir(deepZone, FsPermission.getDirDefault(), true);
dfsAdmin.createEncryptionZone(deepZone, TEST_KEY, NO_TRASH);


On Thu, Dec 15, 2016 at 1:55 AM, Wei-Chiu Chuang 
wrote:

> Hi
> If you have access to Hadoop codebase, take a look at CryptoAdmin class,
> which implements these two commands.
> Internally, the commands are implemented via 
> DistributedFileSystem#createEncryptionZone
> and DistributedFileSystem#listEncryptionZones
>
> Regards,
> Wei-Chiu Chuang
> A very happy Clouderan
>
> On Dec 14, 2016, at 5:39 AM, Aneela Saleem  wrote:
>
> Hi,
>
> I have successfully enables Hadoop with KMS and now I want to write some
> java code to create key, get keys and encrypt a directory using a key. In
> other words, I want to translate this command
>
> hdfs hdfs crypto -createZone -keyName  -path /encryption_zone
>
> and
>
> hdfs hdfs crypto -listZones
>
>
> into java code.
>
>
> Any suggestions will be appreciated.
>
> Thanks
>
>
>

Encrypt a directory using some key (JAVA)

[Aneela Saleem]

Hi,

I have successfully enables Hadoop with KMS and now I want to write some
java code to create key, get keys and encrypt a directory using a key. In
other words, I want to translate this command

hdfs hdfs crypto -createZone -keyName  -path /encryption_zone

and

hdfs hdfs crypto -listZones


into java code.


Any suggestions will be appreciated.

Thanks

issue starting regionserver with SASL authentication failed

[Aneela Saleem]

Hi all,

I'm facing issue starting region server in HBase. I have enabled Kerberos
debugging in Hadoop command line, so when i run the "hadoop fs -ls /"
command, i get following output, I can't interpret this. Can anyone please
tell me is something wrong with Kerberos configuration or everything is
fine ?


16/08/02 18:34:10 DEBUG util.Shell: setsid exited with exit code 0
16/08/02 18:34:10 DEBUG conf.Configuration: parsing URL
jar:file:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.7.2.jar!/core-default.xml
16/08/02 18:34:10 DEBUG conf.Configuration: parsing input stream
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@4fbc7b65
16/08/02 18:34:10 DEBUG conf.Configuration: parsing URL
file:/usr/local/hadoop/etc/hadoop/core-site.xml
16/08/02 18:34:10 DEBUG conf.Configuration: parsing input stream
java.io.BufferedInputStream@69c1adfa
16/08/02 18:34:11 DEBUG lib.MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation
@org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate
of successful kerberos logins and latency (milliseconds)], about=,
always=false, type=DEFAULT, sampleName=Ops)
16/08/02 18:34:11 DEBUG lib.MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation
@org.apache.hadoop.metrics2.annotation.Metric(valueName=Time, value=[Rate
of failed kerberos logins and latency (milliseconds)], about=,
always=false, type=DEFAULT, sampleName=Ops)
16/08/02 18:34:11 DEBUG lib.MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with
annotation @org.apache.hadoop.metrics2.annotation.Metric(valueName=Time,
value=[GetGroups], about=, always=false, type=DEFAULT, sampleName=Ops)
16/08/02 18:34:11 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group
related metrics
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
16/08/02 18:34:11 DEBUG security.Groups:  Creating new Groups object
16/08/02 18:34:11 DEBUG security.Groups: Group mapping
impl=org.apache.hadoop.security.LdapGroupsMapping; cacheTimeout=30;
warningDeltaMs=5000
>>>KinitOptions cache name is /tmp/krb5cc_0
>>>DEBUG   client principal is
nn/hadoop-master@platalyticsrealm
>>>DEBUG  server principal is
krbtgt/platalyticsrealm@platalyticsrealm
>>>DEBUG  key type: 16
>>>DEBUG  auth time: Tue Aug 02 18:23:59 PKT 2016
>>>DEBUG  start time: Tue Aug 02 18:23:59 PKT 2016
>>>DEBUG  end time: Wed Aug 03 06:23:59 PKT 2016
>>>DEBUG  renew_till time: Tue Aug 09 18:23:59 PKT 2016
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL;
>>>DEBUG   client principal is
nn/hadoop-master@platalyticsrealm
>>>DEBUG  server principal is
X-CACHECONF:/krb5_ccache_conf_data/fast_avail/krbtgt/platalyticsrealm@platalyticsrealm
>>>DEBUG  key type: 0
>>>DEBUG  auth time: Thu Jan 01 05:00:00 PKT 1970
>>>DEBUG  start time: null
>>>DEBUG  end time: Thu Jan 01 05:00:00 PKT 1970
>>>DEBUG  renew_till time: null
>>> CCacheInputStream: readFlags()
16/08/02 18:34:11 DEBUG security.UserGroupInformation: hadoop login
16/08/02 18:34:11 DEBUG security.UserGroupInformation: hadoop login commit
16/08/02 18:34:11 DEBUG security.UserGroupInformation: using kerberos
user:nn/hadoop-master@platalyticsrealm
16/08/02 18:34:11 DEBUG security.UserGroupInformation: Using user:
"nn/hadoop-master@platalyticsrealm" with name
nn/hadoop-master@platalyticsrealm
16/08/02 18:34:11 DEBUG security.UserGroupInformation: User entry:
"nn/hadoop-master@platalyticsrealm"
16/08/02 18:34:11 DEBUG security.UserGroupInformation: UGI
loginUser:nn/hadoop-master@platalyticsrealm (auth:KERBEROS)
16/08/02 18:34:12 DEBUG security.UserGroupInformation: Found tgt Ticket
(hex) =
: 61 82 01 72 30 82 01 6E   A0 03 02 01 05 A1 12 1B  a..r0..n
0010: 10 70 6C 61 74 61 6C 79   74 69 63 73 72 65 61 6C  .platalyticsreal
0020: 6D A2 25 30 23 A0 03 02   01 02 A1 1C 30 1A 1B 06  m.%0#...0...
0030: 6B 72 62 74 67 74 1B 10   70 6C 61 74 61 6C 79 74  krbtgt..platalyt
0040: 69 63 73 72 65 61 6C 6D   A3 82 01 2A 30 82 01 26  icsrealm...*0..&
0050: A0 03 02 01 10 A1 03 02   01 01 A2 82 01 18 04 82  
0060: 01 14 A5 A9 41 A6 B7 0E   8F 70 F4 03 41 64 8D DC  Ap..Ad..
0070: 78 2F FB 08 58 C9 39 44   CF D0 8D B0 85 09 62 8C  x/..X.9D..b.
0080: 40 CF 45 13 D3 B9 CD 38   84 92 33 24 B2 0D C1 65  @.E8..3$...e
0090: C7 1B 0D 3E F2 92 A2 8B   58 34 77 5F F6 E3 AA B6  ...>X4w_
00A0: EB 8E 58 46 AC 54 DB 9B   79 3E ED A1 83 0C D3 D3  ..XF.T..y>..
00B0: 02 8B 42 52 6D 92 F1 39   BA E7 56 D4 BA A6 03 B6  ..BRm..9..V.
00C0: 16 5A DC 1A 69 F4 DF A5   CD F6 48 AC 08 32 D3 AD  .Z..i.H..2..
00D0: 22 8E E9 52 00 93 78 41   1C 26 4F 0B 42 2C EF E9  "..R..xA.&O.B,..
00E0: B8 0E 84 39 E4 AF 3A 60   7D 04 EE 70 18 C0 E7 21  ...9..:`...p...!
0

Re: Hadoop-Kerberos aunthentication flow

[Aneela Saleem]

Sorry could not load the image. Please see the attached screen shot

On Tue, Jul 12, 2016 at 5:09 PM, Aneela Saleem 
wrote:

> Hi all,
>
> I have configured Kerberos with Hadoop. I'm facing difficulty in mapping
> the Kerberos architecture and whole flow of authentication to my
> application. Following is my usecase:
>
> We have a web application that calls backend services, which communicates
> with Hadoop ecosystem internally. Now i don't have clear idea how the
> kerberos aunthentication will take place, where the tokens will be stored
> i.e., whether client-side or server side. How the credential cache would be
> managed,when two or more users access the application and access hadoop,
> because when we do kinit the old credential cache is replaced by the new
> one. What would be the complete flow? just like this
>
>
>
>
> Thanks
>

-
To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org
For additional commands, e-mail: user-h...@hadoop.apache.org

Hadoop-Kerberos aunthentication flow

[Aneela Saleem]

Hi all,

I have configured Kerberos with Hadoop. I'm facing difficulty in mapping
the Kerberos architecture and whole flow of authentication to my
application. Following is my usecase:

We have a web application that calls backend services, which communicates
with Hadoop ecosystem internally. Now i don't have clear idea how the
kerberos aunthentication will take place, where the tokens will be stored
i.e., whether client-side or server side. How the credential cache would be
managed,when two or more users access the application and access hadoop,
because when we do kinit the old credential cache is replaced by the new
one. What would be the complete flow? just like this




Thanks

Re: datanode is unable to connect to namenode

[Aneela Saleem]

Thanks Vinaykumar and Gurmukh,

I have made it working successfully through auth_to_local configs. But i
faced much issues.

Actually I have two nodes cluster, one being namenode and datanode, and
second being datanode only. I faced some authentication from keytab related
issues. for example:

I added nn/hadoop-master to both nn.keytab and dn.keytab and did the same
with dn/hadoop-slave (following you github dn.keytab file). but when i
start cluster i got following error:

*Login failure for nn/hadoop-master@platalyticsrealm from keytab
/etc/hadoop/conf/hdfs.keytab: javax.security.auth.login.LoginException:
Checksum failed*

i verified the authentication of nn/hadoop-master with keytab
nn/hadoop-master through kinit, but couldn't do so. because of error
like *could
not verify credentials*

i then removed nn/hadoop-master from dn.keytab then it authenticated
successfully. And i removed all the hadoop-master principals from dn.keytab
and hadoop-slave principals from nn.keytab.  So does it mean that a
principal can't belong to more than one keytabs? And please make some time
to review the attached hdfs-site.xml for both namenode and datanode, and
keytab files. And point me out if something wrong.

Thanks


On Thu, Jun 30, 2016 at 1:21 PM, Vinayakumar B 
wrote:

> Please note, there are two different configs.
>
>
>
> “dfs.datanode.kerberos.principal” and “dfs.namenode.kerberos.principal”
>
>
>
> Following configs can be set, as required.
>
>
>
> dfs.datanode.kerberos.principal à dn/_HOST
>
> dfs.namenode.kerberos.principal  à nn/_HOST
>
>
>
> “nn/_HOST” will be used only in namenode side.
>
>
>
> -Vinay
>
> *From:* Aneela Saleem [mailto:ane...@platalytics.com]
> *Sent:* 30 June 2016 13:24
> *To:* Vinayakumar B 
> *Cc:* user@hadoop.apache.org
>
> *Subject:* Re: datanode is unable to connect to namenode
>
>
>
> Thanks Vinayakumar
>
>
>
> Yes you got it right i was using different principal names i.e.,
> *nn/_HOST* for namenode and *dn/_HOST* for datanode. Setting the same
> principal name for both datanode and namenode i.e.,
> hdfs/_HOST@platalyticsrealm solved the issue. Now datanode
>
> can connect to namenode successfully.
>
>
>
> So my question is, is it mandatory to have same principal name on all
> hosts i.e., hdfs/_HOST@platalyticsrealm, because i found in many
>
> tutorials that the convention is to have different principals for all
> services like
>
> dn/_HOST for datanode
>
> nn/_HOST for namenode
>
> sn/_HOST for secondarynamenode etc
>
>
>
> Secondly for map reduce and yarn, would that mapred-site.xml and
> yarn-site.xml be same on all cluster nodes? just like for hdfs-site.xml
>
>
>
> Thanks
>
>
>
> On Thu, Jun 30, 2016 at 10:51 AM, Vinayakumar B 
> wrote:
>
> Hi Aneela,
>
>
>
> 1. Looks like you have attached the hdfs-site.xml from 'hadoop-master'
> node. For this node datanode connection is successfull as mentioned in
> below logs.
>
>
>
>  2016-06-29 10:01:35,700 INFO
> SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for
> nn/hadoop-master@platalyticsrealm (auth:KERBEROS)
>
> 2016-06-29 10:01:35,744 INFO
> SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
> Authorization successful for nn/hadoop-master@platalyticsrealm
> (auth:KERBEROS) for protocol=interface
> org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol
>
>
>
>  2016-06-29 10:01:36,845 INFO
> org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/
> 192.168.23.206:1004
>
>
>
>
>
> 2. For the other node, 'hadoop-slave' kerberos athentication is
> successfull, but ServiceAuthorizationManager check failed.
>
>
>
> 2016-06-29 10:01:37,474 INFO SecurityLogger.org.apache.hadoop.ipc.Server:
> Auth successful for dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
>
> 2016-06-29 10:01:37,512 WARN
> SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
> Authorization failed for dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
> for protocol=interface
> org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol, expected client
> Kerberos principal is nn/hadoop-slave@platalyticsrealm
>
> 2016-06-29 10:01:37,514 INFO org.apache.hadoop.ipc.Server: Connection from
> 192.168.23.207:32807 for protocol
> org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol is unauthorized for
> user dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
>
>
>
> reason could be mostly, "dfs.datanode.kerberos.principal" configuration in
> both nodes differ. I can see that this configuration in hadoop-master's
> hdfs-site.xml set to 'nn/_HOST@platalyt

Re: datanode is unable to connect to namenode

[Aneela Saleem]

Thanks Vinayakumar

Yes you got it right i was using different principal names i.e., *nn/_HOST*
for namenode and *dn/_HOST* for datanode. Setting the same principal name
for both datanode and namenode i.e., hdfs/_HOST@platalyticsrealm solved the
issue. Now datanode
can connect to namenode successfully.

So my question is, is it mandatory to have same principal name on all hosts
i.e., hdfs/_HOST@platalyticsrealm, because i found in many
tutorials that the convention is to have different principals for all
services like
dn/_HOST for datanode
nn/_HOST for namenode
sn/_HOST for secondarynamenode etc

Secondly for map reduce and yarn, would that mapred-site.xml and
yarn-site.xml be same on all cluster nodes? just like for hdfs-site.xml

Thanks

On Thu, Jun 30, 2016 at 10:51 AM, Vinayakumar B 
wrote:

> Hi Aneela,
>
>
>
> 1. Looks like you have attached the hdfs-site.xml from 'hadoop-master'
> node. For this node datanode connection is successfull as mentioned in
> below logs.
>
>
>
>  2016-06-29 10:01:35,700 INFO
> SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for
> nn/hadoop-master@platalyticsrealm (auth:KERBEROS)
>
> 2016-06-29 10:01:35,744 INFO
> SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
> Authorization successful for nn/hadoop-master@platalyticsrealm
> (auth:KERBEROS) for protocol=interface
> org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol
>
>
>
>  2016-06-29 10:01:36,845 INFO
> org.apache.hadoop.net.NetworkTopology: Adding a new node: /default-rack/
> 192.168.23.206:1004
>
>
>
>
>
> 2. For the other node, 'hadoop-slave' kerberos athentication is
> successfull, but ServiceAuthorizationManager check failed.
>
>
>
> 2016-06-29 10:01:37,474 INFO SecurityLogger.org.apache.hadoop.ipc.Server:
> Auth successful for dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
>
> 2016-06-29 10:01:37,512 WARN
> SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
> Authorization failed for dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
> for protocol=interface
> org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol, expected client
> Kerberos principal is nn/hadoop-slave@platalyticsrealm
>
> 2016-06-29 10:01:37,514 INFO org.apache.hadoop.ipc.Server: Connection from
> 192.168.23.207:32807 for protocol
> org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol is unauthorized for
> user dn/hadoop-slave@platalyticsrealm (auth:KERBEROS)
>
>
>
> reason could be mostly, "dfs.datanode.kerberos.principal" configuration in
> both nodes differ. I can see that this configuration in hadoop-master's
> hdfs-site.xml set to 'nn/_HOST@platalyticsrealm' but it might have been
> set to 'dn/_HOST@platalyticsrealm' in hadoop-slave node's configurations.
>
>
>
> Please change this configuration in all nodes to 'dn/_HOST@platalyticsrealm'
> and restart all NNs and DNs, and check again.
>
>
>
> If this does not help, then please share the hdfs-site.xml of hadoop-slave
> node too.
>
>
>
> -Vinay
>
>
>
> *From:* Aneela Saleem [mailto:ane...@platalytics.com]
> *Sent:* 29 June 2016 21:35
> *To:* user@hadoop.apache.org
> *Subject:* Fwd: datanode is unable to connect to namenode
>
>
>
>
>
> Sent from my iPhone
>
>
> Begin forwarded message:
>
> *From:* Aneela Saleem 
> *Date:* 29 June 2016 at 10:16:36 GMT+5
> *To:* "sreebalineni ." 
> *Subject:* *Re: datanode is unable to connect to namenode*
>
> Attached are the log files for datanode and namenode. Also i have attached
> hdfs-site.xml for namenode please check if there are any issues in
> configuration file.
>
>
>
> I have following two Kerberos Principals:
>
>
>
> nn/hadoop-master
>
> dn/hadoop-slave
>
>
>
> i have copied kdc.conf and krb5.conf on both nodes. Also i copied keytab
> file on datanode. And i have starting services with principal
> nn/hadoop-master.
>
>
>
> On Wed, Jun 29, 2016 at 9:35 AM, sreebalineni . 
> wrote:
>
> Probably sharing both Name node and datanode logs may help.
>
>
>
> On Wed, Jun 29, 2016 at 10:02 AM, Aneela Saleem 
> wrote:
>
> Following is the result of telnet
>
>
>
> Trying 192.168.23.206...
>
> Connected to hadoop-master.
>
> Escape character is '^]'.
>
>
>
> On Wed, Jun 29, 2016 at 3:57 AM, Aneela Saleem 
> wrote:
>
> Thanks Sreebalineni for the response.
>
> This is the result of the *netstat -a | grep 8020* command
>
> tcp0  0 hadoop-master:8020  *:* LISTEN
>
> tcp0  0 hadoop-master:3335

Fwd: datanode is unable to connect to namenode

[Aneela Saleem]

Sent from my iPhoneBegin forwarded message:From: Aneela Saleem <ane...@platalytics.com>Date: 29 June 2016 at 10:16:36 GMT+5To: "sreebalineni ." <sreebalin...@gmail.com>Subject: Re: datanode is unable to connect to namenodeAttached are the log files for datanode and namenode. Also i have attached hdfs-site.xml for namenode please check if there are any issues in configuration file.I have following two Kerberos Principals:nn/hadoop-masterdn/hadoop-slavei have copied kdc.conf and krb5.conf on both nodes. Also i copied keytab file on datanode. And i have starting services with principal nn/hadoop-master.On Wed, Jun 29, 2016 at 9:35 AM, sreebalineni . <sreebalin...@gmail.com> wrote:Probably sharing both Name node and datanode logs may help. On Wed, Jun 29, 2016 at 10:02 AM, Aneela Saleem <ane...@platalytics.com> wrote:Following is the result of telnet Trying 192.168.23.206...Connected to hadoop-master.Escape character is '^]'.On Wed, Jun 29, 2016 at 3:57 AM, Aneela Saleem <ane...@platalytics.com> wrote:Thanks Sreebalineni for the response.This is the result of the netstat -a | grep 8020 commandtcp        0      0 hadoop-master:8020      *:*                     LISTENtcp        0      0 hadoop-master:33356     hadoop-master:8020      ESTABLISHEDtcp        0      0 hadoop-master:8020      hadoop-master:33356     ESTABLISHEDtcp        0      0 hadoop-master:55135     hadoop-master:8020      TIME_WAITAnd this is my /etc/hosts file#127.0.0.1      localhost#127.0.1.1      vm6-VirtualBox192.168.23.206  hadoop-master platalytics.com vm6-VirtualBox192.168.23.207  hadoop-slave# The following lines are desirable for IPv6 capable hosts::1     ip6-localhost ip6-loopbackfe00::0 ip6-localnetff00::0 ip6-mcastprefixff02::1 ip6-allnodesff02::2 ip6-allroutersCan you please tell me what's wrong with above configuration and how can i check whether it is firewall issue?ThanksOn Wed, Jun 29, 2016 at 12:11 AM, sreebalineni . <sreebalin...@gmail.com> wrote:Are you able to telnet ping. Check the firewalls as well
On Jun 29, 2016 12:39 AM, "Aneela Saleem" <ane...@platalytics.com> wrote:Hi all,I have setup two nodes cluster with security enabled. I have everything running successful like namenode, datanode, resourcemanager, nodemanager, jobhistoryserver etc. But datanode is unable to connect to namenode, as i can see only one node on the web UI. checking logs of datanode gives following warning:WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: hadoop-master/192.168.23.206:8020Rest of the things look fine. Please help me in this regard, what could be the issue?





2016-06-29 09:37:09,628 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
registered UNIX signal handlers for [TERM, HUP, INT]
2016-06-29 09:37:10,096 INFO org.apache.hadoop.security.UserGroupInformation: 
Login successful for user dn/hadoop-slave@platalyticsrealm using keytab file 
/etc/hadoop/conf/dn.keytab
2016-06-29 09:37:10,307 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: 
loaded properties from hadoop-metrics2.properties
2016-06-29 09:37:10,442 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: 
Scheduled snapshot period at 10 second(s).
2016-06-29 09:37:10,442 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: 
DataNode metrics system started
2016-06-29 09:37:10,453 INFO 
org.apache.hadoop.hdfs.server.datanode.BlockScanner: Initialized block scanner 
with targetBytesPerSec 1048576
2016-06-29 09:37:10,456 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
Configured hostname is vm7-web
2016-06-29 09:37:10,468 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
Starting DataNode with maxLockedMemory = 0
2016-06-29 09:37:10,503 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
Opened streaming server at /192.168.23.207:1004
2016-06-29 09:37:10,508 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
Balancing bandwith is 1048576 bytes/s
2016-06-29 09:37:10,508 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: 
Number threads for balancing is 5
2016-06-29 09:37:10,659 INFO org.mortbay.log: Logging to 
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2016-06-29 09:37:10,678 INFO 
org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable 
to initialize FileSignerSecretProvider, falling back to use random secrets.
2016-06-29 09:37:10,689 INFO org.apache.hadoop.http.HttpRequestLog: Http 
request log for http.requests.datanode is not defined
2016-06-29 09:37:10,698 INFO org.apache.hadoop.http.HttpServer2: Added global 
filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2016-06-29 09:37:10,704 INFO org.apache.hadoop.http.HttpServer2: Added filter 
static_user_filter 
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to 
context datanode
2016-06-29 09:37:10,704 INFO org.apache.hadoop.http.HttpServer2: Added filter 
static_

Re: datanode is unable to connect to namenode

[Aneela Saleem]

Following is the result of telnet

Trying 192.168.23.206...
Connected to hadoop-master.
Escape character is '^]'.

On Wed, Jun 29, 2016 at 3:57 AM, Aneela Saleem 
wrote:

> Thanks Sreebalineni for the response.
>
> This is the result of the *netstat -a | grep 8020* command
>
> tcp0  0 hadoop-master:8020  *:* LISTEN
> tcp0  0 hadoop-master:33356 hadoop-master:8020
>  ESTABLISHED
> tcp0  0 hadoop-master:8020  hadoop-master:33356
> ESTABLISHED
> tcp0  0 hadoop-master:55135 hadoop-master:8020
>  TIME_WAIT
>
> And this is my */etc/hosts* file
>
> #127.0.0.1  localhost
> #127.0.1.1  vm6-VirtualBox
> 192.168.23.206  hadoop-master platalytics.com vm6-VirtualBox
> 192.168.23.207  hadoop-slave
> # The following lines are desirable for IPv6 capable hosts
> ::1 ip6-localhost ip6-loopback
> fe00::0 ip6-localnet
> ff00::0 ip6-mcastprefix
> ff02::1 ip6-allnodes
> ff02::2 ip6-allrouters
>
>
> Can you please tell me what's wrong with above configuration and how can i
> check whether it is firewall issue?
>
> Thanks
>
> On Wed, Jun 29, 2016 at 12:11 AM, sreebalineni . 
> wrote:
>
>> Are you able to telnet ping. Check the firewalls as well
>> On Jun 29, 2016 12:39 AM, "Aneela Saleem"  wrote:
>>
>>> Hi all,
>>>
>>> I have setup two nodes cluster with security enabled. I have everything
>>> running successful like namenode, datanode, resourcemanager, nodemanager,
>>> jobhistoryserver etc. But datanode is unable to connect to namenode, as i
>>> can see only one node on the web UI. checking logs of datanode gives
>>> following warning:
>>>
>>> *WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem
>>> connecting to server: hadoop-master/192.168.23.206:8020
>>> <http://192.168.23.206:8020>*
>>>
>>> Rest of the things look fine. Please help me in this regard, what could
>>> be the issue?
>>>
>>
>

Re: datanode is unable to connect to namenode

[Aneela Saleem]

Thanks Sreebalineni for the response.

This is the result of the *netstat -a | grep 8020* command

tcp0  0 hadoop-master:8020  *:* LISTEN
tcp0  0 hadoop-master:33356 hadoop-master:8020
 ESTABLISHED
tcp0  0 hadoop-master:8020  hadoop-master:33356
ESTABLISHED
tcp0  0 hadoop-master:55135 hadoop-master:8020
 TIME_WAIT

And this is my */etc/hosts* file

#127.0.0.1  localhost
#127.0.1.1  vm6-VirtualBox
192.168.23.206  hadoop-master platalytics.com vm6-VirtualBox
192.168.23.207  hadoop-slave
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters


Can you please tell me what's wrong with above configuration and how can i
check whether it is firewall issue?

Thanks

On Wed, Jun 29, 2016 at 12:11 AM, sreebalineni . 
wrote:

> Are you able to telnet ping. Check the firewalls as well
> On Jun 29, 2016 12:39 AM, "Aneela Saleem"  wrote:
>
>> Hi all,
>>
>> I have setup two nodes cluster with security enabled. I have everything
>> running successful like namenode, datanode, resourcemanager, nodemanager,
>> jobhistoryserver etc. But datanode is unable to connect to namenode, as i
>> can see only one node on the web UI. checking logs of datanode gives
>> following warning:
>>
>> *WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting
>> to server: hadoop-master/192.168.23.206:8020 <http://192.168.23.206:8020>*
>>
>> Rest of the things look fine. Please help me in this regard, what could
>> be the issue?
>>
>

datanode is unable to connect to namenode

[Aneela Saleem]

Hi all,

I have setup two nodes cluster with security enabled. I have everything
running successful like namenode, datanode, resourcemanager, nodemanager,
jobhistoryserver etc. But datanode is unable to connect to namenode, as i
can see only one node on the web UI. checking logs of datanode gives
following warning:

*WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting
to server: hadoop-master/192.168.23.206:8020 *

Rest of the things look fine. Please help me in this regard, what could be
the issue?

Re: Setting up secure Multi-Node cluster

[Aneela Saleem]

Thanks Rakesh.

On Tue, Jun 28, 2016 at 8:28 AM, Rakesh Radhakrishnan 
wrote:

> Hi Aneela,
>
> IIUC, Namenode, Datanode is using _HOST pattern in their principal and
> needs to create separate principal for NN and DN if running in different
> machines. I hope the below explanation will help you.
>
> "dfs.namenode.kerberos.principal" is typically set to nn/_HOST@REALM.
> Each Namenode will substitute the _HOST with its own fully qualified
> hostname at startup.The _HOST placeholder allows using the same
> configuration setting on both Active and Standby NameNodes in an HA setup
>
> Similarly "dfs.datanode.kerberos.principal" will set to dn/_HOST@REALM.
> DataNode will substitute _HOST with its own fully qualified hostname at
> startup. The _HOST placeholder allows using the same configuration setting
> on all DataNodes.
>
> Again, if you are using HA setup with QJM,
> "dfs.journalnode.kerberos.principal" will set to jn/_HOST@REALM
>
> >>>>>Do i need to copy all the kerberos configuration files like kdc.conf
> and krb5.conf etc on every node in default locations?
> Yes, you need to place these in appropriate paths in all the machines.
>
> Regards,
> Rakesh
>
> On Tue, Jun 28, 2016 at 3:15 AM, Aneela Saleem 
> wrote:
>
>> Hi all,
>>
>> I have configured Kerberos for single node cluster successfully. I used
>> this
>> <http://queryio.com/hadoop-big-data-docs/hadoop-big-data-admin-guide/queryio/hadoop-security-setup-kerberos.html#add_admin>
>>  documentation
>> for configurations. Now i'm enabling security for multi node cluster and i
>> have some confusions about that. Like
>>
>> How principals would be managed for namenode and data node? because till
>> now i had only one principal *hdfs/_HOST@platalyticsrealm *used for both
>> namenode as well as for datanode? Do i need to add separate principals for
>> both namenode and datanode having different hostname? for example:
>> if my namenode hostname is *hadoop-master* then there should be
>> principal added *nn/hadoop-master@platalyticsrealm *(with appropriate
>> keytab file)
>> if my datanode hostname is *hadoop-slave *then there should be principal
>> added *dn/hadoop-slave@platalyticsrealm* (with appropriate keytab file)
>>
>> Do i need to copy all the kerberos configuration files like kdc.conf and
>> krb5.conf etc on every node in default locations?
>>
>> A little guidance would be highly appreciated. Thanks
>>
>
>

Setting up secure Multi-Node cluster

[Aneela Saleem]

Hi all,

I have configured Kerberos for single node cluster successfully. I used this

documentation
for configurations. Now i'm enabling security for multi node cluster and i
have some confusions about that. Like

How principals would be managed for namenode and data node? because till
now i had only one principal *hdfs/_HOST@platalyticsrealm *used for both
namenode as well as for datanode? Do i need to add separate principals for
both namenode and datanode having different hostname? for example:
if my namenode hostname is *hadoop-master* then there should be principal
added *nn/hadoop-master@platalyticsrealm *(with appropriate keytab file)
if my datanode hostname is *hadoop-slave *then there should be principal
added *dn/hadoop-slave@platalyticsrealm* (with appropriate keytab file)

Do i need to copy all the kerberos configuration files like kdc.conf and
krb5.conf etc on every node in default locations?

A little guidance would be highly appreciated. Thanks

Re: Kerberos Impersonation in Hadoop

[Aneela Saleem]

Thanks Chris, It helped.

Sent from my iPhone

> On 24-Jun-2016, at 01:15, Chris Nauroth  wrote:
> 
> Hello Aneela,
> 
> If your cluster has enabled Kerberos security, then the HADOOP_USER_NAME 
> environment variable has no effect.
> 
> It sounds like you want to test a proxy user scenario, in which 
> authentication is performed as user "hdfs" via Kerberos, but then execution 
> of the request (including any group membership resolution and authorization 
> checks) proceeds as user "michael".  There is a different environment 
> variable named HADOOP_PROXY_USER that can be set to achieve this.
> 
> Does that help?
> 
> --Chris Nauroth
> 
> From: Aneela Saleem 
> Date: Thursday, June 23, 2016 at 12:45 PM
> To: "user@hadoop.apache.org" 
> Subject: Kerberos Impersonation in Hadoop
> 
> Hi all,
> 
> I'm trying Kerberos Impersonation in Hadoop. But i can't get the clear idea 
> what the impersonation is? Whether it's effective in doing HADOOP_USER_NAME 
> from command line or it's something else. It's confusing. I can't understand 
> it from the documentation. 
> 
> Actually what i'm trying to do is to simulate LDAP users on my system when 
> accessing HDFS. Since i'm using group mapping from LDAP that's working fine 
> when i run 'hdfs groups' command. I just want to authenticate whether the 
> user i pass in HADOOP_USER_NAME from command line when accessing HDFS, is 
> actually impersonating an LDAP user or not? How can i verify it. Let's have a 
> look on following usecase:
> 
> -I have a service principal i.e., hdfs/platalytics.com@platalyticsrealm
> -I initiate the authenticate request using this service principal and got TGT 
> for this principal
> -Now when i run the command with any proxy user whether it exists or not
> -HADOOP_USER_NAME=michael hdfs dfs -mkdir /temp it allows to create the temp 
> directory on behalf of 'hdfs' ( michael is an LDAP user)
> 
> But when i initiate an authenticate request through user principal i.e., 
> michael/platalytics.com@platalyticsrealm
> and run the command hdfs dfs -mkdir /temp it says michael doestn't have 
> enough permissions.
> 
> How the things are working i can't understand. How can i test LDAP users? I 
> have not configured PAM for ldap authentication, i want to test it without 
> PAM.
> 
> I have enabled impersonation with following configuration parameters:
> 
> hadoop.proxyuser.hdfs.groups
> Admin,hdfs
> hadoop.proxyuser.hdfs.hosts
> platalytics.com
> Thanks

Kerberos Impersonation in Hadoop

[Aneela Saleem]

Hi all,

I'm trying Kerberos Impersonation in Hadoop. But i can't get the clear idea
what the impersonation is? Whether it's effective in doing HADOOP_USER_NAME
from command line or it's something else. It's confusing. I can't
understand it from the documentation.

Actually what i'm trying to do is to simulate LDAP users on my system when
accessing HDFS. Since i'm using group mapping from LDAP that's working fine
when i run *'hdfs groups' *command. I just want to authenticate whether the
user i pass in *HADOOP_USER_NAME* from command line when accessing HDFS, is
actually impersonating an LDAP user or not? How can i verify it. Let's have
a look on following usecase:

-I have a service principal i.e., hdfs/platalytics.com@platalyticsrealm
-I initiate the authenticate request using this service principal and got
TGT for this principal
-Now when i run the command with any proxy user whether it exists or not
*-HADOOP_USER_NAME=michael hdfs dfs -mkdir /temp *it allows to create the
temp directory on behalf of 'hdfs' ( michael is an LDAP user)

But when i initiate an authenticate request through user principal i.e.,
michael/platalytics.com@platalyticsrealm
and run the command *hdfs dfs -mkdir /temp *it says michael doestn't have
enough permissions.

How the things are working i can't understand. How can i test LDAP users? I
have not configured PAM for ldap authentication, i want to test it without
PAM.

I have enabled impersonation with following configuration parameters:


hadoop.proxyuser.hdfs.groups
Admin,hdfs
hadoop.proxyuser.hdfs.hosts
platalytics.com

Thanks

Configure Hadoop Kerberos to authenticate LDAP users

[Aneela Saleem]

Hi all,

I have configured Kerberos on Hadoop cluster which successfully
authenticates users that reside in Kerberos database. Now i want to make
Hadoop Kerberos to authenticate LDAP users directly instead of local users.
I've been looking it for days but didn't find the correct direction. I
followed this

but
this only adds he principals to LDAP (i don't want this) but i just need to
authenticate already existing LDAP users while accessing Hadoop through
Kerberos. I also considered this

but
i don't know how to do this on my ubuntu machine. Suitable response and
guidance would be highly appreciated.

Thanks

Re: KMS for hadoop

[Aneela Saleem]

Hi Paul,

Can you please guide me what are the basic steps to configure KMS with
Hadoop. Because the documentation here
 is very
brief. And i have non-kerberized cluster. Can you please guide us to take a
very well start.

Thanks

On Wed, Jun 1, 2016 at 4:14 PM, Dietrich, Paul 
wrote:

> It can be setup standalone. The configuration property
> hadoop.kms.authentication.type has a default value of simple.
>
>
>
> Paul
>
>
>
> *From:* Hafiz Mujadid [mailto:hafizmujadi...@gmail.com]
> *Sent:* Wednesday, June 01, 2016 1:49 AM
> *To:* Dietrich, Paul 
> *Cc:* user@hadoop.apache.org
> *Subject:* Re: KMS for hadoop
>
>
>
> Thanks Paul for your response.
>
> Do I need to setup Kerberos before enabling KMS? Or KMS can be setup
> standalone ?
>
> Thanks
>
>
>
> On Tue, May 31, 2016 at 6:56 PM, Dietrich, Paul <
> paul.dietr...@honeywell.com> wrote:
>
> Hafiz,
>
> I didn’t find such a guide, but used documentation from Cloudera and
> Hortonworks to augment what you found. KMS is part of Hadoop (in later
> versions), so it is just a matter of setting the configuration parameters
> to enable it. One thing to note is that KMS should be part of a secure
> cluster so you’ll need to do the necessary steps to setup Kerberos et al,
> which could restrict your universe of tools that you use. Also using the
> file based keystore is probably not a good idea if you are looking to use
> this in an “enterprise” environment. Being able to secure and manage a key
> server is not trivial.
>
>
>
> Paul
>
>
>
> *From:* Hafiz Mujadid [mailto:hafizmujadi...@gmail.com]
> *Sent:* Monday, May 30, 2016 11:04 AM
> *To:* user@hadoop.apache.org
> *Subject:* KMS for hadoop
>
>
>
> Hi,
>
> I am new to hadoop and want to enable KMS for hadoop. I have read this
>  kms
> documentation on hadoop but unable to get idea how to get started. Is there
> any detailed getting started guide for kms? does KMS is supported by
> default and we only need to enable it?
>
>
>
>
>
> Thanks
>
>
>
>
>
> --
>
> Regards: HAFIZ MUJADID
>

Re: Hadoop group mapping LDAP integration

[Aneela Saleem]

My issue is still unresolved and i can't make any progress. Someone please
tell me what could be the issue? May be issue is from certificates side.
May be i'm not generating them correctly. Please help me getting in right
direction. I'm new to Hadoop security

On Mon, Aug 24, 2015 at 2:33 PM, Aneela Saleem 
wrote:

> Attached is the core-site.xml file
>
> On Mon, Aug 24, 2015 at 2:32 PM, Aneela Saleem 
> wrote:
>
>> I created it in pem format. Then i came to know hadoop accepts it in jks
>> format so i converted it
>>
>> On Mon, Aug 24, 2015 at 2:31 PM, Muzaffar Ali 
>> wrote:
>>
>>> In which format you created the keystore?
>>>
>>>
>>>
>>> *From:* Aneela Saleem [mailto:ane...@platalytics.com]
>>> *Sent:* Monday, August 24, 2015 2:25 PM
>>> *To:* Muzaffar Ali
>>> *Subject:* Re: Hadoop group mapping LDAP integration
>>>
>>>
>>>
>>> No
>>>
>>>
>>>
>>> On Mon, Aug 24, 2015 at 2:24 PM, Muzaffar Ali 
>>> wrote:
>>>
>>> Have your problem resolved?
>>>
>>>
>>>
>>> *From:* Aneela Saleem [mailto:ane...@platalytics.com]
>>> *Sent:* Sunday, August 23, 2015 3:09 AM
>>> *To:* user@hadoop.apache.org
>>> *Subject:* Fwd: Hadoop group mapping LDAP integration
>>>
>>>
>>>
>>>
>>>
>>> -- Forwarded message --
>>> From: *Aneela Saleem* 
>>> Date: Sun, Aug 23, 2015 at 3:04 AM
>>> Subject: Hadoop group mapping LDAP integration
>>> To: user@hadoop.apache.org
>>>
>>> Hi all,
>>>
>>>
>>>
>>> I'm trying to sync ldap users and groups into my HDFS. It was working
>>> well with ldap:/// but when i switched to ldaps:/// ( LDAP over ssl ) it
>>> shows no groups when i run 'hdfs groups username' command. Attached is my
>>> core-site.xml file.
>>>
>>>
>>>
>>> Can anyone please look at this and guide me what i'm missing?
>>>
>>>
>>>
>>>
>>>
>>
>>
>

Fwd: Hadoop group mapping LDAP integration

[Aneela Saleem]

-- Forwarded message --
From: Aneela Saleem 
Date: Sun, Aug 23, 2015 at 3:04 AM
Subject: Hadoop group mapping LDAP integration
To: user@hadoop.apache.org


Hi all,

I'm trying to sync ldap users and groups into my HDFS. It was working well
with ldap:/// but when i switched to ldaps:/// ( LDAP over ssl ) it shows
no groups when i run 'hdfs groups username' command. Attached is my
core-site.xml file.

Can anyone please look at this and guide me what i'm missing?







	
   fs.default.name
   hdfs://localhost:9000
  

  hadoop.security.group.mapping
  org.apache.hadoop.security.LdapGroupsMapping


  hadoop.security.group.mapping.ldap.bind.user
  cn=admin,dc=platalytics,dc=com


  hadoop.security.group.mapping.ldap.bind.password
  123


  hadoop.security.group.mapping.ldap.url
  ldaps://127.0.0.1:636/dc=platalytics,dc=com


  hadoop.security.group.mapping.ldap.url
  ldaps://127.0.0.1:636/dc=platalytics,dc=com


	hadoop.security.group.mapping.ldap.ssl
	true


hadoop.security.group.mapping.ldap.ssl.keystore
/etc/ldap/cacert.pem


  hadoop.security.group.mapping.ldap.base
  


  hadoop.security.group.mapping.ldap.search.filter.user
  (&(|(objectclass=user)(objectclass=person)(objectclass=applicationProcess))(cn={0}))


  hadoop.security.group.mapping.ldap.search.filter.group
  (objectclass=groupOfNames)


  hadoop.security.group.mapping.ldap.search.attr.member
  member


  hadoop.security.group.mapping.ldap.search.attr.group.name
  cn

Hadoop group mapping LDAP integration

[Aneela Saleem]

Hi all,

I'm trying to sync ldap users and groups into my HDFS. It was working well
with ldap:/// but when i switched to ldaps:/// ( LDAP over ssl ) it shows
no groups when i run 'hdfs groups username' command. Attached is my
core-site.xml file.

Can anyone please look at this and guide me what i'm missing?







	
   fs.default.name
   hdfs://localhost:9000
  

  hadoop.security.group.mapping
  org.apache.hadoop.security.LdapGroupsMapping


  hadoop.security.group.mapping.ldap.bind.user
  cn=admin,dc=platalytics,dc=com


  hadoop.security.group.mapping.ldap.bind.password
  123


  hadoop.security.group.mapping.ldap.url
  ldaps://127.0.0.1:636/dc=platalytics,dc=com


  hadoop.security.group.mapping.ldap.url
  ldaps://127.0.0.1:636/dc=platalytics,dc=com


	hadoop.security.group.mapping.ldap.ssl
	true


hadoop.security.group.mapping.ldap.ssl.keystore
/etc/ldap/cacert.pem


  hadoop.security.group.mapping.ldap.base
  


  hadoop.security.group.mapping.ldap.search.filter.user
  (&(|(objectclass=user)(objectclass=person)(objectclass=applicationProcess))(cn={0}))


  hadoop.security.group.mapping.ldap.search.filter.group
  (objectclass=groupOfNames)


  hadoop.security.group.mapping.ldap.search.attr.member
  member


  hadoop.security.group.mapping.ldap.search.attr.group.name
  cn