[
https://issues.apache.org/jira/browse/HADOOP-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931690#action_12931690
]
Devaraj Das commented on HADOOP-6978:
-------------------------------------
We have noticed that sometimes the C calls like getpwuid_r ends up making
direct calls to the ldap server. It probably is configuration/environment
specific, but in Yahoo! the password entries are maintained by the ldap server.
In order to prevent ldap servers from getting overloaded with password
look-ups, we have a daemon called nscd run on all the compute nodes, that
caches the results of such look-ups. The calls such as getpwuid_r should
terminate at the local nscd daemon, but if, for whatever reason, the nscd
daemon is down on the node, the calls end up talking to the ldap server
directly. Apparently, nscd is not that stable...
We have seen the above happening at Yahoo! and in a couple of occasions brought
down the ldap servers. So I was wondering whether we should reduce the number
of calls to the getpwuid_r and such by caching the resolutions
{uid,gid}->{username,groupname} in Hadoop.. Thoughts?
> Add JNI support for secure IO operations
> ----------------------------------------
>
> Key: HADOOP-6978
> URL: https://issues.apache.org/jira/browse/HADOOP-6978
> Project: Hadoop Common
> Issue Type: New Feature
> Components: io, native, security
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Fix For: 0.22.0
>
> Attachments: hadoop-6978.txt, hadoop-6978.txt
>
>
> In support of MAPREDUCE-2096, we need to add some JNI functionality. In
> particular, we need the ability to use fstat() on an open file stream, and to
> use open() with O_EXCL, O_NOFOLLOW, and without O_CREAT.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
