[
https://issues.apache.org/jira/browse/STORM-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Li resolved STORM-3494.
-----------------------------
Fix Version/s: 2.2.0
Resolution: Fixed
Merged to master
> Use UserGroupInformation to login to HDFS only once per process
> ---------------------------------------------------------------
>
> Key: STORM-3494
> URL: https://issues.apache.org/jira/browse/STORM-3494
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Ethan Li
> Assignee: Ethan Li
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.2.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> UserGroupInformation (UGI) loginUserFromKeytab should be used only once in a
> process to login to hdfs because it overrides static fields. Also
> loginUserFromKeytabAndReturnUGI function is also problematic according to
> hadoop team. So the correct way to connect to hdfs is to use UGI
> loginUserFromKeytab once and only in a process.
> Currently we only use HDFS in hdfs-blobstore. It works correctly. But the
> code is implemented in the hdfs-blobstore plugin. It will be problematic if
> we want to add another plugin which also needs to connect to HDFS.
> So the proposal here is to remove the login piece of code from
> hdfs-blobstore. And explicitly login to hdfs once and only once when the
> server (nimbus, supervisor, etc) or the client (storm cli command) launches.
> It can guarantee one login per process.
> The plugins like hdfs-blobstore then simply assume the process has already
> logged in.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
