[
https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187969#comment-13187969
]
Alejandro Abdelnur commented on HADOOP-7973:
--------------------------------------------
To complicate things a bit more, keep in mind that if you do ask for FS
instance using 2 different UGI instances for the same user you end up with 2
different FS instances in the cache. This behavior -not to choke the NN- forced
Oozie to do a second level cache of FS instances using the username.
In Oozie we've talked about moving all the Hadoop interactions to a a command
pattern where FS and JC get created/closed before/after the command invocation.
And disabling caches completely. This is a huge change in Oozie but I guess
we'll eventually go for it. What I don't know what will be the impact on
security infrastructure (Kerberos) if suddenly, instead of getting new FS on &
off (the first time a user comes to the system), it will be for each app
interaction done by Oozie on behalf of the user.
One possibility would be to make the cache to timeout on inactivity and the FS
instance to reconnect if timedout.
This issue is becoming bigger as we have long running systems using Hadoop,
instead command line calls.
> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>
> Key: HADOOP-7973
> URL: https://issues.apache.org/jira/browse/HADOOP-7973
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 1.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Blocker
> Attachments: HADOOP-7973.patch
>
>
> The way {{FileSystem#close}} works is very problematic. Since the
> {{FileSystems}} are cached, any {{close}} by any caller will cause problems
> for every other reference to it. Will add more detail in the comments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
