Alexander Denissov created HAWQ-1622:
----------------------------------------
Summary: Cache PXF proxy UGI so that cleanup of FileSystem cache
does't have to be done on each request
Key: HAWQ-1622
URL: https://issues.apache.org/jira/browse/HAWQ-1622
Project: Apache HAWQ
Issue Type: Improvement
Reporter: Alexander Denissov
Assignee: Radar Lei
Closing PXF proxy UGIs on each request (implemented in HAWQ-1621) slows down
PXF request response time significantly when several threads work concurrently
as it locks FileSystem cache and holds the lock while the cleanup of DFSClients
is completed.
This can be avoided by caching the proxy UGI for a given proxy user between
requests. Care must be taken to remove the cached entry after some pre-defined
TTL if and only if there are no current threads using any FileSystem entries
held by the cache. A combination of TTL-based cache with ref-counting might be
utilized to achieve this.
For some example of this, see:
https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/UserGroupInformationService.java
Caching UGIs might be tricky when Kerberos support is implemented later, see:
https://issues.apache.org/jira/browse/HIVE-3098?focusedCommentId=13398979&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13398979
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
