[ https://issues.apache.org/jira/browse/HAWQ-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Denissov updated HAWQ-1622: ------------------------------------- Summary: Cache PXF proxy UGI so that cleanup of FileSystem cache doesn't have to be done on each request (was: Cache PXF proxy UGI so that cleanup of FileSystem cache does't have to be done on each request) > Cache PXF proxy UGI so that cleanup of FileSystem cache doesn't have to be > done on each request > ----------------------------------------------------------------------------------------------- > > Key: HAWQ-1622 > URL: https://issues.apache.org/jira/browse/HAWQ-1622 > Project: Apache HAWQ > Issue Type: Improvement > Components: PXF > Reporter: Alexander Denissov > Assignee: Divya Bhargov > Priority: Major > > Closing PXF proxy UGIs on each request (implemented in HAWQ-1621) slows down > PXF request response time significantly when several threads work > concurrently as it locks FileSystem cache and holds the lock while the > cleanup of DFSClients is completed. > This can be avoided by caching the proxy UGI for a given proxy user between > requests. Care must be taken to remove the cached entry after some > pre-defined TTL if and only if there are no current threads using any > FileSystem entries held by the cache. A combination of TTL-based cache with > ref-counting might be utilized to achieve this. > > For some example of this, see: > https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/UserGroupInformationService.java > Caching UGIs might be tricky when Kerberos support is implemented later, see: > https://issues.apache.org/jira/browse/HIVE-3098?focusedCommentId=13398979&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13398979 -- This message was sent by Atlassian JIRA (v7.6.3#76005)