[
https://issues.apache.org/jira/browse/HIVE-16979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511753#comment-16511753
]
Daniel Dai commented on HIVE-16979:
-----------------------------------
Haven't been tested, the new patch use a different cache strategy. UGI object
will not be evicted if it is still in use. Another change is UGI object will
not be shared across sessions in case UserGroupInformation and FileSystem is
not thread safe. Finished session will release UGI object in the cache.
UGICache will keep the idle object around until eviction.
> Cache UGI for metastore
> -----------------------
>
> Key: HIVE-16979
> URL: https://issues.apache.org/jira/browse/HIVE-16979
> Project: Hive
> Issue Type: Improvement
> Reporter: Tao Li
> Assignee: Daniel Dai
> Priority: Major
> Attachments: HIVE-16979.1.patch, HIVE-16979.2.patch,
> HIVE-16979.3.patch, HIVE-16979.4.patch
>
>
> FileSystem.closeAllForUGI is called per request against metastore to dispose
> UGI, which involves talking to HDFS name node and is time consuming. So the
> perf improvement would be caching and reusing the UGI.
> Per FileSystem.closeAllForUG call could take up to 20 ms as E2E latency
> against HDFS. Usually a Hive query could result in several calls against
> metastore, so we can save up to 50-100 ms per hive query.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
