[
https://issues.apache.org/jira/browse/DRILL-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068378#comment-15068378
]
Zelaine Fong commented on DRILL-4161:
-------------------------------------
If Fishbowl does not update their hive metastore very frequently, it is highly
recommended that they change the following setting in their HIVE storage plugin
configuration. The following example shows to change caching policy to 600
seconds after every access.
Here's an example showing the parameters that need to be changed in the Hive
storage plugin configuration to increase the cache invalidation time to 600
seconds, after the last time a cached object was accessed.
"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting,
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
}
> Make Hive Metastore client caching user configurable.
> -----------------------------------------------------
>
> Key: DRILL-4161
> URL: https://issues.apache.org/jira/browse/DRILL-4161
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Drill leverages LoadingCache in hive metastore client, in order to avoid the
> long access time to hive metastore server. However, there is a tradeoff
> between caching stale data and the possibility of cache hit.
> For instance, DRILL-3893 changes cache invalidation policy to "1 minute after
> last write", to avoid the chances of hitting stale data. However, it also
> implies that the cached data would be only valid for 1 minute after
> loading/write.
> It's desirable to allow user to configure the caching policy, per their
> individual use case requirement. In particular, we probably should allow user
> to specify:
> 1) caching invalidation policy : expire after last access, or expire after
> last write.
> 2) cache TTL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
