[
https://issues.apache.org/jira/browse/HIVE-21697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-21697:
------------------------------
Attachment: HIVE-21697.7.patch
> Remove periodical full refresh in HMS cache
> -------------------------------------------
>
> Key: HIVE-21697
> URL: https://issues.apache.org/jira/browse/HIVE-21697
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Priority: Major
> Attachments: HIVE-21697.1.patch, HIVE-21697.2.patch,
> HIVE-21697.3.patch, HIVE-21697.4.patch, HIVE-21697.5.patch,
> HIVE-21697.6.patch, HIVE-21697.7.patch
>
>
> In HIVE-18661, we added periodical notification based refresh in HMS cache.
> We shall remove periodical full refresh to simplify the code as it will no
> longer be used. In the mean time, we introduced mechanism to provide
> monotonic reads through the CachedStore.commitTransaction. This will no
> longer be needed after HIVE-21637. So I will remove related code as well.
> This will provide some performance benefits include:
> 1. We don't have to slow down write to catch up notification logs. Write can
> be done immediately and tag the cache with writeids
> 2. We can read from cache even if updateUsingNotificationEvents is running.
> Read will compare the writeids of the cache so monotonic reads will be
> guaranteed
> I'd like to put a patch separately with HIVE-21637 so it can be tested
> independently. HMW will use periodical notification based refresh to update
> cache. And it will temporarily lift the monotonic reads guarantee until
> HIVE-21637 checkin.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
