[
https://issues.apache.org/jira/browse/KYLIN-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333789#comment-16333789
]
Zhong Yanghong commented on KYLIN-2894:
---------------------------------------
* For a {{CubeInstance}}, its signature for query is its last build time. If
its last build time changes, the cube data is supposed to be changed resulting
in query cache expiration.
* For a {{HybridInstance}}, its signature for query is the maximum last build
time of its realization instances. Once one of the realization's signature
changes, its last build time should become the maximum and the signature of the
related hybrid instance.
* For a {{SQLResponse}}, it may relate to multiple realizations. Then its
signature is the maximum last build time of those realizations.
> Change the query cache expiration strategy by signature checking
> ----------------------------------------------------------------
>
> Key: KYLIN-2894
> URL: https://issues.apache.org/jira/browse/KYLIN-2894
> Project: Kylin
> Issue Type: Sub-task
> Components: Query Engine
> Reporter: Zhong Yanghong
> Assignee: Zhong Yanghong
> Priority: Major
>
> Currently to invalid query cache, {{CacheService}} will either invoke
> {{cleanDataCache}} or {{cleanAllDataCache}}. Both methods will clear all of
> the query cache, which is very inefficient. In eBay PROD environment, there's
> around 400 cubing jobs per day, which means the query cache will be cleared
> very 4 minutes. Then we introduced a signature based cache invalidation
> strategy. The basic idea is as follows:
> * Add a signature for {{SQLResponse}}, here we choose the cube last build time
> * When fetch {{SQLResponse}} for cache, first check whether the signature is
> consistent. If not, this cached value is overdue and will be invalidate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
