[
https://issues.apache.org/jira/browse/HIVE-20440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613731#comment-16613731
]
Antal Sinkovits commented on HIVE-20440:
----------------------------------------
Yes, this is a good idea.
But I would not bother with the eviction handler, instead fill both caches.
Here is the idea:
There are two caches. L1 is a time based one (with a maintenance thread
cleaning it), and L2 is a soft referenced (I don't think I need a cleanup
thread here).
{noformat}
At first I check L1.
If its a miss, i will check L2.
If I miss again, I load the data in both caches.
If its a hit, I would take it from L2 to return (I think it would
make sense to put it back into L1 as well)
If its a hit in L1, I return it.
{noformat}
If the data is in L1, I can be sure that its in L2 as well, because L1 contains
strong references to the object.
Since these are references, its cheap to store them in two places.
Do you see any flaw in this approach?
> Create better cache eviction policy for SmallTableCache
> -------------------------------------------------------
>
> Key: HIVE-20440
> URL: https://issues.apache.org/jira/browse/HIVE-20440
> Project: Hive
> Issue Type: Improvement
> Components: Spark
> Reporter: Antal Sinkovits
> Assignee: Antal Sinkovits
> Priority: Major
> Attachments: HIVE-20440.01.patch, HIVE-20440.02.patch,
> HIVE-20440.03.patch, HIVE-20440.04.patch
>
>
> Enhance the SmallTableCache, to use guava cache with soft references, so that
> we evict when there is memory pressure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
