[
https://issues.apache.org/jira/browse/HBASE-1460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714989#action_12714989
]
Jonathan Gray commented on HBASE-1460:
--------------------------------------
Thanks for review stack.
Posting new patch in a second. To answer your questions:
- Class comments added
- setBuffer is currently used when an already cached block is cached again.
behavior is defined in javadocs. its assumed this will never happen though.
- align moved to Bytes class and unit test added!
- the get() method in the queue is used in the eviction algorithm, could
actually use an iterator instead would keep us from doing extra work in the
queue
- yes, should just name it LRU just separate because i've not had a trunk with
the recently committed one yet
- comments removed (was keeping that around because i had to rework some of it
into the new constructor)
- new patch includes a growing unit test for the cache. i'm planning on adding
one or two more methods but there are three tests which cover almost all
functionality and it passes. I was seeing some failures on the third test, but
then it went away with seemingly no change. I suspect there might be a race
condition in the calling of the eviction thread. The call to the thread is
heavily influenced by the solr implementation, dig in there if you could.
> Concurrent LRU Block Cache
> --------------------------
>
> Key: HBASE-1460
> URL: https://issues.apache.org/jira/browse/HBASE-1460
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: io
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1460-v1.patch
>
>
> The LRU-based block cache that will be committed in HBASE-1192 is thread-safe
> but contains a big lock on the hash map. Under high load, the block cache
> will be hit very heavily from a number of threads, so it needs to be built to
> handle massive concurrency.
> This issue aims to implement a new block cache with LRU eviction, but backed
> by a ConcurrentHashMap and a separate eviction thread. Influence will be
> drawn from Solr's ConcurrentLRUCache, however there are major differences
> because solr treats all cached elements as equal size whereas we are
> dependent on our HeapSize interface with realistic (though approximate) heap
> usage.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
