[
https://issues.apache.org/jira/browse/HADOOP-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tom White updated HADOOP-1398:
------------------------------
Attachment: hadoop-blockcache-v3.patch
This version (v3) changes the cache to a memory sensitive cache, implemented
using SoftReferences
(http://commons.apache.org/collections/api-release/org/apache/commons/collections/map/ReferenceMap.html).
See HADOOP-2624 for background.
Also, block caching can be enabled on a column-family basis. The size of the
block is a system wide setting - this could be adjustable on a per-column basis
in the future, if it were deemed necessary.
I'm still looking at a performance comparison.
> Add in-memory caching of data
> -----------------------------
>
> Key: HADOOP-1398
> URL: https://issues.apache.org/jira/browse/HADOOP-1398
> Project: Hadoop
> Issue Type: New Feature
> Components: contrib/hbase
> Reporter: Jim Kellerman
> Priority: Trivial
> Attachments: hadoop-blockcache-v2.patch, hadoop-blockcache-v3.patch,
> hadoop-blockcache.patch
>
>
> Bigtable provides two in-memory caches: one for row/column data and one for
> disk block caches.
> The size of each cache should be configurable, data should be loaded lazily,
> and the cache managed by an LRU mechanism.
> One complication of the block cache is that all data is read through a
> SequenceFile.Reader which ultimately reads data off of disk via a RPC proxy
> for ClientProtocol. This would imply that the block caching would have to be
> pushed down to either the DFSClient or SequenceFile.Reader
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
