[
https://issues.apache.org/jira/browse/HBASE-22890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zbq.dean updated HBASE-22890:
-----------------------------
Release Note:
Verify cache file integrity before BucketCache retrieve from file if the
IOEngine is persistent. Prevent inconsistency between backingMap and cache file
if cache file has been modified.
The process of verifying integrity is to compare checksum. The checksum is
calculated using java.security.MessageDigest class according to cache files
full path, actual size and last modified time.
The encryption algorithms is configurable and default is MD5. The configuration
option is : hbase.bucketcache.persistent.file.integrity.check.algorithm
was:
Verify cache file integrity before BucketCache retrieve from file if the
IOEngine is persistent. Prevent inconsistency between backingMap and cache file
if cache file has been modified.
The process of verifying integrity is to compare checksum. The checksum is
calculated using java.security.MessageDigest class according to cache files
full path, actual size and last modified time.
The encryption algorithms is configurable and default is MD5.
> Verify the files when RegionServer is starting and BucketCache is in file mode
> ------------------------------------------------------------------------------
>
> Key: HBASE-22890
> URL: https://issues.apache.org/jira/browse/HBASE-22890
> Project: HBase
> Issue Type: Improvement
> Components: BucketCache
> Affects Versions: 1.4.6, 1.4.10, 1.3.5
> Reporter: zbq.dean
> Priority: Major
> Fix For: 1.5.0, 1.4.11
>
> Attachments: HBASE-22890.branch-1.0001.patch
>
>
> We use FileIOEngine in BucketCache and have configured
> <hbase.bucketcache.persistent.path>. If the RegionServer is stopped, and the
> cache data file or the backingMap persistence file is deleted or changed
> before the RegionServer is restarted. When restart RegionServer, the
> backingMap and cache data are Inconsistent, and get or scan command will get
> IOException like follows:
> bucket.BucketCache: Failed reading block
> 18951416e2ca4e8bbc11523f4f5ea576_1125745279 from bucket cache
> java.io.IOException: Invalid HFile block magic:
> \x00\x00\x00\x00\x00\x00\x00\x00
> at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154)
> at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:167)
> at org.apache.hadoop.hbase.io.hfile.HFileBlock.<init>(HFileBlock.java:347)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$1.deserialize(HFileBlock.java:247)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$1.deserialize(HFileBlock.java:226)
> at
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getBlock(BucketCache.java:514)
> ......
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
