Wellington Chevreuil created HBASE-28044: --------------------------------------------
Summary: Reduce frequency of saving backing map in persistence cache Key: HBASE-28044 URL: https://issues.apache.org/jira/browse/HBASE-28044 Project: HBase Issue Type: Sub-task Reporter: Wellington Chevreuil Assignee: Wellington Chevreuil Currently we always write the whole cache mapping into the persistent map file. This is not a lightweight operation, on a full 1.6TB cache with ten millions of block, this can grow as much as 10GB. In the current persistent cache implementation, we flush it to disk every 1s. If we raise the "checkpoint" period, we risk lose more cache events in the event of a recovery. This proposes reduce the frequency needed to save the backing map as follows: 1) Save every block addition/eviction into a single file in disk; 2) Checkpoint at higher intervals, consolidating all transactions into the larger map file; 3) In the event of failure, recovery would consist of loading the latest map file, then applying all the transactions files sequentially; -- This message was sent by Atlassian Jira (v8.20.10#820010)