Saad Mufti created HBASE-20226:
----------------------------------
Summary: Performance Improvement Taking Large Snapshots In Remote
Filesystems
Key: HBASE-20226
URL: https://issues.apache.org/jira/browse/HBASE-20226
Project: HBase
Issue Type: Improvement
Components: snapshots
Affects Versions: 1.4.0
Environment: HBase 1.4.0 running on an AWS EMR cluster with the
hbase.rootdir set to point to a folder in S3
Reporter: Saad Mufti
When taking a snapshot of any table, one of the last steps is to delete the
region manifests, which have already been rolled up into a larger overall
manifest and thus have redundant information.
This proposal is to do the deletion in a thread pool bounded by
hbase.snapshot.thread.pool.max . For large tables with a lot of regions, the
current single threaded deletion is taking longer than all the rest of the
snapshot tasks when the Hbase data and the snapshot folder are both in a remote
filesystem like S3.
I have a patch for this proposal almost ready and will submit it tomorrow for
feedback, although I haven't had a chance to write any tests yet.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
