Hi all, I have some issues in trying to restore Hbase tables from a backup that resides in S3 when the hbase root dir is in an encrypted zone in HDFS.
HBase version: 2.0.2 HDFS version: 3.1.1 Ranger version: 1.2.0 I can successfully create both full and incremental backup and export them to S3 using the commands "hbase backup create full ...." or "hbase backup create incremental ...", but when I try to restore it the job fails with the following error: WARN [LoadIncrementalHFiles-1] tool.LoadIncrementalHFiles: Attempt to bulk load region containing into table SYSTEM.CATALOG with files [family:0 path:hdfs://test/user/hbase/hbase-staging/bulk_output-default-SYSTEM.CATALOG-1617818800191/0/6b877eb391ee49189061946db6fdd21f] failed. This is recoverable and they will be retried. 2021-04-07 20:07:29,339 INFO [main] tool.LoadIncrementalHFiles: Split occurred while grouping HFiles, retry attempt 9 with 1 files remaining to group or split 2021-04-07 20:07:29,375 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles: Trying to load hfile=hdfs://test/user/hbase/hbase-staging/bulk_output-default-SYSTEM.CATALOG-1617818800191/0/6b877eb391ee49189061946db6fdd21f first=Optional[\x00\x00T1] last=Optional[\x00\x00T1\x00B\x000] 2021-04-07 20:07:29,501 WARN [LoadIncrementalHFiles-1] tool.LoadIncrementalHFiles: Attempt to bulk load region containing into table SYSTEM.CATALOG with files [family:0 path:hdfs://test/user/hbase/hbase-staging/bulk_output-default-SYSTEM.CATALOG-1617818800191/0/6b877eb391ee49189061946db6fdd21f] failed. This is recoverable and they will be retried. 2021-04-07 20:07:29,507 INFO [main] tool.LoadIncrementalHFiles: Split occurred while grouping HFiles, retry attempt 10 with 1 files remaining to group or split 2021-04-07 20:07:29,527 ERROR [main] tool.LoadIncrementalHFiles: ------------------------------------------------- Bulk load aborted with some files not yet loaded: ------------------------------------------------- hdfs://test/user/hbase/hbase-staging/bulk_output-default-SYSTEM.CATALOG-1617818800191/0/6b877eb391ee49189061946db6fdd21f The mapreduce job that copies from S3 to /user/hbase/hbase-staging works perfectly. The encryption zone is on HDFS directory /apps/hbase, the hbase rootdir is in /apps/hbase/data and the hbase staging dir is in /apps/hbase/staging. I also tried to create an encryption zone on /user/hbase/hbase-staging, but it didn't work. Does the backup/restore utility supports HDFS encrpytion zone or am I missing something? If it does not support encryption, is there a workaround for this kind of situation (that does not oblige me to remove the encrypted zone if I need to restore a table)? Thanks you very much, Davide
