Hello, if someone is interested in this topic, I found a solution analyzing the source code of the class org.apache.hadoop.hbase.backup.mapreduce.MapReduceRestoreJob.
In here, it uses BackupUtils.getBulkOutputDir() that refers to a configuration HConstants.TEMPORARY_FS_DIRECTORY_KEY (hbase.fs.tmp.dir). Changing this property to a location inside the same encryption zone of hbase.rootdir solves the problem Bye, Davide <https://github.com/apache/hbase/tree/master/hbase-backup/src/main/java/org> Il giorno gio 8 apr 2021 alle ore 08:10 Davide Vergari < [email protected]> ha scritto: > Hi all, > I have some issues in trying to restore Hbase tables from a backup that > resides in S3 when the hbase root dir is in an encrypted zone in HDFS. > > HBase version: 2.0.2 > HDFS version: 3.1.1 > Ranger version: 1.2.0 > > I can successfully create both full and incremental backup and export them > to S3 using the commands "hbase backup create full ...." or "hbase backup > create incremental ...", but when I try to restore it the job fails with > the following error: > > WARN [LoadIncrementalHFiles-1] tool.LoadIncrementalHFiles: Attempt to > bulk load region containing into table SYSTEM.CATALOG with files [family:0 > path:hdfs://test/user/hbase/hbase-staging/bulk_output-default-SYSTEM.CATALOG-1617818800191/0/6b877eb391ee49189061946db6fdd21f] > failed. This is recoverable and they will be retried. > 2021-04-07 20:07:29,339 INFO [main] tool.LoadIncrementalHFiles: Split > occurred while grouping HFiles, retry attempt 9 with 1 files remaining to > group or split > 2021-04-07 20:07:29,375 INFO [LoadIncrementalHFiles-0] > tool.LoadIncrementalHFiles: Trying to load > hfile=hdfs://test/user/hbase/hbase-staging/bulk_output-default-SYSTEM.CATALOG-1617818800191/0/6b877eb391ee49189061946db6fdd21f > first=Optional[\x00\x00T1] last=Optional[\x00\x00T1\x00B\x000] > 2021-04-07 20:07:29,501 WARN [LoadIncrementalHFiles-1] > tool.LoadIncrementalHFiles: Attempt to bulk load region containing into > table SYSTEM.CATALOG with files [family:0 > path:hdfs://test/user/hbase/hbase-staging/bulk_output-default-SYSTEM.CATALOG-1617818800191/0/6b877eb391ee49189061946db6fdd21f] > failed. This is recoverable and they will be retried. > 2021-04-07 20:07:29,507 INFO [main] tool.LoadIncrementalHFiles: Split > occurred while grouping HFiles, retry attempt 10 with 1 files remaining to > group or split > 2021-04-07 20:07:29,527 ERROR [main] tool.LoadIncrementalHFiles: > ------------------------------------------------- > Bulk load aborted with some files not yet loaded: > ------------------------------------------------- > > hdfs://test/user/hbase/hbase-staging/bulk_output-default-SYSTEM.CATALOG-1617818800191/0/6b877eb391ee49189061946db6fdd21f > > The mapreduce job that copies from S3 to /user/hbase/hbase-staging works > perfectly. > > The encryption zone is on HDFS directory /apps/hbase, the hbase rootdir is > in /apps/hbase/data and the hbase staging dir is in /apps/hbase/staging. > > I also tried to create an encryption zone on /user/hbase/hbase-staging, > but it didn't work. > > Does the backup/restore utility supports HDFS encrpytion zone or am I > missing something? If it does not support encryption, is there a workaround > for this kind of situation (that does not oblige me to remove the encrypted > zone if I need to restore a table)? > > > Thanks you very much, > Davide > >
