[
https://issues.apache.org/jira/browse/HBASE-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-27061.
------------------------------------------
Resolution: Fixed
Thanks for the fix, [~sergey.soldatov]. Had merged into master, branch-2 and
branch-2.5.
> two phase bulkload is broken when SFT is in use.
> ------------------------------------------------
>
> Key: HBASE-27061
> URL: https://issues.apache.org/jira/browse/HBASE-27061
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.6.0
> Reporter: Sergey Soldatov
> Assignee: Sergey Soldatov
> Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> In HBASE-26707 for the SFT case, we are writing files directly to the region
> location. For that we are using HRegion.regionDir as the staging directory.
> The problem is that in reality, this dir is pointing to the WAL dir, so for
> S3 deployments that would be pointing to the hdfs. As the result during the
> execution of LoadIncrementalHFiles the process failed with the exception:
> {noformat}
> 2022-05-24 03:31:23,656 ERROR
> org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager: Failed to
> complete bulk load
> java.lang.IllegalArgumentException: Wrong FS
> hdfs://ns1//hbase-wals/data/default/employees/4f367b303da4fed7667fff07fd4c6066/department/acd971097924463da6d6e3a15f9527da
> -expected s3a://hbase
> at
> org.apache.hadoop.fs.s3native.S3xLoginHelper.checkPath(S3xLoginHelper.java:224)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.checkPath(S3AFileSystem.java:1375)
> at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:647)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.makeQualified(S3AFileSystem.java:1337)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.qualify(S3AFileSystem.java:1363)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:3521)
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:511)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:397)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:387)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
> at
> org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$SecureBulkLoadListener.prepareBulkLoad(SecureBulkLoadManager.java:397)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:6994)
> at
> org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$1.run(SecureBulkLoadManager.java:291)
> at
> org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$1.run(SecureBulkLoadManager.java:266)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1879)
> at
> org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager.secureBulkLoadHFiles(SecureBulkLoadManager.java:266)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.bulkLoadHFile(RSRpcServices.java:2453)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45821)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:359)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:339)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
