[
https://issues.apache.org/jira/browse/HBASE-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-25318 started by Mate Szalay-Beko.
------------------------------------------------
> Configure where IntegrationTestImportTsv generates HFiles
> ---------------------------------------------------------
>
> Key: HBASE-25318
> URL: https://issues.apache.org/jira/browse/HBASE-25318
> Project: HBase
> Issue Type: Improvement
> Components: integration tests
> Reporter: Mate Szalay-Beko
> Assignee: Mate Szalay-Beko
> Priority: Minor
>
> Currently IntegrationTestImportTsv is generating HFiles under the working
> directory of the hdfs user executing the tool, before bulkloading it into
> HBase.
> Assuming you encrypt the HBase root directory within HDFS (using HDFS
> Transparent Encryption), you can bulkload HFiles only if they sit in the same
> encryption zone in HDFS as the HBase root directory itself.
> When IntegrationTestImportTsv is executed against a real distributed cluster
> and the working directory of the current user (e.g. /user/hbase) is not in
> the same encryption zone as the HBase root directory (e.g. /hbase/data) then
> you will get an exception:
>
> {code:java}
> ERROR org.apache.hadoop.hbase.regionserver.HRegion: There was a partial
> failure due to IO when attempting to load d :
> hdfs://mycluster/user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc
> org.apache.hadoop.ipc.RemoteException(java.io.IOException):
> /user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc
> can't be moved into an encryption zone.
> {code}
>
> In this ticket I make it configurable where the IntegrationTestImportTsv
> generates the HFiles. From now, one can execute this integration test on HDFS
> Transparent Encryption enabled clusters, like:
> {code:java}
> ./bin/hbase org.apache.hadoop.hbase.mapreduce.IntegrationTestImportTsv -D
> IntegrationTestImportTsv.generatedHFileFolder=/<my hbase encryption zone
> path>/testdata{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
