[ https://issues.apache.org/jira/browse/HBASE-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HBASE-25318 started by Mate Szalay-Beko. ------------------------------------------------ > Configure where IntegrationTestImportTsv generates HFiles > --------------------------------------------------------- > > Key: HBASE-25318 > URL: https://issues.apache.org/jira/browse/HBASE-25318 > Project: HBase > Issue Type: Improvement > Components: integration tests > Reporter: Mate Szalay-Beko > Assignee: Mate Szalay-Beko > Priority: Minor > > Currently IntegrationTestImportTsv is generating HFiles under the working > directory of the hdfs user executing the tool, before bulkloading it into > HBase. > Assuming you encrypt the HBase root directory within HDFS (using HDFS > Transparent Encryption), you can bulkload HFiles only if they sit in the same > encryption zone in HDFS as the HBase root directory itself. > When IntegrationTestImportTsv is executed against a real distributed cluster > and the working directory of the current user (e.g. /user/hbase) is not in > the same encryption zone as the HBase root directory (e.g. /hbase/data) then > you will get an exception: > > {code:java} > ERROR org.apache.hadoop.hbase.regionserver.HRegion: There was a partial > failure due to IO when attempting to load d : > hdfs://mycluster/user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc > org.apache.hadoop.ipc.RemoteException(java.io.IOException): > /user/hbase/test-data/22d8460d-04cc-e032-88ca-2cc20a7dd01c/IntegrationTestImportTsv/hfiles/d/74655e3f8da142cb94bc31b64f0475cc > can't be moved into an encryption zone. > {code} > > In this ticket I make it configurable where the IntegrationTestImportTsv > generates the HFiles. From now, one can execute this integration test on HDFS > Transparent Encryption enabled clusters, like: > {code:java} > ./bin/hbase org.apache.hadoop.hbase.mapreduce.IntegrationTestImportTsv -D > IntegrationTestImportTsv.generatedHFileFolder=/<my hbase encryption zone > path>/testdata{code} -- This message was sent by Atlassian Jira (v8.3.4#803005)