[
https://issues.apache.org/jira/browse/HIVE-11762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737118#comment-14737118
]
Sergio Peña commented on HIVE-11762:
------------------------------------
Hi [~asuresh]
The KeyProvider is used for running tests with encrypted tables. It is also
used on Hive to check the encrypted key strength when attempting to copy tables
between different encryption zones.
But I see the error here is only when configuring the MiniDFS for Hive tests.
If you go to {{Hadoop23Shims.getMiniDfs()}}, you will see that the KeyProvider
is obtained from {{miniDFSCluster}}, and then set again using the same
{{miniDFSCluster}} object. That's the only place where we use the
{{KeyProviderCryptoExtension}}. It is weird we get and set again, but HDFS 2.6
does not have the keyProvider assigned even if the get() method returns one (it
might be a bug). However, this KeyProvider is needed on the MiniDFS to be able
to run TestEncryptedHDFSCliDriver.
Is there a way to get the HDFS version on Hadoop23Shims so so that we use
KeyProvider instead of KeyProviderCryptoExtension?
> TestHCatLoaderEncryption failures when using Hadoop 2.7
> -------------------------------------------------------
>
> Key: HIVE-11762
> URL: https://issues.apache.org/jira/browse/HIVE-11762
> Project: Hive
> Issue Type: Bug
> Components: Shims, Tests
> Reporter: Jason Dere
>
> When running TestHCatLoaderEncryption with -Dhadoop23.version=2.7.0, we get
> the following error during setup():
> {noformat}
> testReadDataFromEncryptedHiveTableByPig[5](org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption)
> Time elapsed: 3.648 sec <<< ERROR!
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hdfs.DFSClient.setKeyProvider(Lorg/apache/hadoop/crypto/key/KeyProviderCryptoExtension;)V
> at
> org.apache.hadoop.hive.shims.Hadoop23Shims.getMiniDfs(Hadoop23Shims.java:534)
> at
> org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.initEncryptionShim(TestHCatLoaderEncryption.java:252)
> at
> org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:200)
> {noformat}
> It looks like between Hadoop 2.6 and Hadoop 2.7, the argument to
> DFSClient.setKeyProvider() changed:
> {noformat}
> @VisibleForTesting
> - public void setKeyProvider(KeyProviderCryptoExtension provider) {
> - this.provider = provider;
> + public void setKeyProvider(KeyProvider provider) {
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
