harishjp commented on a change in pull request #2584:
URL: https://github.com/apache/hive/pull/2584#discussion_r691416567
##########
File path:
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/conf/JdbcStorageConfigManager.java
##########
@@ -77,6 +81,15 @@ public static void copySecretsToJob(Properties props,
Map<String, String> jobSec
String key = props.getProperty(CONFIG_PWD_KEY);
passwd = Utilities.getPasswdFromKeystore(keystore, key);
}
+ if (passwd == null) {
Review comment:
Make sense, I'll add a check to error out if multiple password sources
are set only one of password, keystore or uri should be set. And will do the
same in JdbcDatabaseAccessor. I was a bit lazy, I think I'll refactor the code
to get any password.
##########
File path: pom.xml
##########
@@ -267,6 +269,28 @@
<dependencyManagement>
<dependencies>
<!-- dependencies are always listed in sorted order by groupId,
artifactId -->
+ <dependency>
+ <groupId>com.amazonaws</groupId>
Review comment:
SecretCache is a class which understand how to fetch secrets from AWS
Secrets Service and also provides caching support. The basic framework is
pluggable, I can refactor the entire AWS secrets loading into another project
which can be built when some profile is set.
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/secrets/AWSSecretsManagerSecretSource.java
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.secrets;
+
+import com.amazonaws.secretsmanager.caching.SecretCache;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Preconditions;
+
+import java.io.IOException;
+import java.net.URI;
+
+/**
+ * Implementation of SecretSource which loads secrets from AWS Secrets Manager.
+ * The format of the uri is "aws-sm:///{key-name-or-arn}"
+ * It uses aws secrets cache sdk to fetch and refresh the secret, the
environment must be setup so that the default
+ * client can load the secret else it will fail.
+ * It expects the secret fetched to be a json string with "password" as the
key for password, this is default for
+ * redshift, rds or external database configs. It does not make use of any
other fields.
+ */
+public class AWSSecretsManagerSecretSource implements SecretSource {
+ // Do not create SecretCache here, it fails to initialize in non-aws aware
environments.
+ private volatile SecretCache cache = null;
+ private final ObjectMapper mapper = new ObjectMapper();
+
+ /**
+ * @return Fixed string aws-sm.
+ */
+ @Override
+ public String getURIScheme() {
+ return "aws-sm";
+ }
+
+ /**
+ * This load the secret from aws-secrets manager.
+ * @param uri The uri should be of format: aws-sm:///{key-arn-or-name}
+ * @return The secret fetched from AWS.
+ * @throws IOException
+ */
+ @Override
+ public String getSecret(URI uri) throws IOException {
Review comment:
I do not know if it makes sense to do this. The data in the SecretsCache
is already plain string cached for a long time and making this short lived
gives us no advantage. And if we are worried about it being visible in heap
dump, byte[] is also visible in heap dump. I do not know if one has distinct
advantage over another.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
