Vihang Karajgaonkar created HIVE-16913:
------------------------------------------
Summary: Support per-session S3 credentials
Key: HIVE-16913
URL: https://issues.apache.org/jira/browse/HIVE-16913
Project: Hive
Issue Type: Improvement
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
Currently, the credentials needed to support Hive-on-S3 (or any other
cloud-storage) need to be to the hive-site.xml. Either using a hadoop
credential provider or by adding the keys in the hive-site.xml in plain text
(unsecure)
This limits the usecase to using a single S3 key. If we configure per bucket s3
keys like described [here |
http://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html#Configurations_different_S3_buckets]
it exposes the access to all the buckets to all the hive users.
It is possible that there are different sets of users who would not like to
share there buckets and still be able to process the data using Hive. Enabling
session level credentials will help solve such use-cases. For example,
currently this doesn't work
{noformat}
set fs.s3a.secret.key=my_secret_key;
set fs.s3a.access.key=my_access.key;
{noformat}
Because metastore is unaware of the the keys. This doesn't work either
{noformat}
set fs.s3a.secret.key=my_secret_key;
set fs.s3a.access.key=my_access.key;
set metaconf:fs.s3a.secret.key=my_secret_key;
set metaconf:fs.s3a.access.key=my_access_key;
{noformat}
This is because only a certain metastore configurations defined in
{{HiveConf.MetaVars}} are allowed to be set by the user. If we enable the above
approaches we could potentially allow multiple S3 credentials on a per-session
level basis.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
