Hi,
Can you try appending the following string to the existing value of
hive.security.authorization.sqlstd.confwhitelist
|fs\.s3a\..*
And restart the HiveServer2 to see if this fixes this issue ?
Thanks,
Selva-
From: Anandha L Ranganathan <analog.s...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org>
Date: Monday, December 19, 2016 at 6:27 PM
To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org>
Subject: Re: Unable to connect to S3 after enabling Ranger with Hive
Selva,
Please find the results.
set hive.security.authorization.sqlstd.confwhitelist;
|
hive.security.authorization.sqlstd.confwhitelist=hive\.auto\..*|hive\.cbo\..*|hive\.convert\..*|hive\.exec\.dynamic\.partition.*|hive\.exec\..*\.dynamic\.partitions\..*|hive\.exec\.compress\..*|hive\.exec\.infer\..*|hive\.exec\.mode.local\..*|hive\.exec\.orc\..*|hive\.exec\.parallel.*|hive\.explain\..*|hive\.fetch.task\..*|hive\.groupby\..*|hive\.hbase\..*|hive\.index\..*|hive\.index\..*|hive\.intermediate\..*|hive\.join\..*|hive\.limit\..*|hive\.log\..*|hive\.mapjoin\..*|hive\.merge\..*|hive\.optimize\..*|hive\.orc\..*|hive\.outerjoin\..*|hive\.parquet\..*|hive\.ppd\..*|hive\.prewarm\..*|hive\.server2\.proxy\.user|hive\.skewjoin\..*|hive\.smbjoin\..*|hive\.stats\..*|hive\.tez\..*|hive\.vectorized\..*|mapred\.map\..*|mapred\.reduce\..*|mapred\.output\.compression\.codec|mapred\.job\.queuename|mapred\.output\.compression\.type|mapred\.min\.split\.size|mapreduce\.job\.reduce\.slowstart\.completedmaps|mapreduce\.job\.queuename|mapreduce\.job\.tags|mapreduce\.input\.fileinputformat\.sp
lit\.minsize|mapreduce\.map\..*|mapreduce\.reduce\..*|mapreduce\.output\.fileoutputformat\.compress\.codec|mapreduce\.output\.fileoutputformat\.compress\.type|tez\.am\..*|tez\.task\..*|tez\.runtime\..*|tez.queue.name|hive\.exec\.reducers\.bytes\.per\.reducer|hive\.client\.stats\.counters|hive\.exec\.default\.partition\.name|hive\.exec\.drop\.ignorenonexistent|hive\.counters\.group\.name|hive\.default\.fileformat\.managed|hive\.enforce\.bucketing|hive\.enforce\.bucketmapjoin|hive\.enforce\.sorting|hive\.enforce\.sortmergebucketmapjoin|hive\.cache\.expr\.evaluation|hive\.hashtable\.loadfactor|hive\.hashtable\.initialCapacity|hive\.ignore\.mapjoin\.hint|hive\.limit\.row\.max\.size|hive\.mapred\.mode|hive\.map\.aggr|hive\.compute\.query\.using\.stats|hive\.exec\.rowoffset|hive\.variable\.substitute|hive\.variable\.substitute\.depth|hive\.autogen\.columnalias\.prefix\.includefuncname|hive\.autogen\.columnalias\.prefix\.label|hive\.exec\.check\.crossproducts|hive\.compat|hive\.exec\.conca
tenate\.check\.index|hive\.display\.partition\.cols\.separately|hive\.error\.on\.empty\.partition|hive\.execution\.engine|hive\.exim\.uri\.scheme\.whitelist|hive\.file\.max\.footer|hive\.mapred\.supports\.subdirectories|hive\.insert\.into\.multilevel\.dirs|hive\.localize\.resource\.num\.wait\.attempts|hive\.multi\.insert\.move\.tasks\.share\.dependencies|hive\.support\.quoted\.identifiers|hive\.resultset\.use\.unique\.column\.names|hive\.analyze\.stmt\.collect\.partlevel\.stats|hive\.server2\.logging\.operation\.level|hive\.support\.sql11\.reserved\.keywords|hive\.exec\.job\.debug\.capture\.stacktraces|hive\.exec\.job\.debug\.timeout|hive\.exec\.max\.created\.files|hive\.exec\.reducers\.max|hive\.reorder\.nway\.joins|hive\.output\.file\.extension|hive\.exec\.show\.job\.failure\.debug\.info|hive\.exec\.tasklog\.debug\.timeout
|
0: jdbc:hive2://usw2dxdpmn01:10010> set
hive.security.authorization.sqlstd.confwhitelist.append;
+-----------------------------------------------------------------------+--+
| set |
+-----------------------------------------------------------------------+--+
| hive.security.authorization.sqlstd.confwhitelist.append is undefined |
+-----------------------------------------------------------------------+--+
On Mon, Dec 19, 2016 at 3:12 PM, Selvamohan Neethiraj <sneet...@apache.org>
wrote:
Hi,
Can you also post here the value for the following two parameters:
hive.security.authorization.sqlstd.confwhitelist
hive.security.authorization.sqlstd.confwhitelist.append
Thanks,
Selva-
From: Anandha L Ranganathan <analog.s...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org>
Date: Monday, December 19, 2016 at 5:54 PM
To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org>
Subject: Re: Unable to connect to S3 after enabling Ranger with Hive
Selva,
We are using HDP and here are versions and results.
Hive : 1.2.1.2.4
Ranger: 0.5.0.2.4
0: jdbc:hive2://usw2dxdpmn01:10010> set hive.conf.restricted.list;
+----------------------------------------------------------------------------------------------------------------------------------------+--+
| set
|
+----------------------------------------------------------------------------------------------------------------------------------------+--+
|
hive.conf.restricted.list=hive.security.authorization.enabled,hive.security.authorization.manager,hive.security.authenticator.manager
|
+----------------------------------------------------------------------------------------------------------------------------------------+--+
1 row selected (0.006 seconds)
0: jdbc:hive2://usw2dxdpmn01:10010> set hive.security.command.whitelist;
+-------------------------------------------------------------------------------+--+
| set
|
+-------------------------------------------------------------------------------+--+
| hive.security.command.whitelist=set,reset,dfs,add,list,delete,reload,compile
|
+-------------------------------------------------------------------------------+--+
1 row selected (0.008 seconds)
0: jdbc:hive2://usw2dxdpmn01:10010> set fs.s3a.access.key=xxxxxxxxxxxxxxx;
Error: Error while processing statement: Cannot modify fs.s3a.access.key at
runtime. It is not in list of params that are allowed to be modified at runtime
(state=42000,code=1)
On Mon, Dec 19, 2016 at 2:47 PM, Selvamohan Neethiraj <sneet...@apache.org>
wrote:
Hi,
Which version of Hive and Ranger are you using ? Can you check if Ranger has
added hiveserver2 parameters
hive.conf.restricted.list,hive.security.command.whitelist in the hive
configuration file(s) ?
Can you please list out these parameter values here ?
Thanks,
Selva-
From: Anandha L Ranganathan <analog.s...@gmail.com>
Reply-To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org>
Date: Monday, December 19, 2016 at 5:30 PM
To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org>
Subject: Unable to connect to S3 after enabling Ranger with Hive
Hi,
Unable to create table pointing to S3 after enabling Ranger.
This is database we created before enabling Ranger.
SET fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem;
SET fs.s3a.access.key=xxxxxxx;
SET fs.s3a.secret.key=yyyyyyyyyyyyyyy;
CREATE DATABASE IF NOT EXISTS backup_s3a1
COMMENT "s3a schema test"
LOCATION "s3a://gd-de-dp-db-hcat-backup-schema/";
After Ranger was enabled, we try to create another database but it is throwing
error.
0: jdbc:hive2://usw2dxdpmn01.local:> SET
fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem;
Error: Error while processing statement: Cannot modify fs.s3a.impl at runtime.
It is not in list of params that are allowed to be modified at runtime
(state=42000,code=1)
I configured the credentials in the core-site.xml and always returns
"undefined" when I am trying to see the values for below commands. This is in
our " dev" environment where Ranger is enabled. In other environment where
Ranger is not installed , we are not facing this problem.
0: jdbc:hive2://usw2dxdpmn01:10010> set fs.s3a.impl;
+-----------------------------------------------------+--+
| set |
+-----------------------------------------------------+--+
| fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem |
+-----------------------------------------------------+--+
1 row selected (0.006 seconds)
0: jdbc:hive2://usw2dxdpmn01:10010> set fs.s3a.access.key;
+---------------------------------+--+
| set |
+---------------------------------+--+
| fs.s3a.access.key is undefined |
+---------------------------------+--+
1 row selected (0.005 seconds)
0: jdbc:hive2://usw2dxdpmn01:10010> set fs.s3a.secret.key;
+---------------------------------+--+
| set |
+---------------------------------+--+
| fs.s3a.secret.key is undefined |
+---------------------------------+--+
1 row selected (0.005 seconds)
Any help or pointers is appreciated.
