Loic, Answer would lie in log entries before the lines that report failure and show exception that you have produced below. Can you turn on debugging and post the log? Start by turning it on for com.xasecure.authorization.hive level. If cause does not jump out at you then turn it up to com.xasecure.
From: Loïc Chanel Reply-To: "user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>" Date: Thursday, July 30, 2015 at 3:01 AM To: "user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>" Subject: Re: Hive server identity assertion Sorry for my late answer, I had to work on a different problem. In the meantime, I realized that I am using anger 0.4, and not Ranger 0.5. So this problem may have been solved in Ranger 0.5. Here is all the the logs I get when my user toto tries to access chaneldb on which he should have permission to read because he belongs to the group sysadmin which has all the rights (including admin) on the database : 2015-07-30 11:50:49,891 INFO [HiveServer2-Handler-Pool: Thread-48]: parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: use chaneldb 2015-07-30 11:50:50,295 INFO [HiveServer2-Handler-Pool: Thread-48]: parse.ParseDriver (ParseDriver.java:parse(206)) - Parse Completed 2015-07-30 11:50:50,297 INFO [HiveServer2-Handler-Pool: Thread-48]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=parse start=1438249849885 end=1438249850297 duration=412 from=org.apache.hadoop.hive.ql.Driver> 2015-07-30 11:50:50,302 INFO [HiveServer2-Handler-Pool: Thread-48]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=semanticAnalyze from=org.apache.hadoop.hive.ql.Driver> 2015-07-30 11:50:50,347 INFO [HiveServer2-Handler-Pool: Thread-48]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(714)) - 2: get_database: chaneldb 2015-07-30 11:50:50,347 INFO [HiveServer2-Handler-Pool: Thread-48]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(340)) - ugi=toto ip=unknown-ip-addr cmd=get_database: chaneldb 2015-07-30 11:50:50,348 INFO [HiveServer2-Handler-Pool: Thread-48]: metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(557)) - 2: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2015-07-30 11:50:50,350 INFO [HiveServer2-Handler-Pool: Thread-48]: metastore.ObjectStore (ObjectStore.java:initialize(262)) - ObjectStore, initialize called 2015-07-30 11:50:50,371 INFO [HiveServer2-Handler-Pool: Thread-48]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>(131)) - Using direct SQL, underlying DB is MYSQL 2015-07-30 11:50:50,371 INFO [HiveServer2-Handler-Pool: Thread-48]: metastore.ObjectStore (ObjectStore.java:setConf(245)) - Initialized ObjectStore 2015-07-30 11:50:50,391 INFO [HiveServer2-Handler-Pool: Thread-48]: metadata.HiveUtils (HiveUtils.java:getMetaStoreAuthorizeProviderManagers(353)) - Adding metastore authorization provider: org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider 2015-07-30 11:50:50,395 INFO [HiveServer2-Handler-Pool: Thread-48]: metadata.HiveUtils (HiveUtils.java:getMetaStoreAuthorizeProviderManagers(353)) - Adding metastore authorization provider: org.apache.hadoop.hive.ql.security.authorization.MetaStoreAuthzAPIAuthorizerEmbedOnly 2015-07-30 11:50:50,427 INFO [HiveServer2-Handler-Pool: Thread-48]: ql.Driver (Driver.java:compile(429)) - Semantic Analysis Completed 2015-07-30 11:50:50,427 INFO [HiveServer2-Handler-Pool: Thread-48]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=semanticAnalyze start=1438249850302 end=1438249850427 duration=125 from=org.apache.hadoop.hive.ql.Driver> 2015-07-30 11:50:50,440 INFO [HiveServer2-Handler-Pool: Thread-48]: ql.Driver (Driver.java:getSchema(237)) - Returning Hive schema: Schema(fieldSchemas:null, properties:null) 2015-07-30 11:50:50,440 INFO [HiveServer2-Handler-Pool: Thread-48]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=doAuthorization from=org.apache.hadoop.hive.ql.Driver> 2015-07-30 11:50:50,486 INFO [HiveServer2-Handler-Pool: Thread-48]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=doAuthorization start=1438249850440 end=1438249850486 duration=46 from=org.apache.hadoop.hive.ql.Driver> ==> /var/log/hive/hive-server2.log <== FAILED: HiveAccessControlException Permission denied: user [toto] does not have [USE] privilege on [chaneldb] ==> /var/log/hive/hiveserver2.log <== 2015-07-30 11:50:50,487 ERROR [HiveServer2-Handler-Pool: Thread-48]: ql.Driver (SessionState.java:printError(833)) - FAILED: HiveAccessControlException Permission denied: user [toto] does not have [USE] privilege on [chaneldb] org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException: Permission denied: user [toto] does not have [USE] privilege on [chaneldb] at com.xasecure.authorization.hive.authorizer.XaSecureHiveAuthorizer.checkPrivileges(XaSecureHiveAuthorizer.java:254) at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:727) at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:520) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:457) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1069) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1063) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:109) at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:180) at org.apache.hive.service.cli.operation.Operation.run(Operation.java:256) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:376) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:363) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79) at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Unknown Source) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60) at com.sun.proxy.$Proxy28.executeStatementAsync(Unknown Source) at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:270) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:401) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) 2015-07-30 11:50:50,488 INFO [HiveServer2-Handler-Pool: Thread-48]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=compile start=1438249849844 end=1438249850488 duration=644 from=org.apache.hadoop.hive.ql.Driver> 2015-07-30 11:50:50,488 INFO [HiveServer2-Handler-Pool: Thread-48]: log.PerfLogger (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver> 2015-07-30 11:50:50,488 INFO [HiveServer2-Handler-Pool: Thread-48]: log.PerfLogger (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks start=1438249850488 end=1438249850488 duration=0 from=org.apache.hadoop.hive.ql.Driver> 2015-07-30 11:50:50,490 WARN [HiveServer2-Handler-Pool: Thread-48]: thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(407)) - Error executing statement: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [toto] does not have [USE] privilege on [chaneldb] at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:314) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:111) at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:180) at org.apache.hive.service.cli.operation.Operation.run(Operation.java:256) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:376) at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:363) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79) at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37) at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Unknown Source) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536) at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60) at com.sun.proxy.$Proxy28.executeStatementAsync(Unknown Source) at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:270) at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:401) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException: Permission denied: user [toto] does not have [USE] privilege on [chaneldb] at com.xasecure.authorization.hive.authorizer.XaSecureHiveAuthorizer.checkPrivileges(XaSecureHiveAuthorizer.java:254) at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:727) at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:520) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:457) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1069) at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1063) at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:109) ... 28 more And when I'm trying the command "groups" or even "hdfs groups" on the host running HiveServer I get "toto : nobody UsrSysAdmin SysAdmin ..." Do you or anyone else see where the problem might come from ? Thanks in advance, Loïc Loïc CHANEL Engineering student at TELECOM Nancy Trainee at Worldline - Villeurbanne 2015-07-24 17:30 GMT+02:00 Alok Lal <a...@hortonworks.com<mailto:a...@hortonworks.com>>: Perhaps. It is hard to say definitively without taking a look at the logs. From: Loïc Chanel <loic.cha...@telecomnancy.net<mailto:loic.cha...@telecomnancy.net>> Reply-To: "user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>" <user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>> Date: Friday, July 24, 2015 at 8:10 AM To: "user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>" <user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>> Subject: Re: Hive server identity assertion Exactly ! And I've checked the logs once again, but I can't see any groups mentioned. Does this reveal a special issue ? Thanks, Loïc Loïc CHANEL Engineering student at TELECOM Nancy Trainee at Worldline - Villeurbanne 2015-07-24 16:23 GMT+02:00 Alok Lal <a...@hortonworks.com<mailto:a...@hortonworks.com>>: If the user groups couldn't be asserted, would I see a log indicating that the user cannot be impersonated (like Knox prompts) ? Yes log should show the user and group info being sent to policy engine. For authorizing. I presume you are using ranger 0.5 to connect via beeline to a hiveserver2 instance. Right? (Not that these matter, just to set context.) Thanks From: Loïc Chanel <loic.cha...@telecomnancy.net<mailto:loic.cha...@telecomnancy.net>> Reply-To: "user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>" <user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>> Date: Friday, July 24, 2015 at 12:53 AM To: "user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>" <user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>> Subject: Re: Hive server identity assertion Well, that's what I thought, but the command hdfs groups returns me a group that I use for a policy giving access to a database, and as I get the message "HiveAccessControlException Permission denied" when accessing this database, I think Hive cannot assert the groups the user belongs to. I'm using Hive 0.14.0.2.2. As the problem might come from this, I think it's important to mention that the users are synchronized from a LDAP via SSSD. If the user groups couldn't be asserted, would I see a log indicating that the user cannot be impersonated (like Knox prompts) ? Thanks, Loïc Loïc CHANEL Engineering student at TELECOM Nancy Trainee at Worldline - Villeurbanne 2015-07-23 20:09 GMT+02:00 Don Bosco Durai <bo...@apache.org<mailto:bo...@apache.org>>: Hive uses the same core-site.xml settings as HDFS. So if the group mapping work in HDFS, then it should work in Hive also. And if the user and groups are in linux/unix, then it should have been support out of the box. What version of Hive are you using? (It shouldn’t matter) Thanks Bosco From: Loïc Chanel <loic.cha...@telecomnancy.net<mailto:loic.cha...@telecomnancy.net>> Reply-To: "user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>" <user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>> Date: Thursday, July 23, 2015 at 3:10 AM To: "user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>" <user@ranger.incubator.apache.org<mailto:user@ranger.incubator.apache.org>> Subject: Hive server identity assertion Hi all, As I am now exploring how Ranger works with Hive, I made some policies, but it seems that group policies are not enforced. Therefore, I was wondering how the Ranger plugin running on Hive was asserting the user's identity. I am even more surprised by the fact that I do not have any problem with Ranger plugin working on HDFS, which is running on the exact same node. In parallel, I know that Know plugin, for example, runs in a totally different way, but as it seems that, as does HBase, Hive does not provide with any user mapping function, I thought the identity would be asserted on the node Hive Server is running on, as if the user was a Unix one. Do someone as an idea about how the user groups can be founded by Hive Ranger plugin ? Thanks in advance, Loïc Loïc CHANEL Engineering student at TELECOM Nancy Trainee at Worldline - Villeurbanne
