Yan,
HDP-2.5.0 provides Hive-1.2.1 and a number of Apache patches – as detailed in
HDP documentation. The patch list includes HIVE-13418 and HIVE-13125. Please
refer to HDP documentation for more details.
Hope this helps.
Madhan
On 1/6/17, 8:18 AM, "Yan Zhou" <yzhouopensou...@gmail.com> wrote:
My impressions are 1) HDP package needs to be modified too if it is now
containing Hive 1.2 per its documentation; 2) as for the configuration, the
real issue is the binary compatibility of different versions of the Hive
libs. It might not be safe by just skipping a method call.
Thanks,
On Fri, Jan 6, 2017 at 12:19 AM, Madhan Neethiraj <mad...@apache.org> wrote:
> HiveAuthzContext.getForwardedAddress() was added in HIVE-13418
> <https://issues.apache.org/jira/browse/HIVE-13418>, which has been
> committed only in Hive-2.1.0. Hence you see the issue with Hive-1.2.1.
>
>
>
> Also, support for column-masking and row-filtering was added in HIVE-13125
> <https://issues.apache.org/jira/browse/HIVE-13125>, which is present only
> in Hive-2.1.0. Since Ranger 0.6 supports policies for column-masking and
> row-filtering, it would require Hive-2.1.0.
>
>
>
> Given above, I think the documentation should be updated to show that
> Ranger 0.6 requires Hive-2.1.0 or higher.
>
>
>
> If support for Hive-1.2.1 is required, I would suggest to try the
> following update to Ranger Hive plugin:
>
> - Introduce a configuration in Ranger Hive plugin (like
> ranger.plugin.hive.version-1.2.1); if this is set to true the plugin
> should not call HiveAuthzContext.getForwardedAddress()
>
>
>
> Please note however that even with the above update, column-masking and
> row-filter policies won’t be supported.
>
>
>
> Hope this helps.
>
>
>
> Madhan
>
>
>
>
>
> *From: *Yan Zhou <yzhouopensou...@gmail.com>
> *Reply-To: *"user@ranger.incubator.apache.org" <
> user@ranger.incubator.apache.org>
> *Date: *Thursday, January 5, 2017 at 10:50 PM
> *To: *"user@ranger.incubator.apache.org"
<user@ranger.incubator.apache.org>,
> "dev-subscr...@ranger.incubator.apache.org" <dev-subscribe@ranger.
> incubator.apache.org>
>
> *Subject: *Re: Ranger/Hive Version mismatch in HDP 2.5 ?
>
>
>
> When using Ranger 0.6.0 Hive Plugin with Hive 1.2.1, a beeline SQL command
> throws the following exception:
>
>
>
> Exception in thread "HiveServer2-Handler-Pool: Thread-37"
> java.lang.NoSuchMethodError: org.apache.hadoop.hive.ql.
> security.authorization.plugin.HiveAuthzContext.
> getForwardedAddresses()Ljava/util/List;
>
> at org.apache.ranger.authorization.hive.authorizer.
> RangerHiveAccessRequest.<init>(RangerHiveAccessRequest.java:55)
>
> at org.apache.ranger.authorization.hive.authorizer.
> RangerHiveAccessRequest.<init>(RangerHiveAccessRequest.java:72)
>
> at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.
> checkPrivileges(RangerHiveAuthorizer.java:263)
>
> at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:779)
>
> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:574)
>
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:468)
>
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
>
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
>
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116)
>
> at org.apache.hive.service.cli.operation.SQLOperation.
> prepare(SQLOperation.java:110)
>
> at org.apache.hive.service.cli.operation.SQLOperation.
> runInternal(SQLOperation.java:181)
>
> at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
>
> at org.apache.hive.service.cli.session.HiveSessionImpl.
> executeStatementInternal(HiveSessionImpl.java:388)
>
> at org.apache.hive.service.cli.session.HiveSessionImpl.
> executeStatementAsync(HiveSessionImpl.java:375)
>
> at org.apache.hive.service.cli.CLIService.executeStatementAsync(
> CLIService.java:274)
>
> at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(
> ThriftCLIService.java:486)
>
> at org.apache.hive.service.cli.thrift.TCLIService$Processor$
> ExecuteStatement.getResult(TCLIService.java:1313)
>
> at org.apache.hive.service.cli.thrift.TCLIService$Processor$
> ExecuteStatement.getResult(TCLIService.java:1298)
>
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>
> at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$
> TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692)
>
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(
> TThreadPoolServer.java:285)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
>
>
> The issue seems to be that HiveAuthzContext adds a
> getForwardedAddresses(...) method in Hive 2.1, which is called by Ranger
> 0.6 Hive Plugin
>
> due to Ranger-962.
>
>
>
> I'm using Ranger 0.6 and Hive built from git.
>
>
>
> Thanks.
>
>
>
> On Thu, Jan 5, 2017 at 1:39 PM, Madhan Neethiraj <mad...@apache.org>
> wrote:
>
> Yan,
>
> >> However Ranger 0.6.0 seems to work only with Hive 2.1.0
> Can you please add details of the issues found while using Ranger 0.6 with
> Hive 1.2?
>
> Thanks,
> Madhan
>
>
>
>
> On 1/5/17, 11:46 AM, "Don Bosco Durai" <bo...@apache.org> wrote:
>
> Yan, you should reach out to someone within Hortonworks to answer your
> HDP related question and their version numbering.
>
>
>
> Thanks
>
>
>
> Bosco
>
>
>
>
>
> From: Yan Zhou <yzhouopensou...@gmail.com>
> Reply-To: <user@ranger.incubator.apache.org>
> Date: Thursday, January 5, 2017 at 9:43 AM
> To: <user@ranger.incubator.apache.org>, <dev@ranger.incubator.apache.
> org>
> Subject: Re: Ranger/Hive Version mismatch in HDP 2.5 ?
>
>
>
> Can anyone help to clarify this version issue between Hive and Ranger
> 0.6.0 in HDP 2.5? I'm now in a dilemma where Ranger 0.6.0 forces me to use
> Hive 2.1 while HDP 2.5 only has Hive 1.2.1.
>
>
>
> Many Thanks!
>
>
>
> On Wed, Jan 4, 2017 at 1:08 PM, Yan Zhou <yzhouopensou...@gmail.com>
> wrote:
>
> Ranger's pom.xml has the Hive version as 2.1.0.
>
>
>
> Thanks
>
>
>
> On Wed, Jan 4, 2017 at 12:15 PM, Don Bosco Durai <bo...@apache.org>
> wrote:
>
> I feel we should have a matrix for the component versions supported by
> each Ranger release. Having it on the main page will be good.
>
>
>
> https://cwiki.apache.org/confluence/display/RANGER/Index
>
>
>
> One more thing I noticed is that the Installation and User guide is
> still pointing to release 0.5. If there are no changes, then we should we
> make a note of it some where.
>
>
>
> Thanks
>
>
>
> Bosco
>
>
>
>
>
> From: Yan Zhou <yzhouopensou...@gmail.com>
> Reply-To: <user@ranger.incubator.apache.org>
> Date: Wednesday, January 4, 2017 at 11:58 AM
> To: <user@ranger.incubator.apache.org>
> Subject: Ranger/Hive Version mismatch in HDP 2.5 ?
>
>
>
> According to https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/
> bk_release-notes/content/ch_relnotes_v250.html , HDP 2.5 contains Ranger
> 0.6.0 and Hive 1.2.1. However Ranger 0.6.0 seems to work only with Hive
> 2.1.0.
>
>
>
> Please clarify what versions of Ranger and Hive will work well with
> each other from HDP 2.5.
>
>
>
> Thanks.
>
>
>
>
>
>
>
>
>
