Kalyan, I cherry-picked HIVE-17048 to all 2.x versions: 2.1.2, 2.2.1,2.3.1 and 2.4.0
They are not released yet, but as soon as one of them is released, then we can bump the Hive version of Sentry to the new version. - Sergio On Fri, Aug 11, 2017 at 12:30 PM, Kalyan Kumar Kalvagadda < kkal...@cloudera.com> wrote: > What I've heard from one of the Hive PMC is that they could get the fix to > Hive 2.4. That is the closest. I will follow-up with him again. > > Colm's suggestion is reasonable. Picking Hive 2.0.1 might be safer as it > might have fixes that were found in Hive 2.0.0 > > > > -Kalyan > > On Fri, Aug 11, 2017 at 2:32 AM, Colm O hEigeartaigh <cohei...@apache.org> > wrote: > > > It's unfortunate, but it seems like option 1 is the only solution really > > (although we should pick up Hive 2.0.1 at least). Perhaps the Hive > > community might be receptive to backport the fix to 2.3.x, 2.2.x, 2.1.x > > given that it is a simple fix and a blocker for Sentry? > > > > Colm. > > > > On Thu, Aug 10, 2017 at 11:21 PM, Mat Crocker <mat.croc...@cloudera.com> > > wrote: > > > > > My opinion (non-binding) is option-1. Keep it simple and follow HIVE. > > > > > > On Thu, Aug 10, 2017 at 5:17 PM, Kalyan Kumar Kalvagadda < > > > kkal...@cloudera.com> wrote: > > > > > > > Lina, > > > > > > > > I don't have that information. I need to dig deeper into apache hive > > wiki > > > > to get some understanding of it. > > > > > > > > -Kalyan > > > > > > > > -Kalyan > > > > > > > > On Wed, Aug 9, 2017 at 12:09 PM, Na Li <lina...@cloudera.com> wrote: > > > > > > > > > What is the timeline of back porting the fix to Hive 2.4? And when > do > > > we > > > > > plan to release sentry 2.0? > > > > > > > > > > On Wed, Aug 9, 2017 at 11:57 AM, Kalyan Kumar Kalvagadda < > > > > > kkal...@cloudera.com> wrote: > > > > > > > > > > > Hello all, > > > > > > > > > > > > I would like to share issue that i have discovered while working > on > > > > > > SENTRY-1838 <https://issues.apache.org/jira/browse/SENTRY-1838>( > > > > Support > > > > > > Hive 2.1.1 for sentry 2.0) > > > > > > > > > > > > With the design changes done in Hive between 2.0.0 through 2.1.1, > > > > > > HiveOperation > > > > > > is not passed to HiveSemanticAnalyzerHook which is implemented by > > > > > > sentry. HiveOperation > > > > > > is very basic information that sentry needs. > > > > > > > > > > > > I have reported this issue to hive community.(HIVE-17048 > > > > > > <https://issues.apache.org/jira/browse/HIVE-17048>). This issue > is > > > > fixed > > > > > > in > > > > > > Hive 3.0.0. I will work Hive community to back-port it to Hive > 2.4 > > > > > release. > > > > > > > > > > > > With that said, we can not integrate with Hive 2.1.1 because of > > above > > > > > > issue. > > > > > > > > > > > > We have two options here > > > > > > 1. Settle down to Hive 2.0.0 for sentry 2.0.0 release > > > > > > 2. Follow-up with Hive community and get this fix ported to Hive > > 2.4 > > > > > > release and integrate with hive 2.4. > > > > > > > > > > > > I give +1 to option-1 and target support for Hive 2.4 in sentry > > 2.0.1 > > > > > > release. > > > > > > > > > > > > > > > > > > -Kalyan > > > > > > > > > > > > On Wed, Jul 19, 2017 at 11:06 AM, Kalyan Kumar Kalvagadda < > > > > > > kkal...@cloudera.com> wrote: > > > > > > > > > > > > > Lina, > > > > > > > > > > > > > > Apache community is currently focused on sentry-ha to be > > released. > > > It > > > > > has > > > > > > > highest priority over others. If you are contributing to that, > I > > > > would > > > > > > say > > > > > > > continue doing so. > > > > > > > > > > > > > > -Kalyan > > > > > > > > > > > > > > On Wed, Jul 19, 2017 at 10:39 AM, Na Li <lina...@cloudera.com> > > > > wrote: > > > > > > > > > > > > > >> Kalyan, > > > > > > >> > > > > > > >> I can work on those items. > > > > > > >> > > > > > > >> Can you create seperate Jiras for each one and add more info. > > > > > > >> > > > > > > >> In terms of priority, is finishing sentry-HA a higher priority > > > than > > > > > > these > > > > > > >> (for C6)? > > > > > > >> > > > > > > >> Thanks, > > > > > > >> > > > > > > >> Lina > > > > > > >> > > > > > > >> On Wed, Jul 19, 2017 at 10:02 AM, Kalyan Kumar Kalvagadda < > > > > > > >> kkal...@cloudera.com> wrote: > > > > > > >> > > > > > > >> > Hello all, > > > > > > >> > > > > > > > >> > Either way we need to fix couple of things first. > > > > > > >> > > > > > > > >> > 1. Fix the test failures when hive-authz2 and datanucleus4 > > > > profiles > > > > > > are > > > > > > >> > enabled. > > > > > > >> > 2. Update e2e tests in sentry-tests-hive-v2 package to have > > same > > > > > > >> coverage > > > > > > >> > as tests in sentry-tests-hive package. > > > > > > >> > > > > > > > >> > Does any one wants to take ownership of any of these tasks? > > > > > > >> > > > > > > > >> > > > > > > > >> > -Kalyan > > > > > > >> > > > > > > > >> > On Tue, Jul 18, 2017 at 11:02 AM, Na Li < > lina...@cloudera.com > > > > > > > > wrote: > > > > > > >> > > > > > > > >> > > I agree with Colm too. We can support Hive 2.1.1 only in > > > sentry > > > > > 2.0 > > > > > > >> > > > > > > > > >> > > Lina > > > > > > >> > > > > > > > > >> > > On Tue, Jul 18, 2017 at 10:03 AM, Sergio Pena < > > > > > > >> sergio.p...@cloudera.com> > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > >> > > > I agree with Colm too. Hive 1.1 is too old and due to > the > > > way > > > > > how > > > > > > >> > Sentry > > > > > > >> > > > works with HMS and notifications, it's not worth the > > effort > > > to > > > > > > keep > > > > > > >> > > > supporting Hive 1.1. Let's drop it. > > > > > > >> > > > > > > > > > >> > > > On Tue, Jul 18, 2017 at 8:47 AM, Colm O hEigeartaigh < > > > > > > >> > > cohei...@apache.org> > > > > > > >> > > > wrote: > > > > > > >> > > > > > > > > > >> > > > > I think we should drop support for Hive 1.1.0 in > sentry > > > > 2.0.0. > > > > > > >> Hive > > > > > > >> > > 1.1.0 > > > > > > >> > > > > users can obtain the plugin from the Sentry 1.8.x > series > > > of > > > > > > >> releases. > > > > > > >> > > > > > > > > > > >> > > > > Colm. > > > > > > >> > > > > > > > > > > >> > > > > On Tue, Jul 18, 2017 at 1:18 PM, Kalyan Kumar > > Kalvagadda < > > > > > > >> > > > > kkal...@cloudera.com> wrote: > > > > > > >> > > > > > > > > > > >> > > > > > Hello all, > > > > > > >> > > > > > > > > > > > >> > > > > > Sentry currently supports Hive1.1.0 and Hive 2.0.0 > > based > > > > on > > > > > > the > > > > > > >> > > profile > > > > > > >> > > > > > used. I'm currently working on sentry 2.0.0 release > > > > upstream > > > > > > to > > > > > > >> > > > integrate > > > > > > >> > > > > > with Hive 2.1.1. > > > > > > >> > > > > > > > > > > > >> > > > > > I'm sending this mail seeking inputs on what is the > > > > support > > > > > we > > > > > > >> need > > > > > > >> > > in > > > > > > >> > > > > > Sentry 2.0.0 release. Would that be just Hive 2.x or > > > both > > > > > Hive > > > > > > >> 2.x > > > > > > >> > > and > > > > > > >> > > > > Hive > > > > > > >> > > > > > 1.1.0? > > > > > > >> > > > > > > > > > > > >> > > > > > I'm not sure if we should be supporting Hive 1.1.0 > in > > > > sentry > > > > > > >> 2.0.0. > > > > > > >> > > As > > > > > > >> > > > > Hive > > > > > > >> > > > > > 1.1.0 is pretty old and the notification log > > > > implementation > > > > > in > > > > > > >> this > > > > > > >> > > > > version > > > > > > >> > > > > > which Sentry-HA is dependent on has a lot of bugs > > which > > > > are > > > > > > >> fixed > > > > > > >> > in > > > > > > >> > > > > later > > > > > > >> > > > > > releases. Considering this, shouldn't we integrate > > > sentry > > > > in > > > > > > >> > release > > > > > > >> > > > > 2.0.0 > > > > > > >> > > > > > with Hive 2.1.1 and say that it would no more > support > > > > older > > > > > > >> > versions > > > > > > >> > > > of > > > > > > >> > > > > > Hive? > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > Here are some of the approaches that we could take > for > > > > Hive > > > > > > >> > > integration > > > > > > >> > > > > for > > > > > > >> > > > > > sentry 2.0.0 release. > > > > > > >> > > > > > > > > > > > >> > > > > > *Approach-1:* Maintain support for both Hive 1.1.0 > and > > > > Hive > > > > > > >> 2.1.1 > > > > > > >> > > > > > *Approach-2: *Maintain support for both Hive 1.1.0 > and > > > > Hive > > > > > > >> 2.1.1 > > > > > > >> > for > > > > > > >> > > > now > > > > > > >> > > > > > and remove it by the time sentry 2.0.0 is released. > > > > > > >> > > > > > *Approach-3:* Remove support for Hive 1.1.0 now. > > > > > > >> > > > > > > > > > > > >> > > > > > *Downside* for Approach 1 and 2 is that we need to > put > > > in > > > > > lot > > > > > > of > > > > > > >> > > effort > > > > > > >> > > > > to > > > > > > >> > > > > > have them work together as Hive 2.1.1 has a lot of > > API's > > > > > > changes > > > > > > >> > > which > > > > > > >> > > > > need > > > > > > >> > > > > > changes to a lot classes in sentry-binding package > but > > > > still > > > > > > >> > maintain > > > > > > >> > > > > > backward compatibility. > > > > > > >> > > > > > If we want to take approach-1 for any reason, it's > > worth > > > > > > putting > > > > > > >> > that > > > > > > >> > > > > > effort otherwise we could take approach-3. > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > My next steps would be based on what I hear form the > > > > > > community. > > > > > > >> > > > > > > > > > > > >> > > > > > -Kalyan > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > -- > > > > > > >> > > > > Colm O hEigeartaigh > > > > > > >> > > > > > > > > > > >> > > > > Talend Community Coder > > > > > > >> > > > > http://coders.talend.com > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Mat Crocker > > > Engineering Manager, Cloudera Inc. > > > > > > > > > > > -- > > Colm O hEigeartaigh > > > > Talend Community Coder > > http://coders.talend.com > > >
