Review Request 69924: SENTRY-2440: Add a new thrift API for checking if a user is in admin group

[Hao Hao via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69924/
---

Review request for sentry, kalyan kumar kalvagadda and Na Li.


Repository: sentry


Description
---

SENTRY-2440: Add a new thrift API for checking if a user is in admin group to 
check if a given user is in the Sentry admin group or not.

This is useful for Sentry client to recognize failure earlier than actually 
making a call to privileged API such as 'create_role', 'drop_role'.

This API shouldn't leak any sensitive information because connection to teh 
Sentry server is guarded by 'sentry.service.allow.connect' config, that only 
the trusted service users can connect to the Sentry service.


Diffs
-

  
sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
 0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
  
sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/TIsSentryAdminRequest.java
 PRE-CREATION 
  
sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/TIsSentryAdminResponse.java
 PRE-CREATION 
  
sentry-service/sentry-service-api/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyServiceClient.java
 5fc299b2485e0af6df333e4a288299f39e18b3b7 
  
sentry-service/sentry-service-api/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyServiceClientDefaultImpl.java
 68d864cfbdf18057d87a65a04af8991292aadccf 
  
sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
 2e79e5646ae9102d8c0c28da4260a539254fcd15 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
 236a07bdf5191cdc0f167f20a406b721b3dc506d 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
 30875299ebf81e74a78b396d4aeaf27890083370 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/api/service/thrift/TestSentryServiceIntegration.java
 bfafa7d99735bec07bd81ebe665f4e84e65bd3b7 


Diff: https://reviews.apache.org/r/69924/diff/1/


Testing
---

Unit test.


Thanks,

Hao Hao

Re: Review Request 69536: SENTRY-2471: Table rename should sync Sentry privilege even without location information

[Hao Hao via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69536/
---

(Updated Feb. 7, 2019, 1:04 a.m.)


Review request for sentry, Na Li and Sergio Pena.


Repository: sentry


Description
---

SENTRY-2471: Table rename should sync Sentry privilege even without location 
information


Diffs (updated)
-

  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/NotificationProcessor.java
 7b7d0e1eb7ea1f17dea622e51ccc557e0b76fbff 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestNotificationProcessor.java
 f227bb45835b98e6fef14399a239d7f9975d56d1 


Diff: https://reviews.apache.org/r/69536/diff/3/

Changes: https://reviews.apache.org/r/69536/diff/2-3/


Testing (updated)
---

added new unit tests


Thanks,

Hao Hao

Re: Review Request 69536: SENTRY-2471: Table rename should sync Sentry privilege even without location information

[Hao Hao via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69536/
---

(Updated Dec. 10, 2018, 2:21 a.m.)


Review request for sentry, Na Li and Sergio Pena.


Repository: sentry


Description
---

SENTRY-2471: Table rename should sync Sentry privilege even without location 
information


Diffs (updated)
-

  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/NotificationProcessor.java
 7b7d0e1eb7ea1f17dea622e51ccc557e0b76fbff 


Diff: https://reviews.apache.org/r/69536/diff/2/

Changes: https://reviews.apache.org/r/69536/diff/1-2/


Testing
---

existing unit tests


Thanks,

Hao Hao

Review Request 69536: SENTRY-2471: Table rename should sync Sentry privilege even without location information

[Hao Hao via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69536/
---

Review request for sentry, Na Li and Sergio Pena.


Repository: sentry


Description
---

SENTRY-2471: Table rename should sync Sentry privilege even without location 
information


Diffs
-

  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/NotificationProcessor.java
 7b7d0e1eb7ea1f17dea622e51ccc557e0b76fbff 


Diff: https://reviews.apache.org/r/69536/diff/1/


Testing
---

existing unit tests


Thanks,

Hao Hao

Re: [VOTE] Release Sentry version 2.1.0

[Hao Hao]

+1 Thanks Lina!

Best,
Hao

On Fri, Sep 28, 2018 at 8:31 AM Kalyan Kumar Kalvagadda
 wrote:

> Looks Good.
>
> +1 on the release of 2.1.0
>
>
>
> *Thanks,Kalyan Kumar Kalvagadda* | Software Engineer
> t. (469) 279- <00>5732
> cloudera.com 
>
> [image: Cloudera] 
>
> [image: Cloudera on Twitter]  [image:
> Cloudera on Facebook]  [image: Cloudera
> on LinkedIn] 
> --
>
>
> On Fri, Sep 28, 2018 at 10:21 AM Sergio Pena
>  wrote:
>
> > Thanks Lina.
> >
> > I verified the release yesterday, and I checked the hashes today.
> > The release looks good.
> >
> > +1
> >
> > On Fri, Sep 28, 2018 at 10:04 AM Na Li 
> > wrote:
> >
> > > Hello,
> > >
> > > 1) Can you cast your vote for releasing Sentry 2.1.0? The vote will be
> > > closed this afternoon at 2pm.
> > >
> > > 2) I have updated the following files
> > > apache-sentry-2.1.0-src.tar.gz.sha512
> > > apache-sentry-2.1.0-bin.tar.gz.sha512
> > >
> > > Their content was generated using command "*gpg --print-md SHA512
> > >  >  .sha512*"
> > > The updated content was generated using command "*sha512sum 
> > > >  .sha512*"
> > > The value remains the same, but the display format is different.
> > >
> > > Thanks,
> > >
> > > Lina
> > >
> > > On Thu, Sep 27, 2018 at 10:45 AM Na Li  wrote:
> > >
> > > > Sergio,
> > > >
> > > > It is at https://dist.apache.org/repos/dist/dev/sentry/2.1.0/
> > > >
> > > > Sorry, I did not list it in previous email.
> > > >
> > > > Thanks,
> > > >
> > > > Lina
> > > >
> > > > On Thu, Sep 27, 2018 at 9:20 AM Sergio Pena
> > > >  wrote:
> > > >
> > > >> Lina, where are the .tar.gz packages for 2.1.0 located?
> > > >>
> > > >> - Sergio
> > > >>
> > > >> On Tue, Sep 25, 2018 at 1:24 PM Na Li  >
> > > >> wrote:
> > > >>
> > > >> > This is the release of Apache Sentry, version 2.1.0.
> > > >> > It fixes the following issues:
> > > >> >
> > > >> > https://issues.apache.org/jira/projects/SENTRY/versions/12342213
> > > >> >
> > > >> >
> > > >> > Maven artifacts are available
> > > >> >
> > > >> >
> > > >>
> > >
> https://repository.apache.org/content/repositories/orgapachesentry-1015/
> > > >> >
> > > >> > Tag to be voted
> > > >> > onhttps://
> > > >> >
> > > >>
> > >
> >
> git-wip-us.apache.org/repos/asf/sentry/?p=sentry.git;a=tag;h=refs/tags/release-2.1.0
> > > >> >
> > > >> >
> > > >> > Sentry's KEYS containing the PGP key we used to sign the release:
> > > >> > http://www.apache.org/dist/sentry/KEYS
> > > >> >
> > > >> >  We are voting on the source:tag=release-2.1.0, SHA=
> > > >> >
> > > >> > c5af6f0b81481b640b29d6ed9dde2ad1a399e882 (You can get the hash of
> > the
> > > >> > tag by doing
> > > >> >
> > > >> >  "git rev-list release-2.1.0 | head -n 1" )
> > > >> >
> > > >> >
> > > >> > Vote will be open for 72 hours.
> > > >> >
> > > >> > [ ] +1 approve
> > > >> > [ ] +0 no opinion
> > > >> > [ ] -1 disapprove (and reason why)
> > > >> >
> > > >> > Thanks,
> > > >> >
> > > >> >
> > > >> > Lina
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

> On Sept. 25, 2018, 11:22 p.m., Na Li wrote:
> > sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
> > Lines 419 (patched)
> > <https://reviews.apache.org/r/68822/diff/3/?file=2092016#file2092016line419>
> >
> > I heard today that it is possible to assign user to a role in near 
> > future. So can you change the function name to 
> > "list_sentry_privileges_by_user_and_itsroles"? it will cover privileges
> > 
> > 1) assigned to user directly
> > 2) assigned to roles directly associated to a user
> > 3) assigned to roles directly associated to groups that user belongs to.
> > 
> > Then when that feature of allowing user to be associated to a role is 
> > implemented, we don't need to change this API. And the intention of this 
> > function is to get all privileges for a user directly and indirectly
> 
> Hao Hao wrote:
> Hmm, actually I think list_sentry_privileges_by_user_and_itsroles may be 
> more misleading. Since people may think the API only lists privileges granted 
> to the roles directly associated to that user.

Lina and I discussed offline, and we both think the current naming is good 
enough as the reason stated above.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/#review209007
---


On Sept. 25, 2018, 9:23 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68822/
> ---
> 
> (Updated Sept. 25, 2018, 9:23 p.m.)
> 
> 
> Review request for sentry, Na Li and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> This commit adds a new thrift API list_sentry_privileges_for_user to List 
> sentry privileges granted to the given user and the group the user associated 
> with, filterted based on authorization hierarchy if present.
> Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.
> 
> 
> Diffs
> -
> 
>   
> sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
>  0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
>   
> sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
>  2e79e5646ae9102d8c0c28da4260a539254fcd15 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
>  236a07bdf5191cdc0f167f20a406b721b3dc506d 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
>  3a9623b46f7c4335db18113574170f761da9a4ca 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1eda41b2b6bd940a404cc1ba09a861fe783ead04 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
>  0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
> 
> 
> Diff: https://reviews.apache.org/r/68822/diff/3/
> 
> 
> Testing
> ---
> 
> Unit test.
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

> On Sept. 25, 2018, 11:22 p.m., Na Li wrote:
> > sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
> > Lines 419 (patched)
> > <https://reviews.apache.org/r/68822/diff/3/?file=2092016#file2092016line419>
> >
> > I heard today that it is possible to assign user to a role in near 
> > future. So can you change the function name to 
> > "list_sentry_privileges_by_user_and_itsroles"? it will cover privileges
> > 
> > 1) assigned to user directly
> > 2) assigned to roles directly associated to a user
> > 3) assigned to roles directly associated to groups that user belongs to.
> > 
> > Then when that feature of allowing user to be associated to a role is 
> > implemented, we don't need to change this API. And the intention of this 
> > function is to get all privileges for a user directly and indirectly

Hmm, actually I think list_sentry_privileges_by_user_and_itsroles may be more 
misleading. Since people may think the API only lists privileges granted to the 
roles directly associated to that user.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/#review209007
---


On Sept. 25, 2018, 9:23 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68822/
> ---
> 
> (Updated Sept. 25, 2018, 9:23 p.m.)
> 
> 
> Review request for sentry, Na Li and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> This commit adds a new thrift API list_sentry_privileges_for_user to List 
> sentry privileges granted to the given user and the group the user associated 
> with, filterted based on authorization hierarchy if present.
> Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.
> 
> 
> Diffs
> -
> 
>   
> sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
>  0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
>   
> sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
>  2e79e5646ae9102d8c0c28da4260a539254fcd15 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
>  236a07bdf5191cdc0f167f20a406b721b3dc506d 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
>  3a9623b46f7c4335db18113574170f761da9a4ca 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1eda41b2b6bd940a404cc1ba09a861fe783ead04 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
>  0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
> 
> 
> Diff: https://reviews.apache.org/r/68822/diff/3/
> 
> 
> Testing
> ---
> 
> Unit test.
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

> On Sept. 24, 2018, 11:23 p.m., Na Li wrote:
> > sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
> > Lines 915 (patched)
> > <https://reviews.apache.org/r/68822/diff/2/?file=2091812#file2091812line915>
> >
> > you have got groups in line 908 if the requestor is the same as 
> > principlaName. It should improve performance to reuse it under this 
> > situation
> 
> Hao Hao wrote:
> I don't think the requestor is the same as principal? Here it is getting 
> the principal's group.

Hmm, misinterpret your suggestion, updating it.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/#review208969
-----------


On Sept. 25, 2018, 9:23 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68822/
> ---
> 
> (Updated Sept. 25, 2018, 9:23 p.m.)
> 
> 
> Review request for sentry, Na Li and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> This commit adds a new thrift API list_sentry_privileges_for_user to List 
> sentry privileges granted to the given user and the group the user associated 
> with, filterted based on authorization hierarchy if present.
> Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.
> 
> 
> Diffs
> -
> 
>   
> sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
>  0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
>   
> sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
>  2e79e5646ae9102d8c0c28da4260a539254fcd15 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
>  236a07bdf5191cdc0f167f20a406b721b3dc506d 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
>  3a9623b46f7c4335db18113574170f761da9a4ca 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1eda41b2b6bd940a404cc1ba09a861fe783ead04 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
>  0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
> 
> 
> Diff: https://reviews.apache.org/r/68822/diff/3/
> 
> 
> Testing
> ---
> 
> Unit test.
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/
---

(Updated Sept. 25, 2018, 9:23 p.m.)


Review request for sentry, Na Li and Sergio Pena.


Repository: sentry


Description
---

This commit adds a new thrift API list_sentry_privileges_for_user to List 
sentry privileges granted to the given user and the group the user associated 
with, filterted based on authorization hierarchy if present.
Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.


Diffs (updated)
-

  
sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
 0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
  
sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
 2e79e5646ae9102d8c0c28da4260a539254fcd15 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
 236a07bdf5191cdc0f167f20a406b721b3dc506d 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
 3a9623b46f7c4335db18113574170f761da9a4ca 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 1eda41b2b6bd940a404cc1ba09a861fe783ead04 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
 0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 


Diff: https://reviews.apache.org/r/68822/diff/3/

Changes: https://reviews.apache.org/r/68822/diff/2-3/


Testing
---

Unit test.


Thanks,

Hao Hao

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/#review208972
---




sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
Line 227 (original), 229 (patched)
<https://reviews.apache.org/r/68822/#comment293246>

I was hesitate to add another API to complicate the code base. Also I do 
not see any other consumer of this API other than SentryPolicyStoreProcessor so 
I made change on top of the existing one. But if you prefer adding a new one, I 
will update this.


- Hao Hao


On Sept. 24, 2018, 9:31 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68822/
> ---
> 
> (Updated Sept. 24, 2018, 9:31 p.m.)
> 
> 
> Review request for sentry, Na Li and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> This commit adds a new thrift API list_sentry_privileges_for_user to List 
> sentry privileges granted to the given user and the group the user associated 
> with, filterted based on authorization hierarchy if present.
> Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.
> 
> 
> Diffs
> -
> 
>   
> sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
>  0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
>   
> sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
>  2e79e5646ae9102d8c0c28da4260a539254fcd15 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
>  236a07bdf5191cdc0f167f20a406b721b3dc506d 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
>  3a9623b46f7c4335db18113574170f761da9a4ca 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1eda41b2b6bd940a404cc1ba09a861fe783ead04 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
>  0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/api/service/thrift/TestTSentryPrivilegeToAuthorizable.java
>  PRE-CREATION 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStoreToAuthorizable.java
>  25f94fa05b05abf8c1dbc33e97e5e88ae01794e4 
> 
> 
> Diff: https://reviews.apache.org/r/68822/diff/2/
> 
> 
> Testing
> ---
> 
> Unit test.
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

> On Sept. 24, 2018, 11:23 p.m., Na Li wrote:
> > sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
> > Lines 915 (patched)
> > <https://reviews.apache.org/r/68822/diff/2/?file=2091812#file2091812line915>
> >
> > you have got groups in line 908 if the requestor is the same as 
> > principlaName. It should improve performance to reuse it under this 
> > situation

I don't think the requestor is the same as principal? Here it is getting the 
principal's group.


> On Sept. 24, 2018, 11:23 p.m., Na Li wrote:
> > sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
> > Line 227 (original), 229 (patched)
> > <https://reviews.apache.org/r/68822/diff/2/?file=2091814#file2091814line229>
> >
> > This is semi-public API. It is safer to add function than change the 
> > returned type of existing function. Inside implementation, you can let one 
> > function call another one to re-use code.

I was hesitate to add another API to complicate the code base. Also I do not 
see any other consumer of this API other than SentryPolicyStoreProcessor so I 
made change on top of the existing one. But if you prefer to add a new one to 
avoid unknown dependency, I will update this.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/#review208969
---


On Sept. 24, 2018, 9:31 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68822/
> ---
> 
> (Updated Sept. 24, 2018, 9:31 p.m.)
> 
> 
> Review request for sentry, Na Li and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> This commit adds a new thrift API list_sentry_privileges_for_user to List 
> sentry privileges granted to the given user and the group the user associated 
> with, filterted based on authorization hierarchy if present.
> Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.
> 
> 
> Diffs
> -
> 
>   
> sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
>  0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
>   
> sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
>  2e79e5646ae9102d8c0c28da4260a539254fcd15 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
>  236a07bdf5191cdc0f167f20a406b721b3dc506d 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
>  3a9623b46f7c4335db18113574170f761da9a4ca 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1eda41b2b6bd940a404cc1ba09a861fe783ead04 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
>  0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/api/service/thrift/TestTSentryPrivilegeToAuthorizable.java
>  PRE-CREATION 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStoreToAuthorizable.java
>  25f94fa05b05abf8c1dbc33e97e5e88ae01794e4 
> 
> 
> Diff: https://reviews.apache.org/r/68822/diff/2/
> 
> 
> Testing
> ---
> 
> Unit test.
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

> On Sept. 24, 2018, 3:35 p.m., Sergio Pena wrote:
> > Btw, are you going to have another JIRA for the client to call this new API?
> 
> Hao Hao wrote:
> Sure, I can create a jira for adding the client side logical.

SENTRY-2418 is filed.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/#review208950
---


On Sept. 24, 2018, 9:31 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68822/
> ---
> 
> (Updated Sept. 24, 2018, 9:31 p.m.)
> 
> 
> Review request for sentry, Na Li and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> This commit adds a new thrift API list_sentry_privileges_for_user to List 
> sentry privileges granted to the given user and the group the user associated 
> with, filterted based on authorization hierarchy if present.
> Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.
> 
> 
> Diffs
> -
> 
>   
> sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
>  0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
>   
> sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
>  2e79e5646ae9102d8c0c28da4260a539254fcd15 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
>  236a07bdf5191cdc0f167f20a406b721b3dc506d 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
>  3a9623b46f7c4335db18113574170f761da9a4ca 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1eda41b2b6bd940a404cc1ba09a861fe783ead04 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
>  0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/api/service/thrift/TestTSentryPrivilegeToAuthorizable.java
>  PRE-CREATION 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStoreToAuthorizable.java
>  25f94fa05b05abf8c1dbc33e97e5e88ae01794e4 
> 
> 
> Diff: https://reviews.apache.org/r/68822/diff/2/
> 
> 
> Testing
> ---
> 
> Unit test.
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/
---

(Updated Sept. 24, 2018, 9:31 p.m.)


Review request for sentry, Na Li and Sergio Pena.


Repository: sentry


Description
---

This commit adds a new thrift API list_sentry_privileges_for_user to List 
sentry privileges granted to the given user and the group the user associated 
with, filterted based on authorization hierarchy if present.
Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.


Diffs (updated)
-

  
sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
 0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
  
sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
 2e79e5646ae9102d8c0c28da4260a539254fcd15 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
 236a07bdf5191cdc0f167f20a406b721b3dc506d 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
 3a9623b46f7c4335db18113574170f761da9a4ca 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 1eda41b2b6bd940a404cc1ba09a861fe783ead04 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
 0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/api/service/thrift/TestTSentryPrivilegeToAuthorizable.java
 PRE-CREATION 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStoreToAuthorizable.java
 25f94fa05b05abf8c1dbc33e97e5e88ae01794e4 


Diff: https://reviews.apache.org/r/68822/diff/2/

Changes: https://reviews.apache.org/r/68822/diff/1-2/


Testing
---

Unit test.


Thanks,

Hao Hao

Re: Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

> On Sept. 24, 2018, 3:35 p.m., Sergio Pena wrote:
> > Btw, are you going to have another JIRA for the client to call this new API?

Sure, I can create a jira for adding the client side logical.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/#review208950
---


On Sept. 23, 2018, 8:52 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68822/
> ---
> 
> (Updated Sept. 23, 2018, 8:52 p.m.)
> 
> 
> Review request for sentry, Na Li and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> This commit adds a new thrift API list_sentry_privileges_for_user to List 
> sentry privileges granted to the given user and the group the user associated 
> with, filterted based on authorization hierarchy if present.
> Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.
> 
> 
> Diffs
> -
> 
>   
> sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
>  0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
>   
> sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
>  2e79e5646ae9102d8c0c28da4260a539254fcd15 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
>  236a07bdf5191cdc0f167f20a406b721b3dc506d 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
>  3a9623b46f7c4335db18113574170f761da9a4ca 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  1eda41b2b6bd940a404cc1ba09a861fe783ead04 
>   
> sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
>  0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/api/service/thrift/TestTSentryPrivilegeToAuthorizable.java
>  PRE-CREATION 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
>   
> sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStoreToAuthorizable.java
>  25f94fa05b05abf8c1dbc33e97e5e88ae01794e4 
> 
> 
> Diff: https://reviews.apache.org/r/68822/diff/1/
> 
> 
> Testing
> ---
> 
> Unit test.
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Review Request 68822: SENTRY-2371 Add a new thrift API for getting all privileges a user has

[Hao Hao via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68822/
---

Review request for sentry, Na Li and Sergio Pena.


Repository: sentry


Description
---

This commit adds a new thrift API list_sentry_privileges_for_user to List 
sentry privileges granted to the given user and the group the user associated 
with, filterted based on authorization hierarchy if present.
Under the hood, this API is using sentryStore.listSentryPrivilegesForProvider.


Diffs
-

  
sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/SentryPolicyService.java
 0cbd8ab0a624d4c09aead4097f72762e12d1d21b 
  
sentry-service/sentry-service-api/src/main/resources/sentry_policy_service.thrift
 2e79e5646ae9102d8c0c28da4260a539254fcd15 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryMetrics.java
 236a07bdf5191cdc0f167f20a406b721b3dc506d 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/api/service/thrift/SentryPolicyStoreProcessor.java
 3a9623b46f7c4335db18113574170f761da9a4ca 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 1eda41b2b6bd940a404cc1ba09a861fe783ead04 
  
sentry-service/sentry-service-server/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreInterface.java
 0b4f4aa24bd3002c50bf4d80a6fa361f66052973 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/api/service/thrift/TestTSentryPrivilegeToAuthorizable.java
 PRE-CREATION 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 d8c0ab4fa82ba09c60bc995eb4f53a78a7fae346 
  
sentry-service/sentry-service-server/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStoreToAuthorizable.java
 25f94fa05b05abf8c1dbc33e97e5e88ae01794e4 


Diff: https://reviews.apache.org/r/68822/diff/1/


Testing
---

Unit test.


Thanks,

Hao Hao

Re: Please welcome Kalyan Kalvagadda to Apache PMC!

[Hao Hao]

Congrats!

Best,
Hao

On Mon, Feb 5, 2018 at 3:18 PM, Stephen Moist  wrote:

> Congrats!
>
> > On Feb 5, 2018, at 12:33 PM, Na Li  wrote:
> >
> > Kalyan, Congratulation!
> >
> > On Mon, Feb 5, 2018 at 2:18 PM, Sergio Pena 
> > wrote:
> >
> >> Congratulations Kalyan !!!
> >>
> >> On Mon, Feb 5, 2018 at 2:16 PM, Xinran Yu Tinney <
> yuxinran8...@gmail.com>
> >> wrote:
> >>
> >>> Congratulations!
> >>>
> >>> 2018-02-05 14:14 GMT-06:00 Alexander Kolbasov :
> >>>
>  Please welcome Kalyan Kalvagadda who is joining Apache Sentry PMC!
> 
>  - Alex
> 
> >>>
> >>
>
>

Re: DISCUSSION: Prepare master for next release (2.0.0)

[Hao Hao]

+1

Best,
Hao

On Thu, Jul 27, 2017 at 1:38 AM, Colm O hEigeartaigh 
wrote:

> +1.
>
> Colm.
>
> On Wed, Jul 26, 2017 at 10:45 PM, Vamsee Yarlagadda 
> wrote:
>
> > +1
> >
> > Thanks,
> > Vamsee
> >
> > On Wed, Jul 26, 2017 at 2:11 PM, Na Li  wrote:
> >
> > > Sergio,
> > >
> > > I agree we should do it soon. Can you send out a notice right before
> it?
> > So
> > > we will submit the changes to master instead of sentry-ha-redesign for
> > 2.0
> > >
> > > Thanks,
> > >
> > > Lina
> > >
> > > On Wed, Jul 26, 2017 at 3:57 PM, Sergio Pena  >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Now that branch-1.8 has been created, we need to prepare the current
> > > master
> > > > branch for the next major release. However, the current master won't
> be
> > > > used for that, and the new sentry-ha-redesign branch will become
> > master.
> > > >
> > > > We'd like to do this as soon as possible. This change will be
> > disruptive,
> > > > and all contributors will have to update their master with the new
> one.
> > > >
> > > > Can we do this today or tomorrow? Anytime no later than Friday so
> that
> > we
> > > > have the most updated master to the community.
> > > >
> > > > - Sergio
> > > >
> > >
> >
> >
> >
> > --
> > Thanks,
> > Vamsee
> >
>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>

Re: Requesting to add 'viktorsomogyi' as contributor

[Hao Hao]

Hi Viktor,

I have added you as a contributor.

Best,
Hao

On Tue, May 30, 2017 at 7:24 AM, Viktor Somogyi  wrote:

> Hi folks,
>
> I would like to assign some jiras to myself and contribute to Sentry.
> Could someone please add me as contributor so I can start doing it?
>
> My apache account is: viktorsomogyi
>
> Thanks,
> Viktor
>

Re: Review Request 57375: Unable to truncate table .; from "default" databases

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57375/#review175035
---


Fix it, then Ship it!





sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java
Lines 255 (patched)
<https://reviews.apache.org/r/57375/#comment248351>

nit: // If



sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java
Lines 259 (patched)
<https://reviews.apache.org/r/57375/#comment248352>

Same here


- Hao Hao


On March 16, 2017, 2:41 p.m., Yongzhi Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57375/
> ---
> 
> (Updated March 16, 2017, 2:41 p.m.)
> 
> 
> Review request for sentry and Hao Hao.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Please review fix for SENTRY-1646. Fix the issue by getting the currOutDB 
> inform from qualified table name if exists.
> 
> 
> Diffs
> -
> 
>   
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java
>  d269d5fe0bfd3e8638fcec15d426862f4f15ae35 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestPrivilegesAtTableScopePart2.java
>  8eb0bd69158805a3fbcb6c88d61b26ef922997ef 
> 
> 
> Diff: https://reviews.apache.org/r/57375/diff/2/
> 
> 
> Testing
> ---
> 
> Add the test case in Unit test.
> 
> 
> Thanks,
> 
> Yongzhi Chen
> 
>

Re: Review Request 58975: SENTRY-1669 HMSFollower should read current processed notification ID from database every time it runs

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58975/#review174830
---


Ship it!




Ship It!

- Hao Hao


On May 12, 2017, 5:38 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58975/
> ---
> 
> (Updated May 12, 2017, 5:38 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, 
> Vamsee Yarlagadda, and Vadim Spector.
> 
> 
> Bugs: SENTRY-1669
> https://issues.apache.org/jira/browse/SENTRY-1669
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Sentry should store Id's of all the notifications that it processed in a 
> seperate database.
> HMSFollower before every run should get the last notification ID from this 
> persistent store.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryHmsNotification.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  8fd5278 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/DeltaTransactionBlock.java
>  f590a52 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  ef67865 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  5e6b906 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  e7443eb 
> 
> 
> Diff: https://reviews.apache.org/r/58975/diff/10/
> 
> 
> Testing
> ---
> 
> Sentry stores Id's of all the notifications that it processed in a seperate 
> database.
> HMSFollower before every run gets the last notification ID from this 
> persistent store.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58808: SENTRY-1726 sql changes to store last notification-id processed

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58808/#review174749
---


Ship it!




Ship It!

- Hao Hao


On May 9, 2017, 11:15 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58808/
> ---
> 
> (Updated May 9, 2017, 11:15 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, 
> Vamsee Yarlagadda, and Vadim Spector.
> 
> 
> Bugs: SENTRY-1726
> https://issues.apache.org/jira/browse/SENTRY-1726
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> 1. Created new table to store the last notification id
> 2. Table will store the notification id's of all the notifications processed 
> by sentry server.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.derby.sql
>  ba70715 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.mysql.sql
>  879e732 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.oracle.sql
>  e83ab83 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.postgres.sql
>  c28099b 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 841dcaa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 223835e 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 006d57b 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 20921ea 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  489ad66 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  ec50912 
> 
> 
> Diff: https://reviews.apache.org/r/58808/diff/2/
> 
> 
> Testing
> ---
> 
> Tested with the databases that sentry supports.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58975: SENTRY-1669 HMSFollower should read current processed notification ID from database every time it runs

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58975/#review174744
---




sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Lines 3420 (patched)
<https://reviews.apache.org/r/58975/#comment247954>

Name it to sth. like persistLatestNotificationID to make it more clear.



sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
Lines 599 (patched)
<https://reviews.apache.org/r/58975/#comment247956>

Should we persist lastest notification id other wise?



sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
Lines 600 (patched)
<https://reviews.apache.org/r/58975/#comment247955>

nit: "// Rest"



sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
Lines 3003 (patched)
<https://reviews.apache.org/r/58975/#comment247958>

nit: if ()



sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
Lines 3031 (patched)
<https://reviews.apache.org/r/58975/#comment247957>

nit: " Process"



sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
Lines 3035 (patched)
<https://reviews.apache.org/r/58975/#comment247959>

Assert.fail(..) here.


- Hao Hao


On May 11, 2017, 1:16 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58975/
> ---
> 
> (Updated May 11, 2017, 1:16 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, 
> Vamsee Yarlagadda, and Vadim Spector.
> 
> 
> Bugs: SENTRY-1669
> https://issues.apache.org/jira/browse/SENTRY-1669
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Sentry should store Id's of all the notifications that it processed in a 
> seperate database.
> HMSFollower before every run should get the last notification ID from this 
> persistent store.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryHmsNotification.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  8fd5278 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/DeltaTransactionBlock.java
>  f590a52 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  ef67865 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  9880c40 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  e7443eb 
> 
> 
> Diff: https://reviews.apache.org/r/58975/diff/8/
> 
> 
> Testing
> ---
> 
> Sentry stores Id's of all the notifications that it processed in a seperate 
> database.
> HMSFollower before every run gets the last notification ID from this 
> persistent store.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58975: SENTRY-1669 HMSFollower should read current processed notification ID from database every time it runs

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58975/#review174532
---



Please comment on Jira description why need to store notification id in a 
sperate table. What is the benifit.


sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Lines 3402 (patched)
<https://reviews.apache.org/r/58975/#comment247698>

typo.



sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Lines 3423 (patched)
<https://reviews.apache.org/r/58975/#comment247696>

Does not seems to be the right way to use makePerisitent to me.



sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Line 3435 (original), 3466 (patched)
<https://reviews.apache.org/r/58975/#comment247700>

getLastProcessedNotificationIDCore will return null right? This behavior no 
longer match the method description.


- Hao Hao


On May 10, 2017, 7:27 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58975/
> ---
> 
> (Updated May 10, 2017, 7:27 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, 
> Vamsee Yarlagadda, and Vadim Spector.
> 
> 
> Bugs: SENTRY-1669
> https://issues.apache.org/jira/browse/SENTRY-1669
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Sentry should store Id's of all the notifications that it processed in a 
> seperate database.
> HMSFollower before every run should get the last notification ID from this 
> persistent store.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryHmsNotification.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  8fd5278 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/DeltaTransactionBlock.java
>  f590a52 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  ef67865 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  e271ea4 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  e7443eb 
> 
> 
> Diff: https://reviews.apache.org/r/58975/diff/5/
> 
> 
> Testing
> ---
> 
> Sentry stores Id's of all the notifications that it processed in a seperate 
> database.
> HMSFollower before every run gets the last notification ID from this 
> persistent store.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58825: SENTRY-1695: Waiting for HMS notifications from Thrift should be interruptible

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58825/#review173794
---


Ship it!




Ship It!

- Hao Hao


On May 3, 2017, 7:01 p.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58825/
> ---
> 
> (Updated May 3, 2017, 7:01 p.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, Na Li, Sergio 
> Pena, and Vamsee Yarlagadda.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1695: Waiting for HMS notifications from Thrift should be interruptible
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/SentryPolicyStoreProcessor.java
>  456dabc234031c00455a564aa2bd6ea87690ceb1 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/CounterWait.java
>  f593bffc827863955aa2591b3e387b2c7884242b 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/service/thrift/TestCounterWait.java
>  a70017857b5e48071a9653a211da0fa6d9c51c1e 
> 
> 
> Diff: https://reviews.apache.org/r/58825/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58867: SENTRY-1734 Create/Alter/Drop database/table should check corresponding property before drop priv...

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58867/#review173455
---


Fix it, then Ship it!





sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
Lines 527 (patched)
<https://reviews.apache.org/r/58867/#comment246434>

Do we need to file a jira to track it?


- Hao Hao


On April 29, 2017, 12:21 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58867/
> ---
> 
> (Updated April 29, 2017, 12:21 a.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, Na Li, and 
> Vamsee Yarlagadda.
> 
> 
> Bugs: SENTRY-1734
> https://issues.apache.org/jira/browse/SENTRY-1734
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1734 Create/Alter/Drop database/table should check corresponding 
> property before drop priv...
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  ca4487f279f435c45be9ded7bd00086f2bbd9b02 
> 
> 
> Diff: https://reviews.apache.org/r/58867/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58808: SENTRY-1726 sql changes to store last notification-id processed

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58808/#review173252
---




sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.derby.sql
Lines 30 (patched)
<https://reviews.apache.org/r/58808/#comment246295>

Can you explain more about why using 'signleton'and why have '15' length 
restriction here? And add a comment to it.


- Hao Hao


On April 27, 2017, 6:19 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58808/
> ---
> 
> (Updated April 27, 2017, 6:19 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, 
> Vamsee Yarlagadda, and Vadim Spector.
> 
> 
> Bugs: SENTRY-1726
> https://issues.apache.org/jira/browse/SENTRY-1726
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> 1. Created new table to store the last notification id
> 2. Table is defined to store only one entry.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.derby.sql
>  ba70715 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.mysql.sql
>  879e732 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.oracle.sql
>  e83ab83 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.postgres.sql
>  c28099b 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 841dcaa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 223835e 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 006d57b 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 20921ea 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  489ad66 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  ec50912 
> 
> 
> Diff: https://reviews.apache.org/r/58808/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58773: SENTRY-1727. HMSPathsDumper.cloneToEntry should set authzObjToEntries properly.

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58773/#review173141
---


Ship it!




Ship It!

- Hao Hao


On April 27, 2017, 12:32 a.m., Lei Xu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58773/
> ---
> 
> (Updated April 27, 2017, 12:32 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Bugs: SENTRY-1727
> https://issues.apache.org/jira/browse/SENTRY-1727
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> It is discovered during the process of SENTRY-1713 that 
> HMSPathsDumper.initializeFromDump is trying to clone a new HMSPaths from 
> TPathsDump. Internally it calls cloneToEntry to construct 
> HMSPaths.authzObjToEntries, which is representation of hive authorized 
> objects to path entries mapping. It appears to be cloneToEntry fail to 
> properly construct authzObjToEntries.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPathsDumper.java
>  7e569526feb56dffee142e3a0b106d216afde4e3 
> 
> 
> Diff: https://reviews.apache.org/r/58773/diff/1/
> 
> 
> Testing
> ---
> 
> mvn test -Dtest=TesetHDFSIntegrationEnd2End
> 
> 
> Thanks,
> 
> Lei Xu
> 
>

Re: Review Request 58284: SENTRY-1687 FullUpdateInitializer can be more efficient

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58284/#review173062
---


Ship it!




Ship It!

- Hao Hao


On April 26, 2017, 6:51 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58284/
> ---
> 
> (Updated April 26, 2017, 6:51 a.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, Na Li, Sergio 
> Pena, Vamsee Yarlagadda, and Vadim Spector.
> 
> 
> Bugs: SENTRY-1687
> https://issues.apache.org/jira/browse/SENTRY-1687
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1687 FullUpdateInitializer can be more efficient
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  d876d230f3200c71e9d11681efce8b24cac40b7e 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  e32d4a720e49464f812b12c46e8e0e040cb36fad 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ServiceConstants.java
>  23552c2512902a8500bfacb1c745ca4b10498cc8 
>   
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java
>  f338ce8abddcde128536a0efef77983990325aa6 
>   
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestPathsUpdate.java
>  b5cbea9d295385bb38b912c13903edace04d7589 
>   
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestUpdateableAuthzPaths.java
>  e643e01a45de77ef7f465d6921c5ae9e7ce925a2 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  0eaac80edcb4434d5f8953a44050720c5770faf3 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  8b88c9a2dbbb471d15e177f3d0a129256a19f228 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  57b7f888d1eb9e10447fe3190579aec6e05b661c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  84574f0fca4596d475aed48797a2d4d6fad02915 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  e771ce7de5a6849f48a036b2b1c30b435d8cb520 
> 
> 
> Diff: https://reviews.apache.org/r/58284/diff/3/
> 
> 
> Testing
> ---
> 
> Updated the unit test to test for bigger HMS snapshots
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/
---

(Updated April 21, 2017, 2:20 a.m.)


Review request for sentry.


Repository: sentry


Description
---

SENTRY-1587: Refactor SentryStore transaction to persist a single path 
transcation bundled with corresponding delta path change

Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
 90aaaef0e15306627d7108f12a74a29848055c0b 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
 14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
 c22364fa26c80415827313f4d26f6c53b71b6f6c 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 08520f30f3529af88d7dab9ae3623f28f0e43558 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 1543379eec2403fd1fbe61947c4c38a189177895 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
 PRE-CREATION 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 87e329588990be129197d598dd1db4487f8e0f25 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
 24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 7769f24bb4c062016084bcafe7d50a0f0e013824 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
 0e97466c9ba6973d8e787819e292c3b826472b38 


Diff: https://reviews.apache.org/r/58412/diff/4/

Changes: https://reviews.apache.org/r/58412/diff/3-4/


Testing
---

1. Add new unit test in TestSentryStore
2. Enabled hdfs sync integration test


Thanks,

Hao Hao

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 19, 2017, 12:08 a.m., Vamsee Yarlagadda wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2728-2731 (patched)
> > <https://reviews.apache.org/r/58412/diff/2/?file=1693608#file1693608line2730>
> >
> > If the mOldPaths is empty, we simply log the error and not do anything. 
> > Do you think we should add the new path (mNewPath) irrespective of oldPath 
> > existed or not?

Yes, make sense. Will update it.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172289
-----------


On April 18, 2017, 9:34 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 9:34 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
>  c74384688ca920c79fb2987d225042e135cdfd53 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  81a4c6ecb271d8f04fe8caab1b52e8b4a2813ba1 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/3/
> 
> 
> Testing
> ---
> 
> 1. Add new unit test in TestSentryStore
> 2. Enabled hdfs sync integration test
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 20, 2017, 9:30 p.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
> > Lines 27 (patched)
> > <https://reviews.apache.org/r/58412/diff/3/?file=1693880#file1693880line27>
> >
> > Please add a comment explaining why we need application identity in 
> > this case - it isn't obvious.

I have removed it based our offline discussion. As we are ok to use equality 
comparison by path value. since set semantics provide the guarantee.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172391
-----------


On April 18, 2017, 9:34 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 9:34 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
>  c74384688ca920c79fb2987d225042e135cdfd53 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  81a4c6ecb271d8f04fe8caab1b52e8b4a2813ba1 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/3/
> 
> 
> Testing
> ---
> 
> 1. Add new unit test in TestSentryStore
> 2. Enabled hdfs sync integration test
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 19, 2017, 3:06 a.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
> > Lines 334 (patched)
> > <https://reviews.apache.org/r/58412/diff/3/?file=1693884#file1693884line334>
> >
> > Is there anything to do if oldPathTree is null but newPathTree is not 
> > null?

We may still want to add the newPathTree if oldPathTree is null.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172301
-----------


On April 18, 2017, 9:34 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 9:34 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
>  c74384688ca920c79fb2987d225042e135cdfd53 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  81a4c6ecb271d8f04fe8caab1b52e8b4a2813ba1 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/3/
> 
> 
> Testing
> ---
> 
> 1. Add new unit test in TestSentryStore
> 2. Enabled hdfs sync integration test
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 19, 2017, 3:06 a.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
> > Lines 258 (patched)
> > <https://reviews.apache.org/r/58412/diff/3/?file=1693883#file1693883line258>
> >
> > Would this work correctly if we have two HMSFollowers running 
> > concurrently?

You mean concurrently running from two different server or from the same server?


> On April 19, 2017, 3:06 a.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
> > Lines 438 (patched)
> > <https://reviews.apache.org/r/58412/diff/3/?file=1693883#file1693883line491>
> >
> > Here and in other places in this function there isn't any value in 
> > throwing an exception. We got something wrong from Hive, we should just log 
> > it and continue. Stack trace would add much value.
> > 
> > Also here we know that one of the three is null and yet we try to show 
> > them all as %s so we are likely to get NPE here instead.

Currently if we have SentryInvalidInputException or 
SentryInvalidHMSEventException, it will log the error/exception. I can add to 
print the stacktrace there.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172301
---


On April 18, 2017, 9:34 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 9:34 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
>  c74384688ca920c79fb2987d225042e135cdfd53 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  81a4c6ecb271d8f04fe8caab1b52e8b4a2813ba1 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/3/
> 
> 
> Testing
> ---
> 
> 1. Add new unit test in TestSentryStore
> 2. Enabled hdfs sync integration test
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58593: SENTRY-1710. Reduce datanucleus key cache size for MSentryPermChange and MSentryPathChange tables to avoid holes in change IDs.

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58593/#review172565
---


Ship it!




Ship It!

- Hao Hao


On April 20, 2017, 8:56 p.m., Lei Xu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58593/
> ---
> 
> (Updated April 20, 2017, 8:56 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, kalyan kumar 
> kalvagadda, and Na Li.
> 
> 
> Bugs: SENTRY-1710
> https://issues.apache.org/jira/browse/SENTRY-1710
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> As discussed in SENTRY-1643 and SENTRY-1706, MSentryPermChange and 
> MSentryPathChange tables require that CHANGE ID of such changes to be unique 
> and consecutive. SENTRY-1644 addressed this requirement in the application 
> code, that requires multiple trips for each change record insertion, while it 
> raises the possibility of collision on change ID when there are concurrently 
> updates.
> 
> This patch change the ```key-cache-size``` to 1, as Lina suggested.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPathChange.java
>  4b42ed02d7b6996ee9e460f346ebfefc44be9734 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPermChange.java
>  a97d10ad92739ed17e543372024b6afb652fe0be 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  81a4c6ecb271d8f04fe8caab1b52e8b4a2813ba1 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/DeltaTransactionBlock.java
>  8d3c88ba24eef76c6c07d06fb426e27b1d7d7f6b 
> 
> 
> Diff: https://reviews.apache.org/r/58593/diff/1/
> 
> 
> Testing
> ---
> 
> mvn test -Dtest=TestSentryStore
> 
> 
> Thanks,
> 
> Lei Xu
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172393
---




sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
Lines 256 (patched)
<https://reviews.apache.org/r/58412/#comment245496>

If you look at the latest version it add the reason why we introduce it. 
But sasha suggested to add more comments there, will do.


- Hao Hao


On April 18, 2017, 9:34 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 9:34 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
>  c74384688ca920c79fb2987d225042e135cdfd53 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  81a4c6ecb271d8f04fe8caab1b52e8b4a2813ba1 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/3/
> 
> 
> Testing
> ---
> 
> 1. Add new unit test in TestSentryStore
> 2. Enabled hdfs sync integration test
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Clarification for MPath changes for SENTRY-1587

[Hao Hao]

Hi Alex,

I changed MPath class to include pathID to have correct MPath equality
comparison. Since the original equals() definition only compares 'path'
which for different MPath object it may have the same value.

For pathID (primary key) assignment I change it to be auto-increment by
specifying value-strategy to be "increment ".

Please let me know if you think any of the change is not reasonable. Thanks!

Best,
Hao

On Wed, Apr 19, 2017 at 11:37 AM, Alexander Kolbasov 
wrote:

> Hao,
>
> Can you clarify the changes you propose for MPath class and related
> package.jdo changes for SENTRY-1587?
>
> You suggest changing the identity type from "database" to "application",
> but the pathId is not initialized in constructor and not assigned anywhere.
> What is your intention here?
>
> Also you changed the equals() method to include ID. What kind of equality
> do you require - equality by path value or equality by identity? What is
> your thinking here?
>
> This is in regards to the code review https://reviews.apache.org/r/58412.
>
> Thanks,
>
> - Alex
>

Re: Review Request 58284: SENTRY-1687 FullUpdateInitializer can be more efficient

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58284/#review172288
---


Fix it, then Ship it!





sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
Lines 54 (patched)
<https://reviews.apache.org/r/58284/#comment245393>

typo, 'table' instead of 'partition'


- Hao Hao


On April 18, 2017, 12:26 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58284/
> ---
> 
> (Updated April 18, 2017, 12:26 a.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, Na Li, Sergio 
> Pena, Vamsee Yarlagadda, and Vadim Spector.
> 
> 
> Bugs: SENTRY-1687
> https://issues.apache.org/jira/browse/SENTRY-1687
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1687 FullUpdateInitializer can be more efficient
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ServiceConstants.java
>  23552c2512902a8500bfacb1c745ca4b10498cc8 
>   
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestFullUpdateInitializer.java
>  f338ce8abddcde128536a0efef77983990325aa6 
>   
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestPathsUpdate.java
>  b5cbea9d295385bb38b912c13903edace04d7589 
>   
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestUpdateableAuthzPaths.java
>  e643e01a45de77ef7f465d6921c5ae9e7ce925a2 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
> 
> 
> Diff: https://reviews.apache.org/r/58284/diff/2/
> 
> 
> Testing
> ---
> 
> Updated the unit test to test for bigger HMS snapshots
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58221: SENTRY-1649 move HMS follower to runServer

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58221/#review172273
---


Fix it, then Ship it!




Overall the changes looks good to me. Thanks a lot Lina for picking this up!


sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
Lines 314 (patched)
<https://reviews.apache.org/r/58221/#comment245376>

Nit: this line seems too long. Similar to above ones longer than 100.


- Hao Hao


On April 18, 2017, 3:46 a.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58221/
> ---
> 
> (Updated April 18, 2017, 3:46 a.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, kalyan kumar 
> kalvagadda, and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1649 move HMS follower to runServer
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
>  132db63 
> 
> 
> Diff: https://reviews.apache.org/r/58221/diff/15/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/
---

(Updated April 18, 2017, 9:34 p.m.)


Review request for sentry.


Repository: sentry


Description
---

SENTRY-1587: Refactor SentryStore transaction to persist a single path 
transcation bundled with corresponding delta path change

Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f


Diffs
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
 90aaaef0e15306627d7108f12a74a29848055c0b 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
 14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
 c22364fa26c80415827313f4d26f6c53b71b6f6c 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
 c74384688ca920c79fb2987d225042e135cdfd53 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
 81a4c6ecb271d8f04fe8caab1b52e8b4a2813ba1 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 08520f30f3529af88d7dab9ae3623f28f0e43558 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 16676fb13b0d5015aefe892a6f7e46812ea75124 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
 PRE-CREATION 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 87e329588990be129197d598dd1db4487f8e0f25 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
 24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 7769f24bb4c062016084bcafe7d50a0f0e013824 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
 0e97466c9ba6973d8e787819e292c3b826472b38 


Diff: https://reviews.apache.org/r/58412/diff/3/


Testing (updated)
---

1. Add new unit test in TestSentryStore
2. Enabled hdfs sync integration test


Thanks,

Hao Hao

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/
---

(Updated April 18, 2017, 9:29 p.m.)


Review request for sentry.


Repository: sentry


Description
---

SENTRY-1587: Refactor SentryStore transaction to persist a single path 
transcation bundled with corresponding delta path change

Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
 90aaaef0e15306627d7108f12a74a29848055c0b 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
 14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
 c22364fa26c80415827313f4d26f6c53b71b6f6c 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
 c74384688ca920c79fb2987d225042e135cdfd53 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
 81a4c6ecb271d8f04fe8caab1b52e8b4a2813ba1 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 08520f30f3529af88d7dab9ae3623f28f0e43558 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 16676fb13b0d5015aefe892a6f7e46812ea75124 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
 PRE-CREATION 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 87e329588990be129197d598dd1db4487f8e0f25 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
 24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 7769f24bb4c062016084bcafe7d50a0f0e013824 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
 0e97466c9ba6973d8e787819e292c3b826472b38 


Diff: https://reviews.apache.org/r/58412/diff/3/

Changes: https://reviews.apache.org/r/58412/diff/2-3/


Testing
---


Thanks,

Hao Hao

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 14, 2017, 6:45 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2519 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2519>
> >
> > Is it possible some paths should be removed from existing 
> > MAuthzPathsMapping entry?
> > 
> > For example, user wants to revoke the privilege of some tables from a 
> > role.

Not quite follow your question. If you are asking if there is deletion of 
paths, then the answer is yes, deleteAllAuthzPathsMapping and 
deleteAuthzPathsMapping are for so.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172007
-----------


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 14, 2017, 6:45 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2535 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2535>
> >
> > could the exception type to be more specific? Such as noSuchPath or its 
> > parent exception?

We are calling TransactionBlock.execute which throws  'Exception'. Given this 
cannot see a way to avoid it.


> On April 14, 2017, 6:45 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2562 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2562>
> >
> > should we continue delete the path, and then throw the exceptions? 
> > Those changes work in Hive, so Sentry needs to be more tolerate to errors.

Relaxed the checking here.


> On April 14, 2017, 6:45 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2579 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2579>
> >
> > If deleting one entry fails, should the operation continues for the 
> > rest of entries before throwing exception, or throw exception right away?
> > 
> > The changes are from Hive, and they succeeds at Hive. If exception 
> > happens at Sentry, should we be more tolenrate on the error?

I have updated it to handle one authzObjs only now.


> On April 14, 2017, 6:45 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2646 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2646>
> >
> > what is the sequence when renaming happens? Will the  old path renamed 
> > to new path, and then rename authz path mapping? 
> > 
> > If that is the case, then "mOldPath == null" should be allowed.

We delete the old path first and add the new path. I also relax the checking 
here.


> On April 14, 2017, 6:45 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Line 2473 (original), 2661 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2661>
> >
> > should we throw exception for mOldPath == null? Is that possible that 
> > old path is renamed, and then this function is called?

yeah, I have relax the checking here.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172007
---


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172270
---



- Hao Hao


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 17, 2017, 8:35 p.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2560 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2560>
> >
> > I don't think you need to explicitely do this - it should be sufficient 
> > to remove the path from the set and update the MAuthzPathsMapping object.
> 
> Hao Hao wrote:
> Will try and update.

Just tried with remove 'pm.deletePersistent(mPath);', 
TestSentryStore.testAddDeleteAuthzPathsMapping is failing as the path is not 
deleted.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172131
-----------


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 18, 2017, 4:14 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2545 (patched)
> > <https://reviews.apache.org/r/58412/diff/2/?file=1693608#file1693608line2547>
> >
> > Is it true that only zero or one entry will be returned? 
> > 
> > Can you add comment for this public function?

Will add, it can return a list of paths.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172212
-----------


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 18, 2017, 4:14 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2563 (patched)
> > <https://reviews.apache.org/r/58412/diff/2/?file=1693608#file1693608line2565>
> >
> > Comment of this function execute(deltaTransactionBlock, 
> > TransactionBlock) says "The order of the transaction does not matter 
> > because there is no any return value.". But it seems to me the order could 
> > matter. 
> > 
> > For example, you want to add the mapping between AuthzObj and the paths 
> > before you apply update to it. If the mapping has not been created, would 
> > the update to this mapping fail?

Maybe the comment is somewhat misleading, will revise it. The comment for 
execute is referring the order within one transaction and it is about the 
oreder in terms of how to handle return value. Itf you intent to 'add the 
mapping between AuthzObj and the paths and then apply update to it'. Of course 
you need to implement it such way.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172212
---


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172256
---



- Hao Hao


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 18, 2017, 4:14 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2630 (patched)
> > <https://reviews.apache.org/r/58412/diff/2/?file=1693608#file1693608line2632>
> >
> > Could the authzObj to Paths mapping be many-to-many? i.e., authzObj 
> > does not own the paths.
> > 
> > If this is true, then when deleting the mapping, should we deleting the 
> > mPath while ther mapping may still reference this mPath? In this case, will 
> > deletion of mPath trigger the deletion of other mapping that reference this 
> > mPath?

authzObj will not share the same MPath object with other authzObj. As it 
creates a new MPath each time when adding a new path even if there could ba a 
path has the same location as others.


> On April 18, 2017, 4:14 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2725 (patched)
> > <https://reviews.apache.org/r/58412/diff/2/?file=1693608#file1693608line2727>
> >
> > are you sure only this oldObj owns this mOldPath?

We are not sure, that is why there is checking for 
mAuthzPathsMapping.getPaths().contains(mOldPath) to ensure it is the exact same 
path for oldObj.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172212
---


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 18, 2017, 4:14 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
> > Lines 257 (patched)
> > <https://reviews.apache.org/r/58412/diff/2/?file=1693609#file1693609line257>
> >
> > how long will the notification ID overflow? This ID is saved in DB, and 
> > if it wraps within a reasonable time for big customers, we need to handle 
> > the overflow case. 
> > 
> > For example, instead of check "if (eventId.getEventId() > 
> > currentEventID)" we can have a function "CompareEventId(eventIdA, 
> > eventIdB)". if eventId A is older than eventId B, returns -1, if equal, 
> > returns 0, and newer, returns 1. 
> > 
> > You can have simple implementation, but later can change to handle 
> > overflow.

That's a good point to ask how overflow is handled. However, I think overflow 
should be handled by Hive metastore before they even generate a new 
notification log. And notify its client such as Sentry. It would be interesting 
to check if HMS is handling for overflow.


> On April 18, 2017, 4:14 p.m., Na Li wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
> > Line 343 (original), 348 (patched)
> > <https://reviews.apache.org/r/58412/diff/2/?file=1693609#file1693609line348>
> >
> > If this function only throw SentryInvalidHMSEventException, why not 
> > mark the function with this specific exception instead of "Exception"?

notificationProcessor.processXXX would throw Exception coming from SentryStore.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172212
---


On April 18, 2017, 7:34 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 18, 2017, 7:34 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  08520f30f3529af88d7dab9ae3623f28f0e43558 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  87e329588990be129197d598dd1db4487f8e0f25 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
>  24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  7769f24bb4c062016084bcafe7d50a0f0e013824 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
>  0e97466c9ba6973d8e787819e292c3b826472b38 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: [WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/
---

(Updated April 18, 2017, 7:34 a.m.)


Review request for sentry.


Repository: sentry


Description
---

[WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path 
transcation bundled with corresponding delta path change

Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
 90aaaef0e15306627d7108f12a74a29848055c0b 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
 14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
 c22364fa26c80415827313f4d26f6c53b71b6f6c 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 08520f30f3529af88d7dab9ae3623f28f0e43558 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 16676fb13b0d5015aefe892a6f7e46812ea75124 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
 PRE-CREATION 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 87e329588990be129197d598dd1db4487f8e0f25 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
 24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 7769f24bb4c062016084bcafe7d50a0f0e013824 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
 0e97466c9ba6973d8e787819e292c3b826472b38 


Diff: https://reviews.apache.org/r/58412/diff/2/

Changes: https://reviews.apache.org/r/58412/diff/1-2/


Testing
---


Thanks,

Hao Hao

Re: Review Request 58412: SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/
---

(Updated April 18, 2017, 7:34 a.m.)


Review request for sentry.


Summary (updated)
-

SENTRY-1587: Refactor SentryStore transaction to persist a single path 
transcation bundled with corresponding delta path change


Repository: sentry


Description (updated)
---

SENTRY-1587: Refactor SentryStore transaction to persist a single path 
transcation bundled with corresponding delta path change

Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f


Diffs
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
 90aaaef0e15306627d7108f12a74a29848055c0b 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
 14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
 c22364fa26c80415827313f4d26f6c53b71b6f6c 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 08520f30f3529af88d7dab9ae3623f28f0e43558 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 16676fb13b0d5015aefe892a6f7e46812ea75124 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
 PRE-CREATION 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 87e329588990be129197d598dd1db4487f8e0f25 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
 24ab1a8b392f23bc75759733bef7cecd4bc2ac34 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 7769f24bb4c062016084bcafe7d50a0f0e013824 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
 0e97466c9ba6973d8e787819e292c3b826472b38 


Diff: https://reviews.apache.org/r/58412/diff/2/


Testing
---


Thanks,

Hao Hao

Re: Review Request 58412: [WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 14, 2017, 11:06 p.m., kalyan kumar kalvagadda wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2511 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2511>
> >
> > Currently this method gets called while processing CREATE_DATABASE, 
> > CREATE_TABLE and ADD_PARTITION events.
> > 
> > On receiving CREATE_DATABASE, CREATE_TABLE events, 
> > mAuthzPathsMapping entry is not expected. If we find any we should log 
> > an error.
> > 
> > On receiving CREATE_DATABASE, CREATE_TABLE events, you may want to call 
> > createAuthzPathsMappingCore method instead.

Create_database and alike should not call createAuthzPathsMappingCore. 
createAuthzPathsMappingCore is only for persisting full snapshot that gets from 
HMS from the first startup. This method will throw exceptions if the authzObj 
exist in the mapping which is likely to happen for create_database or alike.


> On April 14, 2017, 11:06 p.m., kalyan kumar kalvagadda wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
> > Line 394 (original), 384 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691538#file1691538line402>
> >
> > When a view is created, location will be empty. Such events should be 
> > ignored instead of throwing exceptions.

The current HMS side plugin(before HA) is not handling view. If we want to 
extend it to handle view events it would be better to handle it in follow up 
jira(SENTRY-1693).


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172039
---


On April 13, 2017, 6:58 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 13, 2017, 6:58 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> [WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  aaea9790282f9136302eec64107cc86391a4d6ff 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  31a309bdc789bd7a997f7654e30f2021ecb5b616 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: [WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172024
---




sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Lines 2519 (patched)
<https://reviews.apache.org/r/58412/#comment245083>

Not quite follow your question. If you are asking if there is deletion of 
paths, then the answer is yes, deleteAllAuthzPathsMapping and 
deleteAuthzPathsMapping are for so.



sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Lines 2535 (patched)
<https://reviews.apache.org/r/58412/#comment245084>

We are calling TransactionBlock.execute which throws  'Exception'. Given 
this cannot see a way to avoid it.



sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Lines 2562 (patched)
<https://reviews.apache.org/r/58412/#comment245086>

Trying to be the same behavior as how HMSPaths is handling delete path from 
AuthzObj. In deletePathsFromAuthzObject it skip the deletion if it cannot find 
it and log the info.


- Hao Hao


On April 13, 2017, 6:58 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 13, 2017, 6:58 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> [WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  aaea9790282f9136302eec64107cc86391a4d6ff 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  31a309bdc789bd7a997f7654e30f2021ecb5b616 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 58412: [WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

> On April 17, 2017, 8:35 p.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2560 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2560>
> >
> > I don't think you need to explicitely do this - it should be sufficient 
> > to remove the path from the set and update the MAuthzPathsMapping object.

Will try and update.


> On April 17, 2017, 8:35 p.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 2776 (patched)
> > <https://reviews.apache.org/r/58412/diff/1/?file=1691537#file1691537line2776>
> >
> > Is this used?

Removed.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/#review172131
-----------


On April 13, 2017, 6:58 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58412/
> ---
> 
> (Updated April 13, 2017, 6:58 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> [WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path 
> transcation bundled with corresponding delta path change
> 
> Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  90aaaef0e15306627d7108f12a74a29848055c0b 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  c22364fa26c80415827313f4d26f6c53b71b6f6c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  aaea9790282f9136302eec64107cc86391a4d6ff 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  16676fb13b0d5015aefe892a6f7e46812ea75124 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  31a309bdc789bd7a997f7654e30f2021ecb5b616 
> 
> 
> Diff: https://reviews.apache.org/r/58412/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: [DISCUSS] Merging Sentry HA branch with master

[Hao Hao]

The proposal is good to me as well. Should we start a vote on this? Or we
can just start to do it if no one is objecting?

Best,
Hao

On Mon, Apr 17, 2017 at 4:42 PM, Vamsee Yarlagadda 
wrote:

> >
> > Cherry-pick any commits that happened since sentry-ha-redesign was
> forked,
> > except a few described below
> > Exclude big refactoring commit (SENTRY-1205) and related commits
> > (SENTRY-1436, SENTRY-1438, SENTRY-1406)
> > Rename master to a dev branch
> > Rename sentry-ha-redesign to master
>
>
> This sounds good to me. Generally having merge commits complicates the git
> history and might get tricky when we are debugging things. I would rather
> stick with the approach of cherry-picks to make the history clear.
>
> Thanks,
> Vamsee
>
> On Tue, Apr 4, 2017 at 7:17 PM, Alexander Kolbasov 
> wrote:
>
> > I would like to make a more concrete proposal and I am interested in
> > opinion from Sentry PMC members on this.
> >
> > I would propose the following approach for merging Sentry HA into Sentry
> > master:
> >
> > Cherry-pick any commits that happened since sentry-ha-redesign was
> forked,
> > except a few described below
> > Exclude big refactoring commit (SENTRY-1205) and related commits
> > (SENTRY-1436, SENTRY-1438, SENTRY-1406)
> > Rename master to a dev branch
> > Rename sentry-ha-redesign to master
> >
> > What does community think about such approach?
> >
> > - Alex
> >
> >
> > > On Mar 22, 2017, at 1:43 PM, Alexander Kolbasov 
> > wrote:
> > >
> > > Hello,
> > >
> > > I would like to start the discussion on merging sentry-ha-redesign
> > branch with master.
> > >
> > > As of now most of the changes from master are merged into
> > sentry-ha-redesign. The major missing part is SENTRY-1205 (Refactor the
> > code for sentry-provider-db and create sentry-service module) and
> > associated issues. This refactoring is very hard to port, especially
> since
> > there is very little information in the JIRA on why it was done and how
> it
> > was done - was it merely moving files around or more then that. I would
> > seriously consider not including this change in 1.8.
> > >
> > > So in regards to the merge we have several options:
> > >
> > > Attempt to merge master into sentry-ha-redesign, resolve any conflicts
> > and later commit the merge to master. This will cause merge commit on
> master
> > > Finish work on sentry-ha-redesign, make sure that relevant commits are
> > ported from master, and then making this a master branch and making
> current
> > master a special branch left for reference purposes. This will likely
> leave
> > SENTRY-1205 and related issues out.
> > > What does community think about this?
> > >
> > > - Alex
> >
> >
>
>
> --
> Thanks,
> Vamsee
>

Re: Review Request 58164: SENTRY-1638 Update SQL script of MSentryPathChange table to add a column for notification ID

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58164/#review171847
---


Ship it!




Ship It!

- Hao Hao


On April 7, 2017, 9:56 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58164/
> ---
> 
> (Updated April 7, 2017, 9:56 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, and Na Li.
> 
> 
> Bugs: SENTRY-1638
> https://issues.apache.org/jira/browse/SENTRY-1638
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> UpdateD SQL script of MSentryPathChange table to add a column for 
> notification ID.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  e981bcf0f327346c09cdbe5785fb8824fc62e704 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.mysql.sql
>  1829e2fa1f02a4339e7af4bf45a169013e9ec65f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.derby.sql
>  84b7903857ba51badde8a769eae86219eeaf3dab 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.mysql.sql
>  69865508103bb8b1dd6148aa48c616d4b683614f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.oracle.sql
>  6a438ea847619af4d4d99d0fb99e036e55bf425f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.postgres.sql
>  a9b7e2c48bb86bc0a81fad00d27387e26ef18c28 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 547bbe8136186658e7fe76ab24934157ea5300ff 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 6474389a18ea59da28d3d7125cf227c7aaa7f7aa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 1ab83432db233ee4e7aa054adc1b82c26248a099 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 0418b298f7cbd8f430733cb329e9fca263bda0f7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  68d2c8d53d7c468269ca2c41986ef8651b94f5c7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  5376c166659b3e8e373c8c20818f5b1290af90c9 
> 
> 
> Diff: https://reviews.apache.org/r/58164/diff/7/
> 
> 
> Testing
> ---
> 
> I have tested it against derby, oracle, postgres, mysql and db2.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Review Request 58412: [WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path transcation bundled with corresponding delta path change

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58412/
---

Review request for sentry.


Repository: sentry


Description
---

[WIP]SENTRY-1587: Refactor SentryStore transaction to persist a single path 
transcation bundled with corresponding delta path change

Change-Id: Ice77f631e92c0e74b42ae644e1b2f90c6089e62f


Diffs
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
 90aaaef0e15306627d7108f12a74a29848055c0b 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
 14e967aa1065f16e8d4c3f61db2f9055959fa9e6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
 c22364fa26c80415827313f4d26f6c53b71b6f6c 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 aaea9790282f9136302eec64107cc86391a4d6ff 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 16676fb13b0d5015aefe892a6f7e46812ea75124 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/NotificationProcessor.java
 PRE-CREATION 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 31a309bdc789bd7a997f7654e30f2021ecb5b616 


Diff: https://reviews.apache.org/r/58412/diff/1/


Testing
---


Thanks,

Hao Hao

Re: Review Request 58221: SENTRY-1649 move HMS follower to runServer

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58221/#review171459
---




sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
Line 312 (original), 317 (patched)
<https://reviews.apache.org/r/58221/#comment244399>

Should we have a new config specific for timeout?



sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
Lines 325 (patched)
<https://reviews.apache.org/r/58221/#comment244400>

Should we call hmsFollower.stop(); before terminate hmsFollowerExecutor?


- Hao Hao


On April 7, 2017, 5:23 p.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58221/
> ---
> 
> (Updated April 7, 2017, 5:23 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, kalyan kumar 
> kalvagadda, and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1649 move HMS follower to runServer
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  6c14f5e46aad4223347d8d057188d31efbb68ed8 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
>  132db635f639cef91ce675d34d717e6125a0a4d1 
> 
> 
> Diff: https://reviews.apache.org/r/58221/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58281/#review171458
---


Fix it, then Ship it!





sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
Lines 2581 (patched)
<https://reviews.apache.org/r/58281/#comment244398>

Can you add a comment for what this test is about? Also a link to the 
sentry jira number. Thanks!


- Hao Hao


On April 8, 2017, 12:38 a.m., Lei Xu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58281/
> ---
> 
> (Updated April 8, 2017, 12:38 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Bugs: SENTRY-1643
> https://issues.apache.org/jira/browse/SENTRY-1643
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> When it relies on the SQL auto increment primary key as ChangeID, it can not 
> guarentee the consectivity of the IDs, because while each transaction claims 
> new ID from the table counter, the concurrent transactions which were not 
> success would not return the claimed changeIDs to the pool, thus it can not 
> guarateen the consectivity of change IDs.
> 
> This patch changes to use application logic to force the consectivity of IDs.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPathChange.java
>  a0d34459d7b2f70e863ef6e078401df81381c91b 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPermChange.java
>  476fbcb2ad26de23757842111beb12b154e1562b 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/DeltaTransactionBlock.java
>  f590a5296c047e1acedd39a4f2e4f1de98008d32 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  802b9c6cbf8e9ad015e37037b809b58c956de746 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  aaa0b9fd30bb68fded67f885af4f77bc71398e77 
> 
> 
> Diff: https://reviews.apache.org/r/58281/diff/1/
> 
> 
> Testing
> ---
> 
> Add a new test to conurrently insert changes. 
> 
> mvn test -Dtest=TestSentryStore.
> 
> 
> Thanks,
> 
> Lei Xu
> 
>

Re: Review Request 58267: SENTRY-1629 sql changed needed for MAuthzPathsMapping.

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58267/#review171375
---


Fix it, then Ship it!





sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Line 59 (original), 59 (patched)
<https://reviews.apache.org/r/58267/#comment244263>

Make it explict package.



sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
Line 2235 (original), 2235 (patched)
<https://reviews.apache.org/r/58267/#comment244267>

Looks like this one is redundant andb be covered by the following test? You 
can just remove it.



sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
Lines 2255 (patched)
<https://reviews.apache.org/r/58267/#comment244264>

'multiple paths'?


- Hao Hao


On April 7, 2017, 5:24 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58267/
> ---
> 
> (Updated April 7, 2017, 5:24 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, Na Li, Sergio Pena, 
> Vamsee Yarlagadda, and Vadim Spector.
> 
> 
> Bugs: SENTRY-1629
> https://issues.apache.org/jira/browse/SENTRY-1629
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Older JDO defination had issues handling multiple paths associtaed with one 
> Authorizable object.
> 
> Submitted code changes address this issue. Here is the snapshot of the changes
> 
> 1. Change the JDO definition to have Path as separate entity. 
> 2. SQL changes needed for the handle the new JDO definition.
> 3. Application changes to use new JDO definition.
> 4. Updated unit test cases to test the case where an authz object is 
> associated with more than one path.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MAuthzPathsMapping.java
>  56e456eb37df6d0a14402f57dbc400be9b72 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MPath.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  e981bcf0f327346c09cdbe5785fb8824fc62e704 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  802b9c6cbf8e9ad015e37037b809b58c956de746 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.derby.sql
>  1883626262bf4f4936f156a7ac74365b9b5873df 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.mysql.sql
>  1829e2fa1f02a4339e7af4bf45a169013e9ec65f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.oracle.sql
>  7de9751892a8ff84067f67d542ac58d33e9148d8 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.postgres.sql
>  adf5f1f39596309183f8c80d2c8ad1f1a7730236 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 547bbe8136186658e7fe76ab24934157ea5300ff 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 6474389a18ea59da28d3d7125cf227c7aaa7f7aa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 1ab83432db233ee4e7aa054adc1b82c26248a099 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 0418b298f7cbd8f430733cb329e9fca263bda0f7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  68d2c8d53d7c468269ca2c41986ef8651b94f5c7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  5376c166659b3e8e373c8c20818f5b1290af90c9 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  aaa0b9fd30bb68fded67f885af4f77bc71398e77 
> 
> 
> Diff: https://reviews.apache.org/r/58267/diff/1/
> 
> 
> Testing
> ---
> 
> Testing done:
> 1. Tested the sql changes with derby, mysql, oracle, postgres and db2.
> 2. Added unit tests to be sure that the actual issues is addressed.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58166: SENTRY-1692 ZK namespace configuration doesn't work

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58166/#review171310
---


Ship it!




Ship It!

- Hao Hao


On April 4, 2017, 5:03 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58166/
> ---
> 
> (Updated April 4, 2017, 5:03 a.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, and Na Li.
> 
> 
> Bugs: SENTRY-1692
> https://issues.apache.org/jira/browse/SENTRY-1692
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1692 ZK namespace configuration doesn't work
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/HAContext.java
>  00eec4eab066829ae1226d0ba3485eab7bd9eeb2 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/LeaderStatusMonitor.java
>  1d79bd1f722505e56941dfd131d782e518515dc4 
> 
> 
> Diff: https://reviews.apache.org/r/58166/diff/1/
> 
> 
> Testing
> ---
> 
> I tested this manually by running the server with namespace provided in the 
> config.
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58093: SENTRY-1676: FullUpdateInitializer#createInitialUpdate should not throw RuntimeExceptio

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58093/#review171309
---


Ship it!




Ship It!

- Hao Hao


On March 31, 2017, 5:07 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58093/
> ---
> 
> (Updated March 31, 2017, 5:07 a.m.)
> 
> 
> Review request for sentry, Lei Xu, Hao Hao, kalyan kumar kalvagadda, and Na 
> Li.
> 
> 
> Bugs: SENTRY-1676
> https://issues.apache.org/jira/browse/SENTRY-1676
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1676: FullUpdateInitializer#createInitialUpdate should not throw 
> RuntimeExceptio
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  146cea2b9467ce82b69bbf402933b1aa350bcd46 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  6c14f5e46aad4223347d8d057188d31efbb68ed8 
> 
> 
> Diff: https://reviews.apache.org/r/58093/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58221: SENTRY-1649 move HMS follower to runServer

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58221/#review171244
---




sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
Lines 310 (patched)
<https://reviews.apache.org/r/58221/#comment244140>

what happens if the submitted task has not been shut down and someone calls 
startHMSFollower? Is it safe to allow so?


- Hao Hao


On April 5, 2017, 10:29 p.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58221/
> ---
> 
> (Updated April 5, 2017, 10:29 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, kalyan kumar 
> kalvagadda, and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1649 move HMS follower to runServer
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
>  132db635f639cef91ce675d34d717e6125a0a4d1 
> 
> 
> Diff: https://reviews.apache.org/r/58221/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 58221: SENTRY-1649 move HMS follower to runServer

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58221/#review171238
---


Fix it, then Ship it!





sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
Lines 310 (patched)
<https://reviews.apache.org/r/58221/#comment244117>

Why not directly shutdown and make it null?



sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
Lines 313 (patched)
<https://reviews.apache.org/r/58221/#comment244116>

Extra line?


- Hao Hao


On April 5, 2017, 10:29 p.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58221/
> ---
> 
> (Updated April 5, 2017, 10:29 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, kalyan kumar 
> kalvagadda, and Sergio Pena.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1649 move HMS follower to runServer
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java
>  132db635f639cef91ce675d34d717e6125a0a4d1 
> 
> 
> Diff: https://reviews.apache.org/r/58221/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 58164: SENTRY-1638 Update SQL script of MSentryPathChange table to add a column for notification ID

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58164/#review171237
---




sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql
Lines 269 (patched)
<https://reviews.apache.org/r/58164/#comment244115>

If you are adding unique index here, you also need to change the jdo 
definition as I do not see it has this constraint.


- Hao Hao


On April 6, 2017, 2:17 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58164/
> ---
> 
> (Updated April 6, 2017, 2:17 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, and Na Li.
> 
> 
> Bugs: SENTRY-1638
> https://issues.apache.org/jira/browse/SENTRY-1638
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> UpdateD SQL script of MSentryPathChange table to add a column for 
> notification ID.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.mysql.sql
>  1829e2fa1f02a4339e7af4bf45a169013e9ec65f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.derby.sql
>  84b7903857ba51badde8a769eae86219eeaf3dab 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.mysql.sql
>  69865508103bb8b1dd6148aa48c616d4b683614f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.oracle.sql
>  6a438ea847619af4d4d99d0fb99e036e55bf425f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1569.postgres.sql
>  a9b7e2c48bb86bc0a81fad00d27387e26ef18c28 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 547bbe8136186658e7fe76ab24934157ea5300ff 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 6474389a18ea59da28d3d7125cf227c7aaa7f7aa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 1ab83432db233ee4e7aa054adc1b82c26248a099 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 0418b298f7cbd8f430733cb329e9fca263bda0f7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  68d2c8d53d7c468269ca2c41986ef8651b94f5c7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  5376c166659b3e8e373c8c20818f5b1290af90c9 
> 
> 
> Diff: https://reviews.apache.org/r/58164/diff/6/
> 
> 
> Testing
> ---
> 
> I have tested it against derby, oracle, postgres, mysql and db2.
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58130: SENTRY-1684 FullUpdateInitializer has a race condition in handling results list

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58130/#review171235
---


Ship it!




Ship It!

- Hao Hao


On April 1, 2017, 12:35 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58130/
> ---
> 
> (Updated April 1, 2017, 12:35 a.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, Na Li, and Vadim 
> Spector.
> 
> 
> Bugs: SENTRY-1684
> https://issues.apache.org/jira/browse/SENTRY-1684
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1684 FullUpdateInitializer has a race condition in handling results 
> list
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/FullUpdateInitializer.java
>  146cea2b9467ce82b69bbf402933b1aa350bcd46 
> 
> 
> Diff: https://reviews.apache.org/r/58130/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58164: SENTRY-1638 Update SQL script of MSentryPathChange table to add a column for notification ID

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58164/#review171184
---




sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-mysql-1.7.0-to-1.8.0.sql
Lines 5 (patched)
<https://reviews.apache.org/r/58164/#comment244064>

Am I missing something? why is still using 008-xxx here? :) Also, again 
same as Sasha pointed out, I would prefer not to usea seperate sql script file 
to update the new table no one is using yet.

BTW, did you get a chance to verify those changes?


- Hao Hao


On April 6, 2017, 12:35 a.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58164/
> ---
> 
> (Updated April 6, 2017, 12:35 a.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, and Na Li.
> 
> 
> Bugs: SENTRY-1638
> https://issues.apache.org/jira/browse/SENTRY-1638
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> UpdateD SQL script of MSentryPathChange table to add a column for 
> notification ID.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.mysql.sql
>  1829e2fa1f02a4339e7af4bf45a169013e9ec65f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/009-SENTRY-1638.derby.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/009-SENTRY-1638.mysql.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/009-SENTRY-1638.oracle.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/009-SENTRY-1638.postgres.sql
>  PRE-CREATION 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 547bbe8136186658e7fe76ab24934157ea5300ff 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 6474389a18ea59da28d3d7125cf227c7aaa7f7aa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 1ab83432db233ee4e7aa054adc1b82c26248a099 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 0418b298f7cbd8f430733cb329e9fca263bda0f7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  68d2c8d53d7c468269ca2c41986ef8651b94f5c7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  5376c166659b3e8e373c8c20818f5b1290af90c9 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-derby-1.7.0-to-1.8.0.sql
>  dc3021a57ac9ea8145379c437ae6a94131c9ae5e 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-mysql-1.7.0-to-1.8.0.sql
>  ef1c651da8998eb57b5d0ca3463317d07c1b3a24 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-oracle-1.7.0-to-1.8.0.sql
>  f4a50de14dd0f961befaded394c6a01c2f459317 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-postgres-1.7.0-to-1.8.0.sql
>  db010bcd7792c9370478a2061e22d14bb78e3184 
> 
> 
> Diff: https://reviews.apache.org/r/58164/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58069: SENTRY-1670 Expose current HMS notification ID as a Sentry gauge (metric)

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58069/#review171038
---


Ship it!




Ship It!

- Hao Hao


On April 4, 2017, 8:33 p.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58069/
> ---
> 
> (Updated April 4, 2017, 8:33 p.m.)
> 
> 
> Review request for sentry, Hao Hao and kalyan kumar kalvagadda.
> 
> 
> Bugs: SENTRY-1670
> https://issues.apache.org/jira/browse/SENTRY-1670
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1670 Expose current HMS notification ID as a Sentry gauge (metric)
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  bbfa713262c5b4d5cecccd95c823a78c9149752c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/SentryMetrics.java
>  6ed2c781b4ac8819f6d17a249c33e32f0116e15a 
> 
> 
> Diff: https://reviews.apache.org/r/58069/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58164: SENTRY-1638 Update SQL script of MSentryPathChange table to add a column for notification ID

[Hao Hao]

> On April 4, 2017, 4:40 p.m., Hao Hao wrote:
> > sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.oracle.sql
> > Lines 1 (patched)
> > <https://reviews.apache.org/r/58164/diff/2/?file=1684331#file1684331line1>
> >
> > Why create a new file for create notification id field  inside 
> > PATHChange table? It should be ok to just add upon the 
> > 008-SENTRY-1569.xxx.sql as there is no release yet.
> 
> kalyan kumar kalvagadda wrote:
> My assumtion was that, we maintain one file per change which is labled 
> with the Jira id so that would have a track of every change and why. That way 
> we can use all thse files while upgrading.
> 
> Thought, adding this change to 008-SENTRY-1569.xxx.sql would mix-up 
> things.

Ok, but I am not sure if there is any impact for upgrade by doing in this way.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58164/#review170981
---


On April 4, 2017, 5:05 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58164/
> ---
> 
> (Updated April 4, 2017, 5:05 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, and Na Li.
> 
> 
> Bugs: SENTRY-1638
> https://issues.apache.org/jira/browse/SENTRY-1638
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> UpdateD SQL script of MSentryPathChange table to add a column for 
> notification ID. Wich include changes to jdo defination and also 
> MSentryPermChange class.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.mysql.sql
>  1829e2fa1f02a4339e7af4bf45a169013e9ec65f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.derby.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.mysql.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.oracle.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.postgres.sql
>  PRE-CREATION 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 547bbe8136186658e7fe76ab24934157ea5300ff 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 6474389a18ea59da28d3d7125cf227c7aaa7f7aa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 1ab83432db233ee4e7aa054adc1b82c26248a099 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 0418b298f7cbd8f430733cb329e9fca263bda0f7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  68d2c8d53d7c468269ca2c41986ef8651b94f5c7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  5376c166659b3e8e373c8c20818f5b1290af90c9 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-derby-1.7.0-to-1.8.0.sql
>  dc3021a57ac9ea8145379c437ae6a94131c9ae5e 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-mysql-1.7.0-to-1.8.0.sql
>  ef1c651da8998eb57b5d0ca3463317d07c1b3a24 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-oracle-1.7.0-to-1.8.0.sql
>  f4a50de14dd0f961befaded394c6a01c2f459317 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-postgres-1.7.0-to-1.8.0.sql
>  db010bcd7792c9370478a2061e22d14bb78e3184 
> 
> 
> Diff: https://reviews.apache.org/r/58164/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58164: SENTRY-1638 Update SQL script of MSentryPathChange table to add a column for notification ID

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58164/#review171003
---




sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-postgres-1.7.0-to-1.8.0.sql
Lines 5 (patched)
<https://reviews.apache.org/r/58164/#comment243837>

Should it be 009-xxx other than 008-xxx if we want to have a seperate file 
for the change?


- Hao Hao


On April 4, 2017, 5:05 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58164/
> ---
> 
> (Updated April 4, 2017, 5:05 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, and Na Li.
> 
> 
> Bugs: SENTRY-1638
> https://issues.apache.org/jira/browse/SENTRY-1638
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> UpdateD SQL script of MSentryPathChange table to add a column for 
> notification ID. Wich include changes to jdo defination and also 
> MSentryPermChange class.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.mysql.sql
>  1829e2fa1f02a4339e7af4bf45a169013e9ec65f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.derby.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.mysql.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.oracle.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.postgres.sql
>  PRE-CREATION 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 547bbe8136186658e7fe76ab24934157ea5300ff 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 6474389a18ea59da28d3d7125cf227c7aaa7f7aa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 1ab83432db233ee4e7aa054adc1b82c26248a099 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 0418b298f7cbd8f430733cb329e9fca263bda0f7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  68d2c8d53d7c468269ca2c41986ef8651b94f5c7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  5376c166659b3e8e373c8c20818f5b1290af90c9 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-derby-1.7.0-to-1.8.0.sql
>  dc3021a57ac9ea8145379c437ae6a94131c9ae5e 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-mysql-1.7.0-to-1.8.0.sql
>  ef1c651da8998eb57b5d0ca3463317d07c1b3a24 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-oracle-1.7.0-to-1.8.0.sql
>  f4a50de14dd0f961befaded394c6a01c2f459317 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-postgres-1.7.0-to-1.8.0.sql
>  db010bcd7792c9370478a2061e22d14bb78e3184 
> 
> 
> Diff: https://reviews.apache.org/r/58164/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58069: SENTRY-1670 Expose current HMS notification ID as a Sentry gauge (metric)

[Hao Hao]

> On April 4, 2017, 6:31 a.m., Hao Hao wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Line 3167 (original), 3187 (patched)
> > <https://reviews.apache.org/r/58069/diff/1/?file=1680728#file1680728line3187>
> >
> > Why do we change from 'Long' to 'long' here? it would result in extra 
> > unbox and autobox.
> 
> Alexander Kolbasov wrote:
> I did to get rid of IntelliJ warning about autoboxing of EMPTY_CHANGE_ID 
> below. This isn't the usual case so we could ignore the warning, but when I 
> checked usages it turns out that everyone is using 
> getLastProcessedChangeIDCore() as long, so everyone unboxes anyway, so the 
> net result is zero.

I saw 'getLastProcessedPermChangeID' , 'getLastProcessedPathChangeID', etc, are 
using it as 'Long'. Did I miss anything?


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58069/#review170958
---


On March 30, 2017, 5:25 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58069/
> -------
> 
> (Updated March 30, 2017, 5:25 a.m.)
> 
> 
> Review request for sentry, Hao Hao and kalyan kumar kalvagadda.
> 
> 
> Bugs: SENTRY-1670
> https://issues.apache.org/jira/browse/SENTRY-1670
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1670 Expose current HMS notification ID as a Sentry gauge (metric)
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  bbfa713262c5b4d5cecccd95c823a78c9149752c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/SentryMetrics.java
>  6ed2c781b4ac8819f6d17a249c33e32f0116e15a 
> 
> 
> Diff: https://reviews.apache.org/r/58069/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58164: SENTRY-1638 Update SQL script of MSentryPathChange table to add a column for notification ID

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58164/#review170981
---




sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.oracle.sql
Lines 1 (patched)
<https://reviews.apache.org/r/58164/#comment243815>

Why create a new file for create notification id field  inside PATHChange 
table? It should be ok to just add upon the 008-SENTRY-1569.xxx.sql as there is 
no release yet.


- Hao Hao


On April 4, 2017, 3:57 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58164/
> ---
> 
> (Updated April 4, 2017, 3:57 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, and Na Li.
> 
> 
> Bugs: SENTRY-1638
> https://issues.apache.org/jira/browse/SENTRY-1638
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> UpdateD SQL script of MSentryPathChange table to add a column for 
> notification ID. Wich include changes to jdo defination and also 
> MSentryPermChange class.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPermChange.java
>  476fbcb2ad26de23757842111beb12b154e1562b 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  e981bcf0f327346c09cdbe5785fb8824fc62e704 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.mysql.sql
>  1829e2fa1f02a4339e7af4bf45a169013e9ec65f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.derby.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.mysql.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.oracle.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.postgres.sql
>  PRE-CREATION 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 547bbe8136186658e7fe76ab24934157ea5300ff 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 6474389a18ea59da28d3d7125cf227c7aaa7f7aa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 1ab83432db233ee4e7aa054adc1b82c26248a099 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 0418b298f7cbd8f430733cb329e9fca263bda0f7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  68d2c8d53d7c468269ca2c41986ef8651b94f5c7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  5376c166659b3e8e373c8c20818f5b1290af90c9 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-derby-1.7.0-to-1.8.0.sql
>  dc3021a57ac9ea8145379c437ae6a94131c9ae5e 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-mysql-1.7.0-to-1.8.0.sql
>  ef1c651da8998eb57b5d0ca3463317d07c1b3a24 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-oracle-1.7.0-to-1.8.0.sql
>  f4a50de14dd0f961befaded394c6a01c2f459317 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-postgres-1.7.0-to-1.8.0.sql
>  db010bcd7792c9370478a2061e22d14bb78e3184 
> 
> 
> Diff: https://reviews.apache.org/r/58164/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58164: SENTRY-1638 Update SQL script of MSentryPathChange table to add a column for notification ID

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58164/#review170980
---




sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPermChange.java
Line 30 (original), 30 (patched)
<https://reviews.apache.org/r/58164/#comment243813>

I don't think we need notification ID on PermChange table? Any reason to do 
otherwise?


- Hao Hao


On April 4, 2017, 3:57 p.m., kalyan kumar kalvagadda wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58164/
> ---
> 
> (Updated April 4, 2017, 3:57 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov, Hao Hao, and Na Li.
> 
> 
> Bugs: SENTRY-1638
> https://issues.apache.org/jira/browse/SENTRY-1638
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> UpdateD SQL script of MSentryPathChange table to add a column for 
> notification ID. Wich include changes to jdo defination and also 
> MSentryPermChange class.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryPermChange.java
>  476fbcb2ad26de23757842111beb12b154e1562b 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/package.jdo
>  e981bcf0f327346c09cdbe5785fb8824fc62e704 
>   
> sentry-provider/sentry-provider-db/src/main/resources/007-SENTRY-1365.mysql.sql
>  1829e2fa1f02a4339e7af4bf45a169013e9ec65f 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.derby.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.mysql.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.oracle.sql
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/resources/008-SENTRY-1638.postgres.sql
>  PRE-CREATION 
>   sentry-provider/sentry-provider-db/src/main/resources/sentry-db2-1.8.0.sql 
> 547bbe8136186658e7fe76ab24934157ea5300ff 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-derby-1.8.0.sql 
> 6474389a18ea59da28d3d7125cf227c7aaa7f7aa 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-mysql-1.8.0.sql 
> 1ab83432db233ee4e7aa054adc1b82c26248a099 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-oracle-1.8.0.sql 
> 0418b298f7cbd8f430733cb329e9fca263bda0f7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql
>  68d2c8d53d7c468269ca2c41986ef8651b94f5c7 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-db2-1.7.0-to-1.8.0.sql
>  5376c166659b3e8e373c8c20818f5b1290af90c9 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-derby-1.7.0-to-1.8.0.sql
>  dc3021a57ac9ea8145379c437ae6a94131c9ae5e 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-mysql-1.7.0-to-1.8.0.sql
>  ef1c651da8998eb57b5d0ca3463317d07c1b3a24 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-oracle-1.7.0-to-1.8.0.sql
>  f4a50de14dd0f961befaded394c6a01c2f459317 
>   
> sentry-provider/sentry-provider-db/src/main/resources/sentry-upgrade-postgres-1.7.0-to-1.8.0.sql
>  db010bcd7792c9370478a2061e22d14bb78e3184 
> 
> 
> Diff: https://reviews.apache.org/r/58164/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> kalyan kumar kalvagadda
> 
>

Re: Review Request 58069: SENTRY-1670 Expose current HMS notification ID as a Sentry gauge (metric)

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58069/#review170958
---




sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
Line 3167 (original), 3187 (patched)
<https://reviews.apache.org/r/58069/#comment243798>

Why do we change from 'Long' to 'long' here? it would result in extra unbox 
and autobox.


- Hao Hao


On March 30, 2017, 5:25 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58069/
> ---
> 
> (Updated March 30, 2017, 5:25 a.m.)
> 
> 
> Review request for sentry, Hao Hao and kalyan kumar kalvagadda.
> 
> 
> Bugs: SENTRY-1670
> https://issues.apache.org/jira/browse/SENTRY-1670
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1670 Expose current HMS notification ID as a Sentry gauge (metric)
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  bbfa713262c5b4d5cecccd95c823a78c9149752c 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/SentryMetrics.java
>  6ed2c781b4ac8819f6d17a249c33e32f0116e15a 
> 
> 
> Diff: https://reviews.apache.org/r/58069/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58166: SENTRY-1692 ZK namespace configuration doesn't work

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58166/#review170957
---




sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/LeaderStatusMonitor.java
Line 96 (original)
<https://reviews.apache.org/r/58166/#comment243797>

Not quite follow the change. If we are complete remove namespace here, why 
do we still keep the namespace config in HAContext? And why do we need it?


- Hao Hao


On April 4, 2017, 5:03 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58166/
> ---
> 
> (Updated April 4, 2017, 5:03 a.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, and Na Li.
> 
> 
> Bugs: SENTRY-1692
> https://issues.apache.org/jira/browse/SENTRY-1692
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1692 ZK namespace configuration doesn't work
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/HAContext.java
>  00eec4eab066829ae1226d0ba3485eab7bd9eeb2 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/LeaderStatusMonitor.java
>  1d79bd1f722505e56941dfd131d782e518515dc4 
> 
> 
> Diff: https://reviews.apache.org/r/58166/diff/1/
> 
> 
> Testing
> ---
> 
> I tested this manually by running the server with namespace provided in the 
> config.
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58094: SENTRY-1683 MetastoreCacheInitializer has a race condition in handling results list

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58094/#review170796
---


Fix it, then Ship it!





sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java
Line 364 (original), 362 (patched)
<https://reviews.apache.org/r/58094/#comment243671>

Could it ever be negative?


- Hao Hao


On March 31, 2017, 10:36 p.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58094/
> ---
> 
> (Updated March 31, 2017, 10:36 p.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, Na Li, and Vadim 
> Spector.
> 
> 
> Bugs: SENTRY-1683
> https://issues.apache.org/jira/browse/SENTRY-1683
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1683 MetastoreCacheInitializer has a race condition in handling 
> results list
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java
>  f9664f02d076f03a6481adbac24cefd72e90e152 
> 
> 
> Diff: https://reviews.apache.org/r/58094/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 58094: SENTRY-1683 MetastoreCacheInitializer has a race condition in handling results list

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58094/#review170792
---




sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java
Line 301 (original), 301 (patched)
<https://reviews.apache.org/r/58094/#comment243667>

Why do we need synchronized here anympre as we are using thread safe vector?


- Hao Hao


On March 31, 2017, 4:21 a.m., Alexander Kolbasov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58094/
> ---
> 
> (Updated March 31, 2017, 4:21 a.m.)
> 
> 
> Review request for sentry, Hao Hao, kalyan kumar kalvagadda, Na Li, and Vadim 
> Spector.
> 
> 
> Bugs: SENTRY-1683
> https://issues.apache.org/jira/browse/SENTRY-1683
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> SENTRY-1683 MetastoreCacheInitializer has a race condition in handling 
> results list
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastoreCacheInitializer.java
>  f9664f02d076f03a6481adbac24cefd72e90e152 
> 
> 
> Diff: https://reviews.apache.org/r/58094/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Alexander Kolbasov
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/
---

(Updated March 24, 2017, 1:12 a.m.)


Review request for sentry.


Repository: sentry


Description
---

Add the logic for propagating Sentry Permissions or Sentry representation of 
HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
service, to a Sentry consumer such as HDFS NameNode. It includes:
1. Delta update retrieval logic from persistent storage.
2. Propagation logic for consumer to get the delta update or a complete 
snapshot.


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
 0e40756b0da0214e83b7127dcd8816f30c9692c8 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
 69aa098bc71362b1aba0ad219483cfce7d389964 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
 16ffa1b530a47b425333ada047d2a7ec96d260ef 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
 16a1604700079cb37f6fbaeceae92c67b2c08d8b 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
 3017c9edb2217cd0152183f927691311a276e850 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
 e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
 be145695af35b34f409e1f8b6e49da247f78e7a0 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
 029f9d5dc89493f7e7086b80387f71fef1ac0805 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
 22c57690a0f74822187335f97cf1eede8d415acf 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
 03c67d674813c59a23c5bc6c57671d79dfd235a0 
  
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
 d12b1342116217685a46367567e2c0f8d7c288e5 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 75f855caec1060f9158562aeb81ec53d8a679941 
  
sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
 5e8b2fac76d6dcf7aee34729e86755797af64406 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java
 1b5eb537ec780e4b2444a7b6744ed4aa75da2d08 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 859c8f882db76e46297c1f6377de32bc2b08f485 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java
 c7912726cda157761753b71d43cfb272ecc5e8c4 


Diff: https://reviews.apache.org/r/57232/diff/9/

Changes: https://reviews.apache.org/r/57232/diff/8-9/


Testing
---


Thanks,

Hao Hao

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 23, 2017, 1:34 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
> > Lines 49 (patched)
> > <https://reviews.apache.org/r/57232/diff/7/?file=1672264#file1672264line49>
> >
> > I had this comment before - it is cleaner to be explicit whether we 
> > return a list of deltas or a full update - either by using different 
> > functions or returning a special result.

You mean change the interface or be more specific in the comments?


> On March 23, 2017, 1:34 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
> > Line 49 (original), 49 (patched)
> > <https://reviews.apache.org/r/57232/diff/7/?file=1672272#file1672272line49>
> >
> > Are we still using SentryPlugin?

Yes, we are still using it when HDFS sync feature is on.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review169817
-----------


On March 22, 2017, 11:08 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 22, 2017, 11:08 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  0e40756b0da0214e83b7127dcd8816f30c9692c8 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  085971b34e3901b7a1d59bd8e7516b25f81ca872 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  16a1604700079cb37f6fbaeceae92c67b2c08d8b 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  3017c9edb2217cd0152183f927691311a276e850 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  be145695af35b34f409e1f8b6e49da247f78e7a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  029f9d5dc89493f7e7086b80387f71fef1ac0805 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  22c57690a0f74822187335f97cf1eede8d415acf 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  03c67d674813c59a23c5bc6c57671d79dfd235a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  d12b1342116217685a46367567e2c0f8d7c288e5 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  75f855caec1060f9158562aeb81ec53d8a679941 
>   
> sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
>  5e8b2fac76d6dcf7aee34729e86755797af64406 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  859c8f882db76e46297c1f6377de32bc2b08f485 
> 
> 
> Diff: https://reviews.apache.org/r/57232/diff/7/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 23, 2017, 1:34 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
> > Lines 42 (patched)
> > <https://reviews.apache.org/r/57232/diff/7/?file=1672266#file1672266line42>
> >
> > Would it make sense to add a metric to measure time to get it? If you 
> > don't want to add it now, please don't forget to file a follow-up JIRA.
> > 
> > Same for PermDeltaRetriever.

Filed https://issues.apache.org/jira/browse/SENTRY-1677/.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review169817
-----------


On March 24, 2017, 12:53 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 24, 2017, 12:53 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  0e40756b0da0214e83b7127dcd8816f30c9692c8 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  16ffa1b530a47b425333ada047d2a7ec96d260ef 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  16a1604700079cb37f6fbaeceae92c67b2c08d8b 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  3017c9edb2217cd0152183f927691311a276e850 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  be145695af35b34f409e1f8b6e49da247f78e7a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  029f9d5dc89493f7e7086b80387f71fef1ac0805 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  22c57690a0f74822187335f97cf1eede8d415acf 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  03c67d674813c59a23c5bc6c57671d79dfd235a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  d12b1342116217685a46367567e2c0f8d7c288e5 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  75f855caec1060f9158562aeb81ec53d8a679941 
>   
> sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
>  5e8b2fac76d6dcf7aee34729e86755797af64406 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  859c8f882db76e46297c1f6377de32bc2b08f485 
> 
> 
> Diff: https://reviews.apache.org/r/57232/diff/8/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 23, 2017, 1:34 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
> > Lines 77 (patched)
> > <https://reviews.apache.org/r/57232/diff/7/?file=1672267#file1672267line78>
> >
> > Since there is no concurrency here, why should we create a new lock 
> > each time? A single lock would work as well.

I feel it is better to create the lock every time instead of use a single lock 
at class level. We do not want to block the process of paths update which 
belongs to different threads.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review169817
-------


On March 22, 2017, 11:08 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 22, 2017, 11:08 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  0e40756b0da0214e83b7127dcd8816f30c9692c8 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  085971b34e3901b7a1d59bd8e7516b25f81ca872 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  16a1604700079cb37f6fbaeceae92c67b2c08d8b 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  3017c9edb2217cd0152183f927691311a276e850 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  be145695af35b34f409e1f8b6e49da247f78e7a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  029f9d5dc89493f7e7086b80387f71fef1ac0805 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  22c57690a0f74822187335f97cf1eede8d415acf 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  03c67d674813c59a23c5bc6c57671d79dfd235a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  d12b1342116217685a46367567e2c0f8d7c288e5 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  75f855caec1060f9158562aeb81ec53d8a679941 
>   
> sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
>  5e8b2fac76d6dcf7aee34729e86755797af64406 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  859c8f882db76e46297c1f6377de32bc2b08f485 
> 
> 
> Diff: https://reviews.apache.org/r/57232/diff/7/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/
---

(Updated March 24, 2017, 12:53 a.m.)


Review request for sentry.


Repository: sentry


Description
---

Add the logic for propagating Sentry Permissions or Sentry representation of 
HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
service, to a Sentry consumer such as HDFS NameNode. It includes:
1. Delta update retrieval logic from persistent storage.
2. Propagation logic for consumer to get the delta update or a complete 
snapshot.


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
 0e40756b0da0214e83b7127dcd8816f30c9692c8 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
 69aa098bc71362b1aba0ad219483cfce7d389964 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
 16ffa1b530a47b425333ada047d2a7ec96d260ef 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
 16a1604700079cb37f6fbaeceae92c67b2c08d8b 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
 3017c9edb2217cd0152183f927691311a276e850 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
 e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
 be145695af35b34f409e1f8b6e49da247f78e7a0 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
 029f9d5dc89493f7e7086b80387f71fef1ac0805 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
 22c57690a0f74822187335f97cf1eede8d415acf 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
 03c67d674813c59a23c5bc6c57671d79dfd235a0 
  
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
 d12b1342116217685a46367567e2c0f8d7c288e5 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 75f855caec1060f9158562aeb81ec53d8a679941 
  
sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
 5e8b2fac76d6dcf7aee34729e86755797af64406 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 859c8f882db76e46297c1f6377de32bc2b08f485 


Diff: https://reviews.apache.org/r/57232/diff/8/

Changes: https://reviews.apache.org/r/57232/diff/7-8/


Testing
---


Thanks,

Hao Hao

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/
---

(Updated March 22, 2017, 11:08 p.m.)


Review request for sentry.


Repository: sentry


Description
---

Add the logic for propagating Sentry Permissions or Sentry representation of 
HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
service, to a Sentry consumer such as HDFS NameNode. It includes:
1. Delta update retrieval logic from persistent storage.
2. Propagation logic for consumer to get the delta update or a complete 
snapshot.


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
 0e40756b0da0214e83b7127dcd8816f30c9692c8 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
 69aa098bc71362b1aba0ad219483cfce7d389964 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
 085971b34e3901b7a1d59bd8e7516b25f81ca872 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
 16a1604700079cb37f6fbaeceae92c67b2c08d8b 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
 3017c9edb2217cd0152183f927691311a276e850 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
 e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
 be145695af35b34f409e1f8b6e49da247f78e7a0 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
 029f9d5dc89493f7e7086b80387f71fef1ac0805 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
 22c57690a0f74822187335f97cf1eede8d415acf 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
 03c67d674813c59a23c5bc6c57671d79dfd235a0 
  
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
 d12b1342116217685a46367567e2c0f8d7c288e5 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 75f855caec1060f9158562aeb81ec53d8a679941 
  
sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
 5e8b2fac76d6dcf7aee34729e86755797af64406 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 859c8f882db76e46297c1f6377de32bc2b08f485 


Diff: https://reviews.apache.org/r/57232/diff/7/

Changes: https://reviews.apache.org/r/57232/diff/6-7/


Testing
---


Thanks,

Hao Hao

Re: Review Request 57657: SENTRY-1658: Fixed possible null pointer dereference in SentryShellHive

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57657/#review169082
---


Ship it!




Ship It!

- Hao Hao


On March 15, 2017, 6:41 p.m., Jan Hentschel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57657/
> ---
> 
> (Updated March 15, 2017, 6:41 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Bugs: SENTRY-1658
> https://issues.apache.org/jira/browse/SENTRY-1658
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Fixed a possible null pointer dereference in **SentryShellHive**.
> 
> 
> Diffs
> -
> 
>   
> sentry-service/sentry-service-client/src/main/java/org/apache/sentry/provider/db/tools/SentryShellHive.java
>  dc7f829 
> 
> 
> Diff: https://reviews.apache.org/r/57657/diff/1/
> 
> 
> Testing
> ---
> 
> Successful run of **mvn clean install**.
> 
> 
> Thanks,
> 
> Jan Hentschel
> 
>

Re: Review Request 57375: Unable to truncate table .; from "default" databases

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57375/#review169080
---




sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java
Lines 253 (patched)
<https://reviews.apache.org/r/57375/#comment241399>

Could you please add a comment here to explain what is the ast structure 
look like here? And what does it evalutate to be '1' mean here.


- Hao Hao


On March 7, 2017, 1:41 p.m., Yongzhi Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57375/
> ---
> 
> (Updated March 7, 2017, 1:41 p.m.)
> 
> 
> Review request for sentry and Hao Hao.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Please review fix for SENTRY-1646. Fix the issue by getting the currOutDB 
> inform from qualified table name if exists.
> 
> 
> Diffs
> -
> 
>   
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingHook.java
>  d269d5fe0bfd3e8638fcec15d426862f4f15ae35 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestPrivilegesAtTableScopePart2.java
>  8eb0bd69158805a3fbcb6c88d61b26ef922997ef 
> 
> 
> Diff: https://reviews.apache.org/r/57375/diff/1/
> 
> 
> Testing
> ---
> 
> Add the test case in Unit test.
> 
> 
> Thanks,
> 
> Yongzhi Chen
> 
>

Re: Review Request 57654: SENTRY-1663: Moved mutable field in UpdateableAuthzPermissions to immutable

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57654/#review169077
---


Ship it!




Ship It!

- Hao Hao


On March 15, 2017, 5:33 p.m., Jan Hentschel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57654/
> ---
> 
> (Updated March 15, 2017, 5:33 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Bugs: SENTRY-1663
> https://issues.apache.org/jira/browse/SENTRY-1663
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Moved the mutable field **ACTION_MAPPING** in **UpdateableAuthzPermissions** 
> to a immutable field.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/UpdateableAuthzPermissions.java
>  3d3fc8d 
> 
> 
> Diff: https://reviews.apache.org/r/57654/diff/1/
> 
> 
> Testing
> ---
> 
> Successful run of **mvn clean install**.
> 
> 
> Thanks,
> 
> Jan Hentschel
> 
>

Re: Review Request 57635: SENTRY-1660: Removed useless code in ResourceAuthorizationProvider

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57635/#review169076
---


Ship it!




Ship It!

- Hao Hao


On March 15, 2017, 9:29 a.m., Jan Hentschel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57635/
> ---
> 
> (Updated March 15, 2017, 9:29 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Bugs: SENTRY-1660
> https://issues.apache.org/jira/browse/SENTRY-1660
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Removed the reference to the **hierarchy** variable in the 
> **ResourceAuthorizationProvider** class.
> 
> 
> Diffs
> -
> 
>   
> sentry-provider/sentry-authorization-provider/src/main/java/org/apache/sentry/provider/common/ResourceAuthorizationProvider.java
>  1ad292e 
> 
> 
> Diff: https://reviews.apache.org/r/57635/diff/1/
> 
> 
> Testing
> ---
> 
> Successful run of **mvn clean install**.
> 
> 
> Thanks,
> 
> Jan Hentschel
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/
---

(Updated March 14, 2017, 8:56 p.m.)


Review request for sentry.


Repository: sentry


Description
---

Add the logic for propagating Sentry Permissions or Sentry representation of 
HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
service, to a Sentry consumer such as HDFS NameNode. It includes:
1. Delta update retrieval logic from persistent storage.
2. Propagation logic for consumer to get the delta update or a complete 
snapshot.


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
 0e40756b0da0214e83b7127dcd8816f30c9692c8 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
 69aa098bc71362b1aba0ad219483cfce7d389964 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
 085971b34e3901b7a1d59bd8e7516b25f81ca872 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
 16a1604700079cb37f6fbaeceae92c67b2c08d8b 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
 3017c9edb2217cd0152183f927691311a276e850 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
 e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
 be145695af35b34f409e1f8b6e49da247f78e7a0 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
 029f9d5dc89493f7e7086b80387f71fef1ac0805 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
 22c57690a0f74822187335f97cf1eede8d415acf 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
 03c67d674813c59a23c5bc6c57671d79dfd235a0 
  
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
 d12b1342116217685a46367567e2c0f8d7c288e5 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 75f855caec1060f9158562aeb81ec53d8a679941 
  
sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
 5e8b2fac76d6dcf7aee34729e86755797af64406 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 f4fc8e3c1116a6adeaa685f75149e65259bb5014 


Diff: https://reviews.apache.org/r/57232/diff/6/

Changes: https://reviews.apache.org/r/57232/diff/5-6/


Testing
---


Thanks,

Hao Hao

Re: Review Request 57570: SENTRY-1644. Partition ACLs disappear after renaming Hive table with partitions

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57570/#review168939
---



Can you add test cases in the intergation test as well, 
TestHDFSIntegrationAdvanced. To ensure cases such as for managed/unmanaged 
table we get expected output.
Also, it looks like the tests are not all green. Would you mind take a look if 
anything related?

- Hao Hao


On March 13, 2017, 9:57 p.m., Lei Xu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57570/
> ---
> 
> (Updated March 13, 2017, 9:57 p.m.)
> 
> 
> Review request for sentry, Alexander Kolbasov and Hao Hao.
> 
> 
> Bugs: SENTRY-1644
> https://issues.apache.org/jira/browse/SENTRY-1644
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> After rename Hive table with partitions, ACLs on the partition paths 
> disappear.
> 
> It is caused by the fact that renaming on sentry does not move any 
> sub-namespace.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
>  e4850f4ee039f011de370ca6b2618161150645de 
>   
> sentry-hdfs/sentry-hdfs-common/src/test/java/org/apache/sentry/hdfs/TestHMSPaths.java
>  0dee1028ac852b12b109e875b2cbbb0bd2efbe03 
> 
> 
> Diff: https://reviews.apache.org/r/57570/diff/1/
> 
> 
> Testing
> ---
> 
> mvn test -Dtest=TestHMSPaths
> 
> 
> Thanks,
> 
> Lei Xu
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 14, 2017, 12:48 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
> > Lines 77 (patched)
> > <https://reviews.apache.org/r/57232/diff/4/?file=1661861#file1661861line78>
> >
> > Why are we using a new lock every time? How does it protect us from 
> > anything? The lock is only useful if others are using the same lock. What 
> > are you trying to synchronize here?

Here as the UpdateableAuthzPaths is local variable. There is not actual 
anything to syncgronize. It is simply because updatePartial requires a lock..


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review168846
-----------


On March 14, 2017, 6:59 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 14, 2017, 6:59 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  0e40756b0da0214e83b7127dcd8816f30c9692c8 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  085971b34e3901b7a1d59bd8e7516b25f81ca872 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  16a1604700079cb37f6fbaeceae92c67b2c08d8b 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  3017c9edb2217cd0152183f927691311a276e850 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  be145695af35b34f409e1f8b6e49da247f78e7a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  029f9d5dc89493f7e7086b80387f71fef1ac0805 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  22c57690a0f74822187335f97cf1eede8d415acf 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  03c67d674813c59a23c5bc6c57671d79dfd235a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  d12b1342116217685a46367567e2c0f8d7c288e5 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  75f855caec1060f9158562aeb81ec53d8a679941 
>   
> sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
>  5e8b2fac76d6dcf7aee34729e86755797af64406 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  f4fc8e3c1116a6adeaa685f75149e65259bb5014 
> 
> 
> Diff: https://reviews.apache.org/r/57232/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 14, 2017, 12:48 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
> > Lines 59 (patched)
> > <https://reviews.apache.org/r/57232/diff/4/?file=1661858#file1661858line59>
> >
> > Can we relax the return type to Collection?
> > This method can be package-private

I do not see the needs for change it Collection?


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review168846
-----------


On March 14, 2017, 6:59 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 14, 2017, 6:59 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  0e40756b0da0214e83b7127dcd8816f30c9692c8 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  085971b34e3901b7a1d59bd8e7516b25f81ca872 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  16a1604700079cb37f6fbaeceae92c67b2c08d8b 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  3017c9edb2217cd0152183f927691311a276e850 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  be145695af35b34f409e1f8b6e49da247f78e7a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  029f9d5dc89493f7e7086b80387f71fef1ac0805 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  22c57690a0f74822187335f97cf1eede8d415acf 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  03c67d674813c59a23c5bc6c57671d79dfd235a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  d12b1342116217685a46367567e2c0f8d7c288e5 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
>  f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  75f855caec1060f9158562aeb81ec53d8a679941 
>   
> sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
>  5e8b2fac76d6dcf7aee34729e86755797af64406 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
>  f4fc8e3c1116a6adeaa685f75149e65259bb5014 
> 
> 
> Diff: https://reviews.apache.org/r/57232/diff/5/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/
---

(Updated March 14, 2017, 6:59 p.m.)


Review request for sentry.


Repository: sentry


Description
---

Add the logic for propagating Sentry Permissions or Sentry representation of 
HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
service, to a Sentry consumer such as HDFS NameNode. It includes:
1. Delta update retrieval logic from persistent storage.
2. Propagation logic for consumer to get the delta update or a complete 
snapshot.


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
 0e40756b0da0214e83b7127dcd8816f30c9692c8 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
 69aa098bc71362b1aba0ad219483cfce7d389964 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
 085971b34e3901b7a1d59bd8e7516b25f81ca872 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
 16a1604700079cb37f6fbaeceae92c67b2c08d8b 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
 3017c9edb2217cd0152183f927691311a276e850 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
 e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
 be145695af35b34f409e1f8b6e49da247f78e7a0 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
 029f9d5dc89493f7e7086b80387f71fef1ac0805 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
 22c57690a0f74822187335f97cf1eede8d415acf 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
 03c67d674813c59a23c5bc6c57671d79dfd235a0 
  
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
 d12b1342116217685a46367567e2c0f8d7c288e5 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java
 f3f51dabdd610fbf6095a9d0494de78ee8f3ded7 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 75f855caec1060f9158562aeb81ec53d8a679941 
  
sentry-tests/sentry-tests-hive-v2/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
 5e8b2fac76d6dcf7aee34729e86755797af64406 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java
 f4fc8e3c1116a6adeaa685f75149e65259bb5014 


Diff: https://reviews.apache.org/r/57232/diff/5/

Changes: https://reviews.apache.org/r/57232/diff/4-5/


Testing
---


Thanks,

Hao Hao

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 14, 2017, 12:48 a.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 3406 (patched)
> > <https://reviews.apache.org/r/57232/diff/4/?file=1661870#file1661870line3407>
> >
> > Collections.emptyList()
> > 
> > Can we recover from this condition?

Manual recover would be delete all the deltas before the given changeID in 
database. Otherwise, auto recover would be whenever the purge delta thread 
takes place.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review168846
-----------


On March 10, 2017, 11:14 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 10, 2017, 11:14 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  0e40756b0da0214e83b7127dcd8816f30c9692c8 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  085971b34e3901b7a1d59bd8e7516b25f81ca872 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  16a1604700079cb37f6fbaeceae92c67b2c08d8b 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  3017c9edb2217cd0152183f927691311a276e850 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  be145695af35b34f409e1f8b6e49da247f78e7a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  029f9d5dc89493f7e7086b80387f71fef1ac0805 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  22c57690a0f74822187335f97cf1eede8d415acf 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  03c67d674813c59a23c5bc6c57671d79dfd235a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  d12b1342116217685a46367567e2c0f8d7c288e5 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  75f855caec1060f9158562aeb81ec53d8a679941 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
>  f2d74bfe5e413ea88c3c77ef303f1c7f9ee16253 
> 
> 
> Diff: https://reviews.apache.org/r/57232/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 14, 2017, 12:48 a.m., Alexander Kolbasov wrote:
> > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
> > Lines 3298 (patched)
> > <https://reviews.apache.org/r/57232/diff/4/?file=1661870#file1661870line3299>
> >
> > Since we only need to know whether there are any changes, can we 
> > optimize query somewhat?

Do you have any suggestions?


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review168846
-------


On March 10, 2017, 11:14 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 10, 2017, 11:14 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  0e40756b0da0214e83b7127dcd8816f30c9692c8 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  085971b34e3901b7a1d59bd8e7516b25f81ca872 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  16a1604700079cb37f6fbaeceae92c67b2c08d8b 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  3017c9edb2217cd0152183f927691311a276e850 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  be145695af35b34f409e1f8b6e49da247f78e7a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  029f9d5dc89493f7e7086b80387f71fef1ac0805 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  22c57690a0f74822187335f97cf1eede8d415acf 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  03c67d674813c59a23c5bc6c57671d79dfd235a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  d12b1342116217685a46367567e2c0f8d7c288e5 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  75f855caec1060f9158562aeb81ec53d8a679941 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
>  f2d74bfe5e413ea88c3c77ef303f1c7f9ee16253 
> 
> 
> Diff: https://reviews.apache.org/r/57232/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 14, 2017, 12:48 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
> > Lines 30 (patched)
> > <https://reviews.apache.org/r/57232/diff/4/?file=1661858#file1661858line30>
> >
> > The first sentence should be a brief description without links.

Yeah, as you suggested last code review I changed it to @code instead of @link.


> On March 14, 2017, 12:48 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
> > Lines 27 (patched)
> > <https://reviews.apache.org/r/57232/diff/4/?file=1661860#file1661860line27>
> >
> > The first sentence should be a short statement without links. 
> > Subsequent sentences may clarify it and add links.

same comment as above. using @code as you suggested.


> On March 14, 2017, 12:48 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
> > Lines 27 (patched)
> > <https://reviews.apache.org/r/57232/diff/4/?file=1661862#file1661862line27>
> >
> > The first sentence should be a short description without links.

same comment as above.


> On March 14, 2017, 12:48 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
> > Line 33 (original), 33 (patched)
> > <https://reviews.apache.org/r/57232/diff/4/?file=1661863#file1661863line33>
> >
> > Same comment about first javadoc sentence.

Same reply as above.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review168846
---


On March 10, 2017, 11:14 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 10, 2017, 11:14 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  0e40756b0da0214e83b7127dcd8816f30c9692c8 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  085971b34e3901b7a1d59bd8e7516b25f81ca872 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  16a1604700079cb37f6fbaeceae92c67b2c08d8b 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  3017c9edb2217cd0152183f927691311a276e850 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  be145695af35b34f409e1f8b6e49da247f78e7a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  029f9d5dc89493f7e7086b80387f71fef1ac0805 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  22c57690a0f74822187335f97cf1eede8d415acf 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  03c67d674813c59a23c5bc6c57671d79dfd235a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/
---

(Updated March 10, 2017, 11:14 p.m.)


Review request for sentry.


Repository: sentry


Description
---

Add the logic for propagating Sentry Permissions or Sentry representation of 
HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
service, to a Sentry consumer such as HDFS NameNode. It includes:
1. Delta update retrieval logic from persistent storage.
2. Propagation logic for consumer to get the delta update or a complete 
snapshot.


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
 0e40756b0da0214e83b7127dcd8816f30c9692c8 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
 69aa098bc71362b1aba0ad219483cfce7d389964 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
 085971b34e3901b7a1d59bd8e7516b25f81ca872 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
 16a1604700079cb37f6fbaeceae92c67b2c08d8b 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
 3017c9edb2217cd0152183f927691311a276e850 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
 e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
 be145695af35b34f409e1f8b6e49da247f78e7a0 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
 029f9d5dc89493f7e7086b80387f71fef1ac0805 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
 22c57690a0f74822187335f97cf1eede8d415acf 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
 03c67d674813c59a23c5bc6c57671d79dfd235a0 
  
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
 d12b1342116217685a46367567e2c0f8d7c288e5 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 75f855caec1060f9158562aeb81ec53d8a679941 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
 f2d74bfe5e413ea88c3c77ef303f1c7f9ee16253 


Diff: https://reviews.apache.org/r/57232/diff/4/

Changes: https://reviews.apache.org/r/57232/diff/3-4/


Testing
---


Thanks,

Hao Hao

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/
---

(Updated March 10, 2017, 10:29 p.m.)


Review request for sentry.


Repository: sentry


Description
---

Add the logic for propagating Sentry Permissions or Sentry representation of 
HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
service, to a Sentry consumer such as HDFS NameNode. It includes:
1. Delta update retrieval logic from persistent storage.
2. Propagation logic for consumer to get the delta update or a complete 
snapshot.


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
 0e40756b0da0214e83b7127dcd8816f30c9692c8 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
 69aa098bc71362b1aba0ad219483cfce7d389964 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
 085971b34e3901b7a1d59bd8e7516b25f81ca872 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
 16a1604700079cb37f6fbaeceae92c67b2c08d8b 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
 3017c9edb2217cd0152183f927691311a276e850 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
 e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
 be145695af35b34f409e1f8b6e49da247f78e7a0 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
 029f9d5dc89493f7e7086b80387f71fef1ac0805 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
 22c57690a0f74822187335f97cf1eede8d415acf 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
 03c67d674813c59a23c5bc6c57671d79dfd235a0 
  
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
 d12b1342116217685a46367567e2c0f8d7c288e5 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 6ea6d3f44d7a2c7cd2b375af74f3b19d5731aed6 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 75f855caec1060f9158562aeb81ec53d8a679941 
  
sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
 f2d74bfe5e413ea88c3c77ef303f1c7f9ee16253 


Diff: https://reviews.apache.org/r/57232/diff/3/

Changes: https://reviews.apache.org/r/57232/diff/2-3/


Testing
---


Thanks,

Hao Hao

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

[Hao Hao]

> On March 6, 2017, 10:38 p.m., Lei Xu wrote:
> > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
> > Lines 35 (patched)
> > <https://reviews.apache.org/r/57232/diff/2/?file=1653451#file1653451line35>
> >
> > If we retrieve `Perm` and `Path` separately, would that be a case that 
> > the deltas from them are not consistent, because the `latestChangeID` might 
> > be different in them?
> > 
> > For example, if we do the following sequence:
> > 
> > * Retrieve path deltas.
> > * Create table
> > * Grant permissions on the table.
> > * Retrieve perm deltas.
> > 
> > In such order, would some perm deltas can not be applied to the path, 
> > as they do not exist on NN yet?

Perm and Path deltas do not share the same changeID, and cannot be correlated. 
They are triggered by two seperate streams of events. NN periodically pull 
these two kind of deltas every configurable of time(default is 0.5 seconds). 
During that time, as you described Namenode cache could be in inconsistent. But 
after that time it should be consistent. This problem will still exist even we 
put those two kinds of update in one.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57232/#review168057
---


On March 2, 2017, 7:39 a.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57232/
> ---
> 
> (Updated March 2, 2017, 7:39 a.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add the logic for propagating Sentry Permissions or Sentry representation of 
> HMS Paths, from a persistent storage instead of in memoery cache of Sentry 
> service, to a Sentry consumer such as HDFS NameNode. It includes:
> 1. Delta update retrieval logic from persistent storage.
> 2. Propagation logic for consumer to get the delta update or a complete 
> snapshot.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/DeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ThriftSerializer.java
>  69aa098bc71362b1aba0ad219483cfce7d389964 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/MetastorePlugin.java
>  085971b34e3901b7a1d59bd8e7516b25f81ca872 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermDeltaRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHDFSServiceProcessor.java
>  e4f3f580ee0a6bf83d5a32001bff130c0bff13aa 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  e68c7080cec007e3cdf2c89efb9701eef9cca0a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  f68f690436301c79e93b0742996dda25a750c0c3 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  6d5c607273bb08597780b655d7b59cd41f0844bb 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  fe2baa6a446874185e8344bb16d76d803826d1f3 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  0f0d0a743d5a5f9f1678ac7c5217b9a27537e85b 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  c1186ba405a05be70336e9169a1454208df016ca 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  91f15c0f3999a25a20dfb45f2f28abbda54366f4 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegration.java
>  f2d74bfe5e413ea88c3c77ef303f1c7f9ee16253 
> 
> 
> Diff: https://reviews.apache.org/r/57232/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>

Re: Review Request 55706: SENTRY-1566: Make full Perm/Path snapshot available for NN plugin

[Hao Hao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55706/
---

(Updated March 9, 2017, 11:43 p.m.)


Review request for sentry.


Repository: sentry


Description
---

Read full permission and path snapshot from SentryDB and make the update 
available for NN plugin upon requests.
In detail:
1. Added Path/PermissionImage classes to represent corresponding 
Path/Permission snapshot read from DB.
2. Refactor full snapshot retriever APIs in SentryStore to become a single 
transaction to ensure snapshot consistency.
3. Added Path/PermissionImageRetriever classes to retrieve Path/PermissionImage 
from DB and convert to corresponding Path/PermissionUpdate, which later would 
be consumed by NN plugin.


Diffs (updated)
-

  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
 992c8b731810297be847e3b802698332aefe8f8f 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
 PRE-CREATION 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
 e68c7080cec007e3cdf2c89efb9701eef9cca0a0 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
 f68f690436301c79e93b0742996dda25a750c0c3 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
 6d5c607273bb08597780b655d7b59cd41f0844bb 
  
sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
 fe2baa6a446874185e8344bb16d76d803826d1f3 
  
sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
 0f0d0a743d5a5f9f1678ac7c5217b9a27537e85b 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/PathsImage.java
 PRE-CREATION 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/PermissionsImage.java
 PRE-CREATION 
  
sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
 a5a835e7d463e4de95ef537f76f2872105278c84 
  
sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
 91f15c0f3999a25a20dfb45f2f28abbda54366f4 


Diff: https://reviews.apache.org/r/55706/diff/5/

Changes: https://reviews.apache.org/r/55706/diff/4-5/


Testing (updated)
---

Unit test added in TestSentryStore. End to end can be peroformed after 
following changes in Sentry-1613 and Sentry-1587.


Thanks,

Hao Hao

Re: Review Request 55706: SENTRY-1566: Make full Perm/Path snapshot available for NN plugin

[Hao Hao]

> On March 8, 2017, 6:05 a.m., Alexander Kolbasov wrote:
> > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
> > Lines 36 (patched)
> > <https://reviews.apache.org/r/55706/diff/4/?file=1654005#file1654005line36>
> >
> > What is seqNum?

This seqNum is a legacy field because of old design. I am going to remove it in 
next patch: Sentry-1613.


- Hao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55706/#review168247
-------


On March 2, 2017, 9:43 p.m., Hao Hao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55706/
> ---
> 
> (Updated March 2, 2017, 9:43 p.m.)
> 
> 
> Review request for sentry.
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Read full permission and path snapshot from SentryDB and make the update 
> available for NN plugin upon requests.
> In detail:
> 1. Added Path/PermissionImage classes to represent corresponding 
> Path/Permission snapshot read from DB.
> 2. Refactor full snapshot retriever APIs in SentryStore to become a single 
> transaction to ensure snapshot consistency.
> 3. Added Path/PermissionImageRetriever classes to retrieve 
> Path/PermissionImage from DB and convert to corresponding 
> Path/PermissionUpdate, which later would be consumed by NN plugin.
> 
> 
> Diffs
> -
> 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ImageRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/PathsUpdate.java
>  992c8b731810297be847e3b802698332aefe8f8f 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PathImageRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/PermImageRetriever.java
>  PRE-CREATION 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryHdfsMetricsUtil.java
>  e68c7080cec007e3cdf2c89efb9701eef9cca0a0 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java
>  f68f690436301c79e93b0742996dda25a750c0c3 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateForwarder.java
>  6d5c607273bb08597780b655d7b59cd41f0844bb 
>   
> sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/UpdateablePermissions.java
>  fe2baa6a446874185e8344bb16d76d803826d1f3 
>   
> sentry-hdfs/sentry-hdfs-service/src/test/java/org/apache/sentry/hdfs/TestUpdateForwarder.java
>  0f0d0a743d5a5f9f1678ac7c5217b9a27537e85b 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/PathsImage.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/PermissionsImage.java
>  PRE-CREATION 
>   
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java
>  c1186ba405a05be70336e9169a1454208df016ca 
>   
> sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java
>  91f15c0f3999a25a20dfb45f2f28abbda54366f4 
> 
> 
> Diff: https://reviews.apache.org/r/55706/diff/4/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hao Hao
> 
>