Re: Question about incrementing IDs for delta changes

2017-06-08 Thread Lei Xu
Hi, Sasha It is still problematic due to the max(change ID) scanning. One possible way to address it is that, in HMSFollower#run(), it ask sentryStore to cache max change IDs of these two tables with in sentryStore itself. So that each time, when HMSFollower wake up, it refresh sentryStore's

Re: Review Request 59862: SENTRY-1795. Delta tables should not have holes.

2017-06-06 Thread Lei Xu
://reviews.apache.org/r/59862/diff/3/ Changes: https://reviews.apache.org/r/59862/diff/2-3/ Testing --- mvn test -Dtest=TestSentryStore#testConcurrentUpdateChanges Passed on my laptop. Thanks, Lei Xu

Re: Review Request 59862: SENTRY-1795. Delta tables should not have holes.

2017-06-06 Thread Lei Xu
> > (PermissionsUpdate) update)); > > ``` Thanks, fixed. - Lei ----------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59862/#review177109 ---

Re: Review Request 59862: SENTRY-1795. Delta tables should not have holes.

2017-06-06 Thread Lei Xu
://reviews.apache.org/r/59862/diff/2/ Changes: https://reviews.apache.org/r/59862/diff/1-2/ Testing --- mvn test -Dtest=TestSentryStore#testConcurrentUpdateChanges Passed on my laptop. Thanks, Lei Xu

Review Request 59862: SENTRY-1795. Delta tables should not have holes.

2017-06-06 Thread Lei Xu
=TestSentryStore#testConcurrentUpdateChanges Passed on my laptop. Thanks, Lei Xu

Re: Review Request 59767: SENTRY-1713. Enable TestHDFSIntegrationEnd2End.testEnd2End

2017-06-02 Thread Lei Xu
- sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java 1ed92eae Diff: https://reviews.apache.org/r/59767/diff/1/ Testing --- mvn test -Dtest=TestHDFSintegrationEnd2End. Thanks, Lei Xu

Review Request 59767: SENTRY-1731. Enable TestHDFSIntegrationEnd2End.testEnd2End

2017-06-02 Thread Lei Xu
://reviews.apache.org/r/59767/diff/1/ Testing --- mvn test -Dtest=TestHDFSintegrationEnd2End. Thanks, Lei Xu

Re: Review Request 59167: SENTRY-1580: Provide pooled client connection model with HA

2017-05-16 Thread Lei Xu
he signature? * can we rename `destroyObject` to `destory`, to on-par with `create()`? sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/transport/TTransportWrapper.java Lines 79 (patched) <https://reviews.apache.org/r/59167/#comment248544> Should we also put the

Re: Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-04-10 Thread Lei Xu
--- Add a new test to conurrently insert changes. mvn test -Dtest=TestSentryStore. Thanks, Lei Xu

Re: Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-04-10 Thread Lei Xu
eviews.apache.org/r/58281/#review171464 ------- On April 10, 2017, 3:47 p.m., Lei Xu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58281/ >

Re: Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-04-10 Thread Lei Xu
to log it rather then print to stdout It is removed in the version 2 patch. - Lei --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58281/#review171494 ---------

Re: Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-04-10 Thread Lei Xu
nerated e-mail. To reply, visit: https://reviews.apache.org/r/58281/#review171464 ------- On April 10, 2017, 12:54 p.m., Lei Xu wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/

Re: Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-04-10 Thread Lei Xu
--- Add a new test to conurrently insert changes. mvn test -Dtest=TestSentryStore. Thanks, Lei Xu

Re: Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-04-10 Thread Lei Xu
ly, visit: https://reviews.apache.org/r/58281/#review171458 ----------- On April 7, 2017, 5:38 p.m., Lei Xu wrote: > > --- > This is an automatically generated e-mail.

Re: Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-04-10 Thread Lei Xu
> > it cannot verify the fix for the original issue. Yes, it did. - Lei --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58281/#review171444 ---

Review Request 58281: SENTRY-1643. AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-04-07 Thread Lei Xu
-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java aaa0b9fd30bb68fded67f885af4f77bc71398e77 Diff: https://reviews.apache.org/r/58281/diff/1/ Testing --- Add a new test to conurrently insert changes. mvn test -Dtest=TestSentryStore. Thanks, Lei Xu

Re: Review Request 57570: SENTRY-1644. Partition ACLs disappear after renaming Hive table with partitions

2017-04-06 Thread Lei Xu
ally generated e-mail. To reply, visit: https://reviews.apache.org/r/57570/#review171081 ----------- On April 4, 2017, 1:54 p.m., Lei Xu wrote: > > --- > This is an automatically generated e-mail.

Re: Review Request 58221: SENTRY-1649 move HMS follower to runServer

2017-04-06 Thread Lei Xu
rvice/thrift/SentryService.java Lines 307 (patched) <https://reviews.apache.org/r/58221/#comment244111> Please put open brace to the same line as the function signature. Also there should be a space between `if` and parenthesis. Please remove the blank line at li

Re: Review Request 57901: SENTRY-1593 Implement client failover for all the sentry clients

2017-03-30 Thread Lei Xu
t243537> Please add some comments to this utility class. - Lei Xu On March 29, 2017, 4:32 p.m., kalyan kumar kalvagadda wrote: > > --- > This is an automatically

Review Request 58005: SENTRY-1626. Tables contain createTime should either remove it or update the time using SQL

2017-03-28 Thread Lei Xu
-oracle-1.8.0.sql 0418b298f sentry-provider/sentry-provider-db/src/main/resources/sentry-postgres-1.8.0.sql 68d2c8d53 Diff: https://reviews.apache.org/r/58005/diff/1/ Testing --- run `mvn test` under `sentry-provider/sentry-provider-db`. Thanks, Lei Xu

Re: Review Request 57570: SENTRY-1644. Partition ACLs disappear after renaming Hive table with partitions

2017-03-27 Thread Lei Xu
File Attachments (updated) SENTRY-1644.003-master.patch https://reviews.apache.org/media/uploaded/files/2017/03/28/1b82d5bc-0267-42d3-9a9a-9515ed7cb25e__SENTRY-1644.003-master.patch Thanks, Lei Xu

Re: Review Request 57570: SENTRY-1644. Partition ACLs disappear after renaming Hive table with partitions

2017-03-27 Thread Lei Xu
function clearer. - Lei --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57570/#review170064 --- On March 22, 2017, 2:27

Re: Review Request 57570: SENTRY-1644. Partition ACLs disappear after renaming Hive table with partitions

2017-03-22 Thread Lei Xu
.apache.org/r/57570/#review168939 --- On March 22, 2017, 2:27 p.m., Lei Xu wrote: > > --- > This is an automatically generated e-mail. To reply, v

Re: Review Request 57570: SENTRY-1644. Partition ACLs disappear after renaming Hive table with partitions

2017-03-22 Thread Lei Xu
/ Changes: https://reviews.apache.org/r/57570/diff/1-2/ Testing --- mvn test -Dtest=TestHMSPaths Thanks, Lei Xu

Review Request 57570: SENTRY-1644. Partition ACLs disappear after renaming Hive table with partitions

2017-03-13 Thread Lei Xu
--- mvn test -Dtest=TestHMSPaths Thanks, Lei Xu

Re: Review Request 57232: SENTRY-1613: Add propagating logic for Perm/Path updates in Sentry service

2017-03-06 Thread Lei Xu
l#execute-- sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 3349 (patched) <https://reviews.apache.org/r/57232/#comment240135> Please add 2 spaces before `throws Exception`. - Lei Xu On Marc

Re: Review Request 55706: SENTRY-1566: Make full Perm/Path snapshot available for NN plugin

2017-03-06 Thread Lei Xu
rivate function. sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java Lines 2369 (patched) <https://reviews.apache.org/r/55706/#comment240061> Will `query.execute()` return `null`? http://www.datanucleus.org/java

Re: Review Request 57266: SENTRY-1642 Integrate Sentry build with Error Prone

2017-03-03 Thread Lei Xu
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57266/#review167877 --- Ship it! Ship It! - Lei Xu On March 2, 2017, 10:29 p.m

Re: Review Request 57219: SENTRY-1601 Implement HMS Notification barrier on the server side

2017-03-03 Thread Lei Xu
/thrift/TestCounterWait.java Lines 74 (patched) <https://reviews.apache.org/r/57219/#comment239837> We should remove these commented statements. - Lei Xu On March 2, 2017, 7:08 p.m., Alexander Kolbasov wrote: > > -

Re: Review Request 57220: SENTRY-1638. Update MSentryPermChange table to add a column for notification ID

2017-03-02 Thread Lei Xu
ange`. I will do that in a new patch. - Lei --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57220/#review167691 ----------

Re: Review Request 56952: SENTRY-1637. Periodically purge Delta change tables.

2017-02-28 Thread Lei Xu
=TestSentryStore Thanks, Lei Xu

Re: Review Request 56952: SENTRY-1637. Periodically purge Delta change tables.

2017-02-24 Thread Lei Xu
The code itself is very simple now. Do you think a test is needed? - Lei --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56952/#review166622 --------

Re: Review Request 56952: SENTRY-1637. Periodically purge Delta change tables.

2017-02-24 Thread Lei Xu
Thanks, Lei Xu

Review Request 56952: SENTRY-1637. Periodically purge Delta change tables.

2017-02-22 Thread Lei Xu
/ Testing --- mvn test -Dtest=TestSentryStore Thanks, Lei Xu

Re: Review Request 56720: SENTRY-1611. Periodically purge MSentryPerm/PathChange table.

2017-02-21 Thread Lei Xu
-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStore.java dfaac1555fe729efab3f8fdccb914978001fee56 Diff: https://reviews.apache.org/r/56720/diff/ Testing --- Run `mvn test -Dtest=TestSentryStore` Thanks, Lei Xu

Re: Review Request 56720: SENTRY-1611. Periodically purge MSentryPerm/PathChange table.

2017-02-21 Thread Lei Xu
ated e-mail. To reply, visit: https://reviews.apache.org/r/56720/#review166238 ------- On Feb. 21, 2017, 1:47 p.m., Lei Xu wrote: > > --- > This is a

Re: Review Request 56720: SENTRY-1611. Periodically purge MSentryPerm/PathChange table.

2017-02-21 Thread Lei Xu
/sentry/provider/db/service/persistent/SentryStore.java (line 428) <https://reviews.apache.org/r/56720/#comment238149> Yea, these two statements were added in this patch. - Lei Xu On Feb. 21, 2017, 1:47 p.m., Lei Xu

Re: Review Request 56720: SENTRY-1611. Periodically purge MSentryPerm/PathChange table.

2017-02-21 Thread Lei Xu
-Dtest=TestSentryStore` Thanks, Lei Xu

Re: Review Request 56720: SENTRY-1611. Periodically purge MSentryPerm/PathChange table.

2017-02-17 Thread Lei Xu
/56720/diff/ Testing --- Run `mvn test -Dtest=TestSentryStore` Thanks, Lei Xu

Re: Review Request 56720: SENTRY-1611. Periodically purge MSentryPerm/PathChange table.

2017-02-17 Thread Lei Xu
generated e-mail. To reply, visit: https://reviews.apache.org/r/56720/#review165908 ----------- On Feb. 16, 2017, 3:37 p.m., Lei Xu wrote: > > --- > Thi

Review Request 56720: SENTRY-1611. Periodically purge MSentryPerm/PathChange table.

2017-02-15 Thread Lei Xu
=TestSentryStore` Thanks, Lei Xu

Re: Review Request 55706: SENTRY-1566: Make full Perm/Path snapshot available for NN plugin

2017-02-07 Thread Lei Xu
iews.apache.org/r/55706/#comment236272> `pm` is not used? - Lei Xu On Feb. 2, 2017, 11:53 p.m., Hao Hao wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > http

Re: Review Request 55246: SENTRY-1536: Refactor SentryStore transaction management to allow for extra transactions for a single permission update

2017-01-18 Thread Lei Xu
vider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/HMSFollower.java (line 445) <https://reviews.apache.org/r/55246/#comment233357> Shouldn't `onDropSentryPrivilege` happen after `dropPrivilege`? What if `dropPrivilege` fails in

Re: Review Request 55246: SENTRY-1536: Refactor SentryStore transaction management to allow for extra transactions for a single permission update

2017-01-12 Thread Lei Xu
> On Jan. 11, 2017, 1:21 p.m., Lei Xu wrote: > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/SentryPlugin.java, > > line 446 > > <https://reviews.apache.org/r/55246/diff/2/?file=1600435#file1600435line446> > > > > Le

Re: Review Request 55246: SENTRY-1536: Refactor SentryStore transaction management to allow for extra transactions for a single permission update

2017-01-11 Thread Lei Xu
//reviews.apache.org/r/55246/#comment232468> What is the impact of putting a `null` as value in the map? It looks like that setting `privilegesUpdateMap=null` has the same result. - Lei Xu On Jan. 10, 2017, 2:31 a.m., Hao Hao wrote: > > -