Re: Question about incrementing IDs for delta changes

Hi Lei, > On Jun 7, 2017, at 11:34 PM, Lei Xu wrote: > > Hi, Sasha > > It is still problematic due to the max(change ID) scanning. > > One possible way to address it is that, in HMSFollower#run(), it ask > sentryStore to cache max change IDs of these two tables with in > sentryStore itself. >

Re: Question about incrementing IDs for delta changes

Hi, Sasha It is still problematic due to the max(change ID) scanning. One possible way to address it is that, in HMSFollower#run(), it ask sentryStore to cache max change IDs of these two tables with in sentryStore itself. So that each time, when HMSFollower wake up, it refresh sentryStore's cach

Re: Review Request 59887: SENTRY-1794 HMSFollower not persisting last processed notifications when partition is altered

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59887/#review177278 --- Fix it, then Ship it! sentry-provider/sentry-provider-db/src/m

Re: Review Request 59869: SENTRY-1796: Add better debug logging for retrieving the delta changes

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59869/#review177277 --- sentry-provider/sentry-provider-db/src/main/java/org/apache/sentr

Question about incrementing IDs for delta changes

Lei, Thank you for fixing SENTRY-1795. I remember we had some issues earlier with this approach - see some discussion in SENTRY-1706 where you encountered a deadlock issue. Do you think that this is still a problem or not? - Sasha

Re: Review Request 59869: SENTRY-1796: Add better debug logging for retrieving the delta changes

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59869/ --- (Updated June 8, 2017, 1:05 a.m.) Review request for sentry, Alexander Kolbasov

Re: Review Request 59869: SENTRY-1796: Add better debug logging for retrieving the delta changes

> On June 7, 2017, 9:56 p.m., Alexander Kolbasov wrote: > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryUtil.java > > Lines 59 (patched) > > > > > > It seems tha

Re: Review Request 59869: SENTRY-1796: Add better debug logging for retrieving the delta changes

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59869/ --- (Updated June 7, 2017, 11:09 p.m.) Review request for sentry, Alexander Kolbaso

Re: Review Request 59869: SENTRY-1796: Add better debug logging for retrieving the delta changes

- Vamsee --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59869/#review177229 --- On June 7, 2017, 10:21 p.m., Vamsee Yarlagadda wrote: >

Re: Review Request 59869: SENTRY-1796: Add better debug logging for retrieving the delta changes

> On June 7, 2017, 9:56 p.m., Alexander Kolbasov wrote: > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/model/MSentryUtil.java > > Lines 59 (patched) > > > > > > It seems tha

Re: Review Request 59897: SENTRY-1792: Ensure DB to sort delta changes by CHANGE_ID

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59897/#review177242 --- Ship it! Ship It! - Alexander Kolbasov On June 7, 2017, 10:2

Review Request 59897: SENTRY-1792: Ensure DB to sort delta changes by CHANGE_ID

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59897/ --- Review request for sentry and Alexander Kolbasov. Repository: sentry Descript

Re: Review Request 59869: SENTRY-1796: Add better debug logging for retrieving the delta changes

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59869/ --- (Updated June 7, 2017, 10:21 p.m.) Review request for sentry, Alexander Kolbaso

Re: Review Request 59895: SENTRY-1782: Add an HMS image ID to the thrift schema definition for hdfs/sentry requests

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59895/#review177234 --- sentry-hdfs/sentry-hdfs-common/src/main/resources/sentry_hdfs_ser

Re: Review Request 59869: SENTRY-1792: HDFS sync should be aware of holes in delta table

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59869/#review177229 --- sentry-provider/sentry-provider-db/src/main/java/org/apache/sentr

Review Request 59895: SENTRY-1782: Add an HMS image ID to the thrift schema definition for hdfs/sentry requests

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59895/ --- Review request for sentry, Alexander Kolbasov, kalyan kumar kalvagadda, and Vams

Re: Review Request 59887: SENTRY-1794 HMSFollower not persisting last processed notifications when partition is altered

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59887/#review177232 --- Ship it! Ship It! - Vamsee Yarlagadda On June 7, 2017, 5:17

Re: Review Request 59720: SENTRY-1781: Define a schema and and persist an ID for new HMS snapshots

> On June 2, 2017, 3:18 p.m., Na Li wrote: > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java > > Lines 2618 (patched) > > > > > > do you want to d

Re: Review Request 59720: SENTRY-1781: Define a schema and and persist an ID for new HMS snapshots

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59720/#review177197 --- sentry-provider/sentry-provider-db/src/main/java/org/apache/sentr

Re: Review Request 59720: SENTRY-1781: Define a schema and and persist an ID for new HMS snapshots

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59720/ --- (Updated June 7, 2017, 9:24 p.m.) Review request for sentry, Alexander Kolbasov

Re: Review Request 59720: SENTRY-1781: Full snapshots should persist a generation ID in the datastore

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59720/ --- (Updated June 7, 2017, 5:40 p.m.) Review request for sentry, Alexander Kolbasov

Review Request 59887: SENTRY-1794 HMSFollower not persisting last processed notifications when partition is altered

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59887/ --- Review request for sentry, Alexander Kolbasov, Brian Towles, Hao Hao, Na Li, Ser

Re: Review Request 59720: SENTRY-1781: Full snapshots should persist a generation ID in the datastore

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59720/ --- (Updated June 7, 2017, 5:15 p.m.) Review request for sentry and Alexander Kolba

Re: Review Request 59720: SENTRY-1781: Full snapshots should persist a generation ID in the datastore

> On June 3, 2017, 1:15 a.m., Alexander Kolbasov wrote: > > This looks good. The important question is whether it will work correctly > > if two instances of Sentry Server are trying to create a full snapshot or > > one of the tries to create a full snapshot while another one updates an > > ex

Re: Review Request 58536: SENTRY-1678 Add test class to test refactored refactored config code

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58536/#review177173 --- Another thing. We should change the JIRA subject to reflect what t

Re: Review Request 58536: SENTRY-1678 Add test class to test refactored refactored config code

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58536/#review177172 --- sentry-core/sentry-core-common/src/test/java/org/apache/sentry/co