Re: Review Request 61047: SENTRY-1854 HMSFollower should handle notifications even if HDFS sync is disabled.

> On July 24, 2017, 8:57 p.m., Alexander Kolbasov wrote: > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStore.java > > Lines 171 (patched) > > > > > > The

Re: SENTRY-1855: PERM/PATH transactions can fail to commit to the sentry database under load

> > Reducing the time between reading max(changeID in DB) and transaction > commit will reduce the chance of key conflict. That is the whole point of > re-order the blocks. > Why would this affect anything? Whenever you read max(changeID) inside a transaction you should get exactly the same v

Re: SENTRY-1855: PERM/PATH transactions can fail to commit to the sentry database under load

Lia, I don’t understand why it would be any different, but there is no harm in trying. > On Jul 24, 2017, at 11:50 PM, Na Li wrote: > > Sasha, > > "How do you know that it improves performance? Do you see the improvement > in your experiments? If you do see improvements, let’s do the change.

Re: SENTRY-1855: PERM/PATH transactions can fail to commit to the sentry database under load

Sasha, "How do you know that it improves performance? Do you see the improvement in your experiments? If you do see improvements, let’s do the change." [Lina] I have not run the test. Vamsee will run the test on cluster 30 minutes later today for current approach with re-order transactions. So we

Re: Review Request 61047: SENTRY-1854 HMSFollower should handle notifications even if HDFS sync is disabled.

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61047/#review181257 --- sentry-provider/sentry-provider-db/src/main/java/org/apache/sentr

Re: SENTRY-1855: PERM/PATH transactions can fail to commit to the sentry database under load

Lina, > On Jul 24, 2017, at 10:20 PM, Na Li wrote: > > Sasha, > > I found the following approach that could improve the performance of the > current approach and second approach a lot with minor code change How do you know that it improves performance? Do you see the improvement in your exper

Re: SENTRY-1855: PERM/PATH transactions can fail to commit to the sentry database under load

Sasha, I found the following approach that could improve the performance of the current approach and second approach a lot with minor code change Right now, the execution order of the transaction blocks in a transaction is 1. Delta transaction block to save the perm change and path change 2. The

Re: Review Request 60883: SENTRY-1825: Dropping a Hive database/table doesn't cleanup the permissions associated with it

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60883/#review181255 --- Ship it! Ship It! - Alexander Kolbasov On July 21, 2017, 8:2

Re: SENTRY-1855: PERM/PATH transactions can fail to commit to the sentry database under load

Approach 2.2, it uses auto-increment of changeID. When there is a temporary hole at the start of the list, even sending full snapshot could skip the transaction in flight. It does not function correctly. For example, the list is 1,2,3,5,6, and transaction with changeID 4 is not done. First, S

Re: SENTRY-1855: PERM/PATH transactions can fail to commit to the sentry database under load

Lina, can you describe what problem have you discovered with approach 2.2? On Mon, Jul 24, 2017 at 6:37 PM, Na Li wrote: > Sasha, > > I realize a serious issue in approach 2.2) that it does not function > correctly. So I re-iterate the approaches and their pros and cons. > > This leaves us the

Re: SENTRY-1855: PERM/PATH transactions can fail to commit to the sentry database under load

Sasha, I realize a serious issue in approach 2.2) that it does not function correctly. So I re-iterate the approaches and their pros and cons. This leaves us the option 2.1) and 2.3). The question is if the performance of approach 2.1) acceptable for now, and we can work on the long term solution

Re: Splitting permissions and path updates

That's good. Permission updates - even full permission update shouldn't be very expensive, so we may be more willing to send one when we detect improper holes. On Mon, Jul 24, 2017 at 5:59 PM, Na Li wrote: > Sasha, > > Yes. > > SentryHDFSServiceProcessor.get_authz_updates gets perm update and pa

Re: Splitting permissions and path updates

Sasha, Yes. SentryHDFSServiceProcessor.get_authz_updates gets perm update and path update separately, and puts them into the response object. Thanks, Lina On Mon, Jul 24, 2017 at 10:48 AM, Alexander Kolbasov wrote: > Does it mean that without any changes the current code may send e.g. full >

Re: Review Request 60955: SENTRY-1853: Add the log level access mechanism

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60955/#review181219 --- sentry-service/sentry-service-server/src/test/java/org/apache/sen

Re: Review Request 60955: SENTRY-1853: Add the log level access mechanism

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60955/#review181217 --- sentry-service/sentry-service-server/src/main/java/org/apache/sen

Re: Splitting permissions and path updates

Does it mean that without any changes the current code may send e.g. full update for permissions and partial update for paths or visa versa? - Alex On Mon, Jul 24, 2017 at 5:36 PM, Na Li wrote: > Sasha, > > When NameNode plugin asks for updates, it includes info for both permission > and path.

Re: JIRA versions

Yep, thanks! What is the timeline for 2.0.0 btw? Colm. On Mon, Jul 24, 2017 at 4:40 PM, Kalyan Kumar Kalvagadda < kkal...@cloudera.com> wrote: > Thanks Sasha for acting on it. > > -Kalyan > > On Mon, Jul 24, 2017 at 10:37 AM, Alexander Kolbasov > wrote: > > > Removed. > > > > On Mon, Jul 24, 20

Re: JIRA versions

Thanks Sasha for acting on it. -Kalyan On Mon, Jul 24, 2017 at 10:37 AM, Alexander Kolbasov wrote: > Removed. > > On Mon, Jul 24, 2017 at 5:02 PM, Colm O hEigeartaigh > wrote: > > > I still see the following version in JIRA: > > > > https://issues.apache.org/jira/projects/SENTRY/versions/12336

Re: JIRA versions

Removed. On Mon, Jul 24, 2017 at 5:02 PM, Colm O hEigeartaigh wrote: > I still see the following version in JIRA: > > https://issues.apache.org/jira/projects/SENTRY/versions/12336080 > > Colm. > > On Mon, Jul 24, 2017 at 3:57 PM, Alexander Kolbasov > wrote: > >> Done! >> >> On Mon, Jul 24, 2017

Re: Splitting permissions and path updates

Sasha, When NameNode plugin asks for updates, it includes info for both permission and path. However, the processing is separate. It is possible for Sentry to send full snapshot of permission and delta change to HDFS. At Sentry, perm and path processing share the same class, but they have their ow

Re: Sentry-jdk-1.7-v2 - Build # 100 - Still Failing

Thanks Dapeng! On Mon, Jul 24, 2017 at 10:12 AM, Sun, Dapeng wrote: > The failure is happened on ubuntu-eu2, I have removed this node from build > list. > > -Original Message- > From: Apache Jenkins Server [mailto:jenk...@builds.apache.org] > Sent: Monday, July 24, 2017 4:06 PM > To: s..

Re: JIRA versions

I still see the following version in JIRA: https://issues.apache.org/jira/projects/SENTRY/versions/12336080 Colm. On Mon, Jul 24, 2017 at 3:57 PM, Alexander Kolbasov wrote: > Done! > > On Mon, Jul 24, 2017 at 10:21 AM, Colm O hEigeartaigh > wrote: > >> Can someone with admin privileges on JIR

Re: JIRA versions

Done! On Mon, Jul 24, 2017 at 10:21 AM, Colm O hEigeartaigh wrote: > Can someone with admin privileges on JIRA make this happen? Just bulk move > all sentry-ha-redesign issues to 2.0.0 + delete the sentry-ha-redesign > version once this is done. > > Colm. > > On Thu, Jul 20, 2017 at 4:26 PM, Kal

Re: Work on Sentry 1.8.0 release has started

Hi, I'll start the 1.8 branch cut today. Please do not commit anything to master for that time. Btw, there was a discussion about making sentry-ha-redesign the new master. So, this means that after this branch cut, we won't use master until sentry-ha-redesign is renamed. Please commit any change

Re: JIRA versions

I am in the process of doing that On Mon, Jul 24, 2017 at 4:41 PM, Mat Crocker wrote: > Hi Colm, > > Can you confirm this is the set you are trying to migrate? > > "project = sentry and (fixVersion = sentry-ha-redesign or affectedVersion = > sentry-ha-redesign) " > > Happy to help. > > > > On Mo

Re: JIRA versions

Hi Colm, Can you confirm this is the set you are trying to migrate? "project = sentry and (fixVersion = sentry-ha-redesign or affectedVersion = sentry-ha-redesign) " Happy to help. On Mon, Jul 24, 2017 at 3:21 AM, Colm O hEigeartaigh wrote: > Can someone with admin privileges on JIRA make t

Sentry-jdk-1.7-v2 - Build # 103 - Still unstable

The Apache Jenkins build system has built Sentry-jdk-1.7-v2 (build #103) Status: Still unstable Check console output at https://builds.apache.org/job/Sentry-jdk-1.7-v2/103/ to view the results.

Re: JIRA versions

Can someone with admin privileges on JIRA make this happen? Just bulk move all sentry-ha-redesign issues to 2.0.0 + delete the sentry-ha-redesign version once this is done. Colm. On Thu, Jul 20, 2017 at 4:26 PM, Kalyan Kumar Kalvagadda < kkal...@cloudera.com> wrote: > Colm, > > I agree with you.

RE: Sentry-jdk-1.7-v2 - Build # 100 - Still Failing

The failure is happened on ubuntu-eu2, I have removed this node from build list. -Original Message- From: Apache Jenkins Server [mailto:jenk...@builds.apache.org] Sent: Monday, July 24, 2017 4:06 PM To: s...@apache.org; ak...@cloudera.com; cohei...@apache.org; hao@cloudera.com; dev@s

Sentry-jdk-1.7-v2 - Build # 100 - Still Failing

The Apache Jenkins build system has built Sentry-jdk-1.7-v2 (build #100) Status: Still Failing Check console output at https://builds.apache.org/job/Sentry-jdk-1.7-v2/100/ to view the results.

Sentry-jdk-1.7-v2 - Build # 99 - Failure

The Apache Jenkins build system has built Sentry-jdk-1.7-v2 (build #99) Status: Failure Check console output at https://builds.apache.org/job/Sentry-jdk-1.7-v2/99/ to view the results.