[jira] [Commented] (SENTRY-1760) HMSFollower should detect when a full snapshot from HMS is required

2017-06-27 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065068#comment-16065068 ] Na Li commented on SENTRY-1760: --- [~spena] Will this update include automatically detecting when the Name

[jira] [Commented] (SENTRY-1712) Add capability to force Sentry to get full snapshot from HMS

2017-06-26 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063793#comment-16063793 ] Na Li commented on SENTRY-1712: --- Both sentry-1712 and sentry-1520 add capability to force getting full HMS

[jira] [Commented] (SENTRY-1693) HMSFollower should handle adding a view with empty path.

2017-06-26 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063596#comment-16063596 ] Na Li commented on SENTRY-1693: --- the current behavior of HMSFollower is that when the location is null, the

[jira] [Commented] (SENTRY-1693) HMSFollower should handle adding a view with empty path.

2017-06-26 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063390#comment-16063390 ] Na Li commented on SENTRY-1693: --- The problem of this issue is "HMSFollower gets stuck once it fails to

[jira] [Commented] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-06-23 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061641#comment-16061641 ] Na Li commented on SENTRY-1643: --- The deadlock log is in sentry-1706 for the code change in sentry-1743. >

[jira] [Commented] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-06-23 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061637#comment-16061637 ] Na Li commented on SENTRY-1643: --- The approach in sentry-1643 and sentry-1795 is the same: increase changeID

[jira] [Commented] (SENTRY-1643) AutoIncrement ChangeID of MSentryPermChange/MSentryPathChange may be error-prone

2017-06-23 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061636#comment-16061636 ] Na Li commented on SENTRY-1643: --- The code review for this issue is in

[jira] [Commented] (SENTRY-1800) Flaky testConcurrentUpdateChanges test

2017-06-10 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045779#comment-16045779 ] Na Li commented on SENTRY-1800: --- Are we seeing the same issue as in Sentry-1792? The changeID is

[jira] [Updated] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1784: -- Attachment: (was: SENTRY-1784.006-sentry-ha-redesign.with1771.patch) > DBUpdateForwarder returns empty

[jira] [Updated] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1784: -- Attachment: SENTRY-1784.006-sentry-ha-redesign.with1771.patch v6. Add more comments and update based on code

[jira] [Commented] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037813#comment-16037813 ] Na Li commented on SENTRY-1771: --- We cannot reproduce the concurrent requests for perm changes on Vamsee's

[jira] [Updated] (SENTRY-1792) Too many full snapshot for Permission change request

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1792: -- Attachment: SENTRY-1792.001-sentry-ha-redesign.patch v1. extract continuous list from non-continuous list that

[jira] [Updated] (SENTRY-1792) Too many full snapshot for Permission change request

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1792: -- Status: Patch Available (was: Open) > Too many full snapshot for Permission change request >

[jira] [Commented] (SENTRY-1792) Too many full snapshot for Permission change request

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037796#comment-16037796 ] Na Li commented on SENTRY-1792: --- The possible cause is when multiple threads are updating perm change table

[jira] [Commented] (SENTRY-1750) HMSFollower does not handle view update correctly

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037792#comment-16037792 ] Na Li commented on SENTRY-1750: --- Sentry does not need to handle view explicitly. Since its path is null, a

[jira] [Commented] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037691#comment-16037691 ] Na Li commented on SENTRY-1771: --- Sentry-1771 had two issues: 1) concurrent requests for perm changes 2) too

[jira] [Created] (SENTRY-1792) Too many full snapshot for Permission change request

2017-06-05 Thread Na Li (JIRA)
Na Li created SENTRY-1792: - Summary: Too many full snapshot for Permission change request Key: SENTRY-1792 URL: https://issues.apache.org/jira/browse/SENTRY-1792 Project: Sentry Issue Type: Bug

[jira] [Commented] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037590#comment-16037590 ] Na Li commented on SENTRY-1771: --- At Vamsee's cluster, we don't see concurrent requests. This issue is

[jira] [Commented] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037548#comment-16037548 ] Na Li commented on SENTRY-1771: --- "2017-06-05 11:01:32,165 ERROR

[jira] [Commented] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037384#comment-16037384 ] Na Li commented on SENTRY-1771: --- "2017-06-05 11:04:45,790 ERROR

[jira] [Updated] (SENTRY-1752) HMSFollower gets stuck once it fails to process a notification event

2017-06-05 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1752: -- Resolution: Fixed Status: Resolved (was: Patch Available) The code change to fix the issue is in

[jira] [Commented] (SENTRY-1773) Delta change cleaner should leave way more then a single entry intact

2017-06-04 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036290#comment-16036290 ] Na Li commented on SENTRY-1773: --- " How many deltas can be generated in 1/2 seconds?" It depends on the user

[jira] [Commented] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-06-02 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035521#comment-16035521 ] Na Li commented on SENTRY-1771: --- using Vamsee's cluster and added logging message, issue command "cat

[jira] [Updated] (SENTRY-1752) HMSFollower gets stuck once it fails to process a notification event

2017-06-02 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1752: -- Attachment: SENTRY-1752.005-sentry-ha-redesign.patch v5. rebase on latest code > HMSFollower gets stuck once

[jira] [Updated] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-06-01 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1784: -- Attachment: SENTRY-1784.004-sentry-ha-redesign.patch v4. Add more comment in DBUpdateForwarder to explain how

[jira] [Commented] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-06-01 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033258#comment-16033258 ] Na Li commented on SENTRY-1784: --- [~spena] you may mix HMS follower at Sentry getting full snapshot from

[jira] [Resolved] (SENTRY-1790) NoClassDefFoundError: org/apache/sentry/binding/hive/SentryOnFailureHookContextImpl

2017-06-01 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li resolved SENTRY-1790. --- Resolution: Cannot Reproduce Fix Version/s: sentry-ha-redesign > NoClassDefFoundError: >

[jira] [Commented] (SENTRY-1790) NoClassDefFoundError: org/apache/sentry/binding/hive/SentryOnFailureHookContextImpl

2017-06-01 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033217#comment-16033217 ] Na Li commented on SENTRY-1790: --- I cannot reproduce the exception in sentry-ha-redesign branch. close this

[jira] [Commented] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-06-01 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033187#comment-16033187 ] Na Li commented on SENTRY-1784: --- Info about the fix: HDFS starts requesting changeID = 0 instead of 1.

[jira] [Commented] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-05-31 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032228#comment-16032228 ] Na Li commented on SENTRY-1771: --- We should add code at the client side to catch it if there are more

[jira] [Commented] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-31 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031921#comment-16031921 ] Na Li commented on SENTRY-1784: --- [~akolb] the requested ID is always the (returned latest changeID from

[jira] [Commented] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-31 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031872#comment-16031872 ] Na Li commented on SENTRY-1784: --- [~akolb] with the code change, when there is no change, the requested ID

[jira] [Commented] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-05-31 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031857#comment-16031857 ] Na Li commented on SENTRY-1771: --- [~akolb] The patch and review are not up to date. I have not found the fix

[jira] [Updated] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-05-31 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1771: -- Attachment: (was: SENTRY-1771.002-sentry-ha-redesign.patch) > HDFS client concurrently requests full

[jira] [Updated] (SENTRY-1771) HDFS client concurrently requests full permission update multiple times

2017-05-31 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1771: -- Attachment: (was: SENTRY-1771.001-sentry-ha-redesign.patch) > HDFS client concurrently requests full

[jira] [Created] (SENTRY-1790) NoClassDefFoundError: org/apache/sentry/binding/hive/SentryOnFailureHookContextImpl

2017-05-31 Thread Na Li (JIRA)
Na Li created SENTRY-1790: - Summary: NoClassDefFoundError: org/apache/sentry/binding/hive/SentryOnFailureHookContextImpl Key: SENTRY-1790 URL: https://issues.apache.org/jira/browse/SENTRY-1790 Project:

[jira] [Commented] (SENTRY-1773) Delta change cleaner should leave way more then a single entry intact

2017-05-31 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16031321#comment-16031321 ] Na Li commented on SENTRY-1773: --- That will minimize the chance we have to take full snapshot. The idea

[jira] [Updated] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-30 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1784: -- Attachment: SENTRY-1784.002-sentry-ha-redesign.patch v2. Update log message based on code review >

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-30 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029916#comment-16029916 ] Na Li commented on SENTRY-1771: --- If deltas are purged only after they are delivered to client, then we

[jira] [Updated] (SENTRY-1752) HMSFollower gets stuck once it fails to process a notification event

2017-05-27 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1752: -- Attachment: SENTRY-1752.004-sentry-ha-redesign.patch v4. Trigger build. The last failure is a unit test,

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-26 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16026906#comment-16026906 ] Na Li commented on SENTRY-1771: --- The likely reason is that the created changeID from datanucleus is not

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-26 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16026859#comment-16026859 ] Na Li commented on SENTRY-1771: --- Using Vamsee's cluster with patch v2 running for overnight, we can see the

[jira] [Updated] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-25 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1784: -- Attachment: SENTRY-1784.001-sentry-ha-redesign.patch v1. trigger build after SENTRY-1785 is committed. >

[jira] [Updated] (SENTRY-1752) HMSFollower gets stuck once it fails to process a notification event

2017-05-25 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1752: -- Attachment: SENTRY-1752.004-sentry-ha-redesign.patch v4, trigger build after SENTRY-1785 is committed >

[jira] [Updated] (SENTRY-1752) HMSFollower gets stuck once it fails to process a notification event

2017-05-25 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1752: -- Attachment: SENTRY-1752.004-sentry-ha-redesign.patch v4. update test to fix failure. > HMSFollower gets stuck

[jira] [Updated] (SENTRY-1752) HMSFollower gets stuck once it fails to process a notification event

2017-05-25 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1752: -- Attachment: SENTRY-1752.003-sentry-ha-redesign.patch v3. Sentry-1669 has changed the behavior and update the

[jira] [Updated] (SENTRY-1750) HMSFollower does not handle view update correctly

2017-05-25 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1750: -- Priority: Minor (was: Major) > HMSFollower does not handle view update correctly >

[jira] [Commented] (SENTRY-1750) HMSFollower does not handle view update correctly

2017-05-25 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024907#comment-16024907 ] Na Li commented on SENTRY-1750: --- [~spena] is it possible for Hive to add type in notification, so we can

[jira] [Commented] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-25 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024804#comment-16024804 ] Na Li commented on SENTRY-1784: --- In original code, the first seqNum is 1, not zero. So the processing

[jira] [Updated] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1784: -- Status: Patch Available (was: In Progress) > DBUpdateForwarder returns empty update list to HDFS instead of

[jira] [Updated] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1784: -- Attachment: SENTRY-1784.001-sentry-ha-redesign.patch v1. HDFS starts requesting changeID = 0 instead of 1.

[jira] [Updated] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1784: -- Issue Type: Bug (was: Sub-task) Parent: (was: SENTRY-872) > DBUpdateForwarder returns empty

[jira] [Created] (SENTRY-1784) DBUpdateForwarder returns empty update list to HDFS instead of full update

2017-05-24 Thread Na Li (JIRA)
Na Li created SENTRY-1784: - Summary: DBUpdateForwarder returns empty update list to HDFS instead of full update Key: SENTRY-1784 URL: https://issues.apache.org/jira/browse/SENTRY-1784 Project: Sentry

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024048#comment-16024048 ] Na Li commented on SENTRY-1771: --- "So the old code returned an emty list when there are no updates. Why do

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023984#comment-16023984 ] Na Li commented on SENTRY-1771: --- in v2, there are 3 types of changes: 1. Sync

[jira] [Commented] (SENTRY-1781) Full snapshots should persist a generation ID in the datastore

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023818#comment-16023818 ] Na Li commented on SENTRY-1781: --- Note that this requires the HDFS to get this generation ID together with

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023789#comment-16023789 ] Na Li commented on SENTRY-1771: --- This is a bug created by "SENTRY-1613: Add propagating logic for Perm/Path

[jira] [Updated] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1771: -- Attachment: SENTRY-1771.002-sentry-ha-redesign.patch v2. Fix bug in DBUpdateForwarder. Add more logging >

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023682#comment-16023682 ] Na Li commented on SENTRY-1771: --- IN DBUpdateForwarder, there is a bug that causes sentry server returns

[jira] [Assigned] (SENTRY-1750) HMSFollower does not handle view update correctly

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li reassigned SENTRY-1750: - Assignee: Na Li > HMSFollower does not handle view update correctly >

[jira] [Commented] (SENTRY-1630) out of sequence error in HMSFollower

2017-05-24 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16022957#comment-16022957 ] Na Li commented on SENTRY-1630: --- I have updated testing code TestHDFSIntegrationBase. The hive-site.xml is

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-23 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021912#comment-16021912 ] Na Li commented on SENTRY-1771: --- SENTRY-1773 "Delta change cleaner should leave way more then a single

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-23 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021908#comment-16021908 ] Na Li commented on SENTRY-1771: --- For the two scenarios that Sentry could generate multiple full perm

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-23 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021834#comment-16021834 ] Na Li commented on SENTRY-1771: --- I have added more logging in DBUpdateForwarder and delay in generating

[jira] [Commented] (SENTRY-1778) Close query as soon as it is done

2017-05-22 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16020608#comment-16020608 ] Na Li commented on SENTRY-1778: --- I don't think we have any query leak without this change. > Close query

[jira] [Updated] (SENTRY-1778) Close query as soon as it is done

2017-05-22 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1778: -- Description: Make sure we close all query results after we have finished with them. Failure to do so will

[jira] [Created] (SENTRY-1778) Close query as soon as it is done

2017-05-22 Thread Na Li (JIRA)
Na Li created SENTRY-1778: - Summary: Close query as soon as it is done Key: SENTRY-1778 URL: https://issues.apache.org/jira/browse/SENTRY-1778 Project: Sentry Issue Type: Sub-task

[jira] [Updated] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-19 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1771: -- Status: Patch Available (was: In Progress) > HDFS client requests full permission update multiple times >

[jira] [Updated] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-19 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1771: -- Attachment: SENTRY-1771.001-sentry-ha-redesign.patch v1. when curent time < wait until, return instead of

[jira] [Comment Edited] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-19 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017768#comment-16017768 ] Na Li edited comment on SENTRY-1771 at 5/19/17 6:10 PM: I think the reason we

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-19 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017768#comment-16017768 ] Na Li commented on SENTRY-1771: --- I think the reason we have 6 concurrent full snapshot requests from HDFS

[jira] [Updated] (SENTRY-1705) Do not start HMSFollower if Hive isn't configured

2017-05-18 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1705: -- Attachment: SENTRY-1705.006-sentry-ha-redesign.patch v6. Remove extra check for metastore URI in HMSFollower.

[jira] [Resolved] (SENTRY-1747) HMSFollower shouldn't create local hive during tests

2017-05-18 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li resolved SENTRY-1747. --- Resolution: Fixed This is resolved by sentry-1757 > HMSFollower shouldn't create local hive during tests >

[jira] [Resolved] (SENTRY-1723) HDFS e2e tests should wait for HMSFollower to start

2017-05-18 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li resolved SENTRY-1723. --- Resolution: Fixed sentry-1757 has fixed the issue by changing service orders in test, and make it the same

[jira] [Updated] (SENTRY-1705) Do not start HMSFollower if Hive isn't configured

2017-05-18 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1705: -- Attachment: SENTRY-1705.005-sentry-ha-redesign.patch v5. Add util function to get metastore URI > Do not

[jira] [Commented] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-17 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014963#comment-16014963 ] Na Li commented on SENTRY-1771: --- Based on what I have found, it seems that the reason we are getting full

[jira] [Updated] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-17 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1771: -- Issue Type: Sub-task (was: Task) Parent: SENTRY-872 > HDFS client requests full permission update

[jira] [Created] (SENTRY-1771) HDFS client requests full permission update multiple times

2017-05-17 Thread Na Li (JIRA)
Na Li created SENTRY-1771: - Summary: HDFS client requests full permission update multiple times Key: SENTRY-1771 URL: https://issues.apache.org/jira/browse/SENTRY-1771 Project: Sentry Issue Type:

[jira] [Updated] (SENTRY-1757) Avoid using local hive meta store with wrong configuration

2017-05-16 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1757: -- Attachment: SENTRY-1757.007-sentry-ha-redesign.patch v7. Change how the services are started to avoid

[jira] [Updated] (SENTRY-1757) Avoid using local hive meta store with wrong configuration

2017-05-16 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1757: -- Attachment: SENTRY-1757.006-sentry-ha-redesign.patch v6. re-trigger the build. build 2716 had build issue. >

[jira] [Updated] (SENTRY-1757) Avoid using local hive meta store with wrong configuration

2017-05-16 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1757: -- Attachment: SENTRY-1757.006-sentry-ha-redesign.patch v6. Update based on review. Add more comments > Avoid

[jira] [Commented] (SENTRY-1705) Do not start HMSFollower if Hive isn't configured

2017-05-15 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011039#comment-16011039 ] Na Li commented on SENTRY-1705: --- The code change is based on updates in sentry-1757. So it should be

[jira] [Updated] (SENTRY-1705) Do not start HMSFollower if Hive isn't configured

2017-05-15 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1705: -- Attachment: SENTRY-1705.003-sentry-ha-redesign.patch v3. check if hmsFollower and its executor are null when

[jira] [Updated] (SENTRY-1705) Do not start HMSFollower if Hive isn't configured

2017-05-12 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1705: -- Attachment: SENTRY-1705.002-sentry-ha-redesign.patch v2. check metastore uri at stopping HMSFollower > Do not

[jira] [Updated] (SENTRY-1705) Do not start HMSFollower if Hive isn't configured

2017-05-12 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1705: -- Attachment: SENTRY-1705.001-sentry-ha-redesign.patch v1. > Do not start HMSFollower if Hive isn't configured

[jira] [Updated] (SENTRY-1705) Do not start HMSFollower if Hive isn't configured

2017-05-12 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1705: -- Status: Patch Available (was: In Progress) > Do not start HMSFollower if Hive isn't configured >

[jira] [Assigned] (SENTRY-1765) CounterWait.update throw exception when input is old

2017-05-12 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li reassigned SENTRY-1765: - Assignee: kalyan kumar kalvagadda (was: Na Li) the latest change in SENTRY-1669 may fix this. >

[jira] [Created] (SENTRY-1765) CounterWait.update throw exception when input is old

2017-05-12 Thread Na Li (JIRA)
Na Li created SENTRY-1765: - Summary: CounterWait.update throw exception when input is old Key: SENTRY-1765 URL: https://issues.apache.org/jira/browse/SENTRY-1765 Project: Sentry Issue Type: Bug

[jira] [Comment Edited] (SENTRY-1723) HDFS e2e tests should wait for HMSFollower to start

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007601#comment-16007601 ] Na Li edited comment on SENTRY-1723 at 5/12/17 4:03 AM: I have reproduced the

[jira] [Commented] (SENTRY-1723) HDFS e2e tests should wait for HMSFollower to start

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007601#comment-16007601 ] Na Li commented on SENTRY-1723: --- I have reproduced the failure locally. from the log, I found

[jira] [Updated] (SENTRY-1717) Sentry should emit log messages when it is ready to serve requests.

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1717: -- Attachment: SENTRY-1717.001-sentry-ha-redesign.patch v1. add log for service ready > Sentry should emit log

[jira] [Updated] (SENTRY-1717) Sentry should emit log messages when it is ready to serve requests.

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1717: -- Status: Patch Available (was: In Progress) > Sentry should emit log messages when it is ready to serve

[jira] [Commented] (SENTRY-1717) Sentry should emit log messages when it is ready to serve requests.

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007134#comment-16007134 ] Na Li commented on SENTRY-1717: --- for "Sentry HMS support is ready", important check is that 1) HMSFollower

[jira] [Updated] (SENTRY-1717) Sentry should emit log messages when it is ready to serve requests.

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1717: -- Attachment: (was: SENTRY-1757.005-sentry-ha-redesign.patch) > Sentry should emit log messages when it is

[jira] [Updated] (SENTRY-1717) Sentry should emit log messages when it is ready to serve requests.

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1717: -- Attachment: SENTRY-1757.005-sentry-ha-redesign.patch v5. add more logging message in failed test class

[jira] [Updated] (SENTRY-1757) Avoid using local hive meta store using wrong configuration

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1757: -- Attachment: SENTRY-1757.005-sentry-ha-redesign.patch v5. add more logging message in failed test class

[jira] [Assigned] (SENTRY-1717) Sentry should emit log messages when it is ready to serve requests.

2017-05-11 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li reassigned SENTRY-1717: - Assignee: Na Li (was: Alexander Kolbasov) > Sentry should emit log messages when it is ready to serve

[jira] [Updated] (SENTRY-1757) Avoid using local hive meta store using wrong configuration

2017-05-10 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated SENTRY-1757: -- Attachment: SENTRY-1757.004-sentry-ha-redesign.patch v4. in test make sure hive-site.xml is available before

[jira] [Commented] (SENTRY-1756) Passive nodes should still follow latest notification ID

2017-05-10 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005804#comment-16005804 ] Na Li commented on SENTRY-1756: --- [~spena] can you put review link here? > Passive nodes should still

[jira] [Commented] (SENTRY-1716) HMSFollower doesn't need to save path info when HDFS sync is disabled

2017-05-10 Thread Na Li (JIRA)
[ https://issues.apache.org/jira/browse/SENTRY-1716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005800#comment-16005800 ] Na Li commented on SENTRY-1716: --- This means "(SENTRY-1760) HMSFollower should detect when a full snapshot

<    7   8   9   10   11   12   13   14   15   >