Thanks Lina, this explains the mystery! - Alex
On Fri, Jun 23, 2017 at 4:37 PM, Na Li <lina...@cloudera.com> wrote: > Sasha, > > > After looking into history of this issue, I found related Jira > sentry-1643, sentry-1706, sentry-1710 and sentry-1795. > > > The approach in sentry-1643 and sentry-1795 is the same: increase changeID > in application instead of letting DB automatically increases changeID. > > The difference is the implementation detail. > > sentry-1643 increases the changeID in both application and DB > (value-strategy of is NOT removed in package.jdo). This causes deadlock. > > sentry-1795 increases the changeID only in application (value-strategy is > removed in package.jdo). This does not have deadlock issue. > > The conclusion is that we don't have deadlock issue with sentry-1795. And > there is no further actions to take for this issue. > > > <class name="MSentryPermChange" table="SENTRY_PERM_CHANGE" > identity-type="application" detachable="true"> > > <field name="changeID" primary-key="true" > value-strategy="increment" key-cache-size="1"> > > <column name="CHANGE_ID" jdbc-type="BIGINT" allows-null="false"/> > > </field> > > <field name ="permChange"> > > > Thanks, > > > Lina > > On Fri, Jun 23, 2017 at 5:57 PM, Alexander Kolbasov <ak...@cloudera.com> > wrote: > >> Hi Lina, >> >> what would you suggest? >> >> - Sasha >> >> On Fri, Jun 23, 2017 at 1:21 PM, Na Li <lina...@cloudera.com> wrote: >> >>> Sasha, >>> >>> It seems we still have the deadlock issue with fix in sentry-1795. It >>> may not show now, but could happen with heavy traffic. >>> >>> Thanks, >>> >>> Lina >>> >>> >>> ---------- Forwarded message ---------- >>> From: Alexander Kolbasov <ak...@cloudera.com> >>> Date: Thu, Jun 8, 2017 at 1:39 AM >>> Subject: Re: Question about incrementing IDs for delta changes >>> To: Lei Xu <l...@cloudera.com> >>> Cc: dev <dev@sentry.apache.org> >>> >>> >>> Hi Lei, >>> >>> > On Jun 7, 2017, at 11:34 PM, Lei Xu <l...@cloudera.com> wrote: >>> > >>> > Hi, Sasha >>> > >>> > It is still problematic due to the max(change ID) scanning. >>> > >>> > One possible way to address it is that, in HMSFollower#run(), it ask >>> > sentryStore to cache max change IDs of these two tables with in >>> > sentryStore itself. >>> > So that each time, when HMSFollower wake up, it refresh sentryStore's >>> > cached change ID value. And since DB write is fenced, so the cached >>> > change IDs should be valid during the execution ? >>> >>> The fencing is a best-effort - it isn’t guaranteed to always fence. >>> There may be some delay between the time >>> a new leader is elected and the old leader stops processing >>> notifications. >>> >>> > >>> > Do you know whether SentryPolicyStoreProcessor is also fenced by >>> > leaderMonitor? SentryPolicyStoreProcessor also has a write path to >>> > the DB via SentryStore. >>> >>> No, it isn’t fenced at all. Only HMSFollower is. >>> >>> - Alex >>> >>> > >>> > On Wed, Jun 7, 2017 at 10:05 PM, Alexander Kolbasov < >>> ak...@cloudera.com> wrote: >>> >> Lei, >>> >> >>> >> Thank you for fixing SENTRY-1795. I remember we had some issues >>> earlier with >>> >> this approach - see some discussion in SENTRY-1706 where you >>> encountered a >>> >> deadlock issue. >>> >> >>> >> Do you think that this is still a problem or not? >>> >> >>> >> - Sasha >>> > >>> > >>> > >>> > -- >>> > Lei (Eddy) Xu >>> > Software Engineer, Cloudera >>> >>> >>> >> >
