[ https://issues.apache.org/jira/browse/SENTRY-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Na Li updated SENTRY-1771: -------------------------- Attachment: SENTRY-1771.001-sentry-ha-redesign.patch v1. when curent time < wait until, return instead of sleeping. This avoid 6 concurrent requests for full snapshot from HDFS. Also synchronize the function SentryAuthorizationInfo.update() to make sure requests always contain the latest changeID > HDFS client requests full permission update multiple times > ---------------------------------------------------------- > > Key: SENTRY-1771 > URL: https://issues.apache.org/jira/browse/SENTRY-1771 > Project: Sentry > Issue Type: Sub-task > Components: Sentry > Affects Versions: sentry-ha-redesign > Reporter: Na Li > Assignee: Na Li > Priority: Critical > Labels: security > Attachments: SENTRY-1771.001-sentry-ha-redesign.patch > > > When running performance test, we found there are OOM, and the memory dump > shows that there are 6 requests from HDFS to get full update for permission > when OOM happens. > Server sends back full update if the sequence number from client is 1 at > DBUpdateForwarder.getAllUpdatesFrom() > Example of call stack at the Sentry server is > 166,999K (17.4%) (org.datanucleus.ExecutionContextThreadedImpl) > <-- Java Local (org.datanucleus.ExecutionContextThreadedImpl) > [@c01196f0,@c07dbb10] > 99,019K (10.3%) (j.u.HashSet, j.u.HashMap) > <-- > org.datanucleus.ExecutionContextThreadedImpl.reachabilityPersistedIds <-- > Java Local (org.datanucleus.ExecutionContextThreadedImpl) > [@c01196f0,@c07dbb10] > 85,196K (8.9%) (org.datanucleus.identity.OIDImpl) > <-- {j.u.HashSet} <-- > org.datanucleus.ExecutionContextThreadedImpl.reachabilityPersistedIds <-- > Java Local (org.datanucleus.ExecutionContextThreadedImpl) > [@c01196f0,@c07dbb10] > 64,606K (6.7%) > <-- org.datanucleus.identity.OIDImpl.toString <-- {j.u.HashSet} <-- > org.datanucleus.ExecutionContextThreadedImpl.reachabilityPersistedIds <-- > Java Local (org.datanucleus.ExecutionContextThreadedImpl) > [@c01196f0,@c07dbb10] > GC root stack trace: > > org.datanucleus.store.rdbms.scostore.JoinSetStore.iterator(JoinSetStore.java:914) > org.datanucleus.store.types.backed.Set.loadFromStore(Set.java:323) > org.datanucleus.store.types.backed.Set.initialise(Set.java:272) > org.datanucleus.store.types.SCOUtils.createSCOWrapper(SCOUtils.java:256) > org.datanucleus.store.types.SCOUtils.newSCOInstance(SCOUtils.java:142) > > org.datanucleus.store.rdbms.mapping.java.AbstractContainerMapping.replaceFieldWithWrapper(AbstractContainerMapping.java:399) > > org.datanucleus.store.rdbms.mapping.java.AbstractContainerMapping.postFetch(AbstractContainerMapping.java:417) > > org.datanucleus.store.rdbms.request.FetchRequest.execute(FetchRequest.java:420) > > org.datanucleus.store.rdbms.RDBMSPersistenceHandler.fetchObject(RDBMSPersistenceHandler.java:324) > > org.datanucleus.state.AbstractStateManager.loadFieldsFromDatastore(AbstractStateManager.java:1122) > > org.datanucleus.state.JDOStateManager.loadUnloadedFieldsInFetchPlan(JDOStateManager.java:3000) > > org.datanucleus.state.AbstractStateManager.loadFieldsInFetchPlan(AbstractStateManager.java:1064) > > org.datanucleus.ExecutionContextImpl.performDetachAllOnTxnEndPreparation(ExecutionContextImpl.java:4574) > > org.datanucleus.ExecutionContextImpl.preCommit(ExecutionContextImpl.java:4259) > > org.datanucleus.ExecutionContextImpl.transactionPreCommit(ExecutionContextImpl.java:654) > > org.datanucleus.TransactionImpl.internalPreCommit(TransactionImpl.java:379) > org.datanucleus.TransactionImpl.commit(TransactionImpl.java:268) > org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:98) > > org.apache.sentry.provider.db.service.persistent.TransactionManager.executeTransaction(TransactionManager.java:113) > > org.apache.sentry.provider.db.service.persistent.SentryStore.retrieveFullPermssionsImage(SentryStore.java:2225) > > org.apache.sentry.hdfs.PermImageRetriever.retrieveFullImage(PermImageRetriever.java:56) > > org.apache.sentry.hdfs.PermImageRetriever.retrieveFullImage(PermImageRetriever.java:40) > > org.apache.sentry.hdfs.DBUpdateForwarder.getAllUpdatesFrom(DBUpdateForwarder.java:85) > > org.apache.sentry.hdfs.SentryPlugin.getAllPermsUpdatesFrom(SentryPlugin.java:178) > > org.apache.sentry.hdfs.SentryHDFSServiceProcessor.get_all_authz_updates_from(SentryHDFSServiceProcessor.java:48) > > org.apache.sentry.hdfs.service.thrift.SentryHDFSService$Processor$get_all_authz_updates_from.getResult(SentryHDFSService.java:403) > > org.apache.sentry.hdfs.service.thrift.SentryHDFSService$Processor$get_all_authz_updates_from.getResult(SentryHDFSService.java:388) > org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > > org.apache.sentry.hdfs.SentryHDFSServiceProcessorFactory$ProcessorWrapper.process(SentryHDFSServiceProcessorFactory.java:47) > > org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123) > > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.15#6346)