[ https://issues.apache.org/jira/browse/IMPALA-8444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16825394#comment-16825394 ]
bharath v commented on IMPALA-8444: ----------------------------------- [~csringhofer] Nice catch. That should make it much faster. > Analysis perf regression after IMPALA-7616 > ------------------------------------------ > > Key: IMPALA-8444 > URL: https://issues.apache.org/jira/browse/IMPALA-8444 > Project: IMPALA > Issue Type: Bug > Components: Frontend > Affects Versions: Impala 3.2.0 > Reporter: bharath v > Assignee: Fredy Wijaya > Priority: Critical > > The patch for IMPALA-7616 caused a performance regression in analysis time > when run in an environment with ~1k roles and 10.5k. privileges. The > regression is evident when run as a role that has a large number of > privileges. > Following is the stack to look for when jstacking the coordnator. > {noformat} > "Thread-21" #49 prio=5 os_prio=0 tid=0x000000000cd6e000 nid=0x6a3d runnable > [0x00007fa28e4a0000] > java.lang.Thread.State: RUNNABLE > at java.lang.String.toLowerCase(String.java:2670) > at > org.apache.impala.catalog.PrincipalPrivilege.buildPrivilegeName(PrincipalPrivilege.java:82) > at > org.apache.impala.catalog.PrincipalPrivilege.getName(PrincipalPrivilege.java:143) > at > org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:423) > - locked <0x00007fa376987100> (a > org.apache.impala.catalog.AuthorizationPolicy) > at > org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:443) > - locked <0x00007fa376987100> (a > org.apache.impala.catalog.AuthorizationPolicy) > at > org.apache.sentry.provider.cache.SimpleCacheProviderBackend.getPrivileges(SimpleCacheProviderBackend.java:75) > at > org.apache.sentry.policy.db.SimpleDBPolicyEngine.getPrivileges(SimpleDBPolicyEngine.java:98) > at > org.apache.sentry.provider.common.ResourceAuthorizationProvider.getPrivileges(ResourceAuthorizationProvider.java:147) > at > org.apache.sentry.provider.common.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:120) > at > org.apache.sentry.provider.common.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:107) > at > org.apache.impala.authorization.AuthorizationChecker.hasAccess(AuthorizationChecker.java:215) > at > org.apache.impala.authorization.AuthorizationChecker.checkAccess(AuthorizationChecker.java:128) > at > org.apache.impala.analysis.AnalysisContext.authorizePrivilegeRequest(AnalysisContext.java:592) > at > org.apache.impala.analysis.AnalysisContext.authorize(AnalysisContext.java:564) > at > org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:415) > at > org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1240) > at > org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1210) > at > org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1182) > at > org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:158) > {noformat} > Issue worsens when running concurrent workloads, because the underlying > Sentry {{listPrivileges()}} is synchronized and that serializes all the query > analysis requests. > {noformat} > public synchronized Set<String> listPrivileges(Set<String> groups, <==== > ActiveRoleSet roleSet) { > Set<String> privileges = Sets.newHashSet(); > if (roleSet != ActiveRoleSet.ALL) { > throw new UnsupportedOperationException("Impala does not support role > subsets."); > } > {noformat} > Notes: > - If the authorization metadata footprint is small, this issue can be ignored. > - One workaround is to run the query using a role that has very small number > of privileges (revoke privileges that are not necessary to run a given query). > - Another workaround is to disable Sentry authorization. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org