bharath v created IMPALA-8444: --------------------------------- Summary: Analysis perf regression after IMPALA-7616 Key: IMPALA-8444 URL: https://issues.apache.org/jira/browse/IMPALA-8444 Project: IMPALA Issue Type: Bug Affects Versions: Impala 3.2.0 Reporter: bharath v Assignee: Fredy Wijaya
The patch for IMPALA-7616 caused a performance regression in analysis time when run in an environment with ~1k roles and 10.5. privileges. The regression is evident when run as a role that has a large number of privileges. Following is the stack to look for when jstacking the coordnator. {noformat} "Thread-21" #49 prio=5 os_prio=0 tid=0x000000000cd6e000 nid=0x6a3d runnable [0x00007fa28e4a0000] java.lang.Thread.State: RUNNABLE at java.lang.String.toLowerCase(String.java:2670) at org.apache.impala.catalog.PrincipalPrivilege.buildPrivilegeName(PrincipalPrivilege.java:82) at org.apache.impala.catalog.PrincipalPrivilege.getName(PrincipalPrivilege.java:143) at org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:423) - locked <0x00007fa376987100> (a org.apache.impala.catalog.AuthorizationPolicy) at org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:443) - locked <0x00007fa376987100> (a org.apache.impala.catalog.AuthorizationPolicy) at org.apache.sentry.provider.cache.SimpleCacheProviderBackend.getPrivileges(SimpleCacheProviderBackend.java:75) at org.apache.sentry.policy.db.SimpleDBPolicyEngine.getPrivileges(SimpleDBPolicyEngine.java:98) at org.apache.sentry.provider.common.ResourceAuthorizationProvider.getPrivileges(ResourceAuthorizationProvider.java:147) at org.apache.sentry.provider.common.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:120) at org.apache.sentry.provider.common.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:107) at org.apache.impala.authorization.AuthorizationChecker.hasAccess(AuthorizationChecker.java:215) at org.apache.impala.authorization.AuthorizationChecker.checkAccess(AuthorizationChecker.java:128) at org.apache.impala.analysis.AnalysisContext.authorizePrivilegeRequest(AnalysisContext.java:592) at org.apache.impala.analysis.AnalysisContext.authorize(AnalysisContext.java:564) at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:415) at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1240) at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1210) at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1182) at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:158) {noformat} Issue worsens when running concurrent workloads, because the underlying Sentry {{listPrivileges()}} is synchronized and that serializes all the query analysis requests. {noformat} public synchronized Set<String> listPrivileges(Set<String> groups, <==== ActiveRoleSet roleSet) { Set<String> privileges = Sets.newHashSet(); if (roleSet != ActiveRoleSet.ALL) { throw new UnsupportedOperationException("Impala does not support role subsets."); } {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org