[
https://issues.apache.org/jira/browse/IMPALA-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981499#comment-16981499
]
xuzhou commented on IMPALA-9195:
--------------------------------
I didn't note IMPALA-9002, but I also had tried the method it introduced and
'show tables' can benifit a lot when users only having a few tables' access
among lots of tables in the database, however it will benifit little if the
users have most tables' access as 'SELECT' privilege has been placed in the
first in enum DBModelAction(SENTRY-1292).
So I tried using multithreaded execution to accelerate show tables/databases.
It's my pleasure to contribute code to impala. Please assign this Jira to me,
thanks.
> Using multithreaded execution to accelerate ‘show tables/databases’
> -------------------------------------------------------------------
>
> Key: IMPALA-9195
> URL: https://issues.apache.org/jira/browse/IMPALA-9195
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: xuzhou
> Priority: Critical
>
> Impala version: 2.12
> Using sentry for authentication
> While users with multi group-policies(group-policy may be nested) executing
> 'show tables/databases'，it seems to be awful with a long latency. In my case,
> the database has 910 tables, the user waiting 65.886 seconds to get 160
> tables.
> I study the code and find that while executing Frontend.getTableNames:
> for table in tables:
> for action in actions(all actions defined in DBModelAction):
> ResourceAuthorizationProvider.hasAccess
> It seems that 'hasAccess' is responsable for bad performance while checking
> users with complex group-policies.
> I tried to use 16 threads in getTablesNames and it costs 4.752 seconds in my
> case.
> The code seems to be the same while using sentry service in the latest
> impala. I'm not sure that if any promotion has been done in the latest sentry
> service as I failed to migrate file-based sentry authentication to the sentry
> service. I see that ranger is supported in the latest impala, does ranger
> have the similar problem?
> It seems 'show tables/databases' can benefit from multithreaded execution
> while using sentry , is it reasonable to support such operations in query
> option MT_DOP?
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
