[ https://issues.apache.org/jira/browse/SENTRY-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962495#comment-15962495 ]
Hadoop QA commented on SENTRY-1687: ----------------------------------- Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12862638/SENTRY-1687.001-sentry-ha-redesign.patch against sentry-ha-redesign. {color:green}Overall:{color} +1 all checks pass {color:green}SUCCESS:{color} all tests passed Console output: https://builds.apache.org/job/PreCommit-SENTRY-Build/2494/console This message is automatically generated. > FullUpdateInitializer can be more efficient > ------------------------------------------- > > Key: SENTRY-1687 > URL: https://issues.apache.org/jira/browse/SENTRY-1687 > Project: Sentry > Issue Type: Sub-task > Components: Sentry > Affects Versions: sentry-ha-redesign > Reporter: Alexander Kolbasov > Assignee: Alexander Kolbasov > Priority: Minor > Attachments: SENTRY-1687.001-sentry-ha-redesign.patch > > > The FullUpdateInitializer follows the {{MetastoreCacheInitializer}}. It reads > a bunch of information from HMS and uses Thrift structures to pass around, > but in the end it just constructs {{Map<String, Set<String>>}}. It uses > concurrent fetches from HMS, but synchronizes a lot on common data structures > to update them. > I think that we can refactor all this code to make it faster and consume less > memory. The idea is the following: > Use background threads to collect Thrift results from HMS calls (database, > table and partition data). Then we can use a single thread to construct the > resulting update and return it without using intermediate Thrift methods. -- This message was sent by Atlassian JIRA (v6.3.15#6346)