[
https://issues.apache.org/jira/browse/SENTRY-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15982745#comment-15982745
]
Hadoop QA commented on SENTRY-1687:
-----------------------------------
Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12864894/SENTRY-1687.001-sentry-ha-redesign.patch
against sentry-ha-redesign.
{color:red}Overall:{color} -1 due to 2 errors
{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed:
org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationEnd2End
Console output:
https://builds.apache.org/job/PreCommit-SENTRY-Build/2561/console
This message is automatically generated.
> FullUpdateInitializer can be more efficient
> -------------------------------------------
>
> Key: SENTRY-1687
> URL: https://issues.apache.org/jira/browse/SENTRY-1687
> Project: Sentry
> Issue Type: Sub-task
> Components: Sentry
> Affects Versions: sentry-ha-redesign
> Reporter: Alexander Kolbasov
> Assignee: Alexander Kolbasov
> Attachments: SENTRY-1687.001-sentry-ha-redesign.patch
>
>
> The FullUpdateInitializer follows the {{MetastoreCacheInitializer}}. It reads
> a bunch of information from HMS and uses Thrift structures to pass around,
> but in the end it just constructs {{Map<String, Set<String>>}}. It uses
> concurrent fetches from HMS, but synchronizes a lot on common data structures
> to update them.
> I think that we can refactor all this code to make it faster and consume less
> memory. The idea is the following:
> Use background threads to collect Thrift results from HMS calls (database,
> table and partition data). Then we can use a single thread to construct the
> resulting update and return it without using intermediate Thrift methods.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
