[
https://issues.apache.org/jira/browse/IMPALA-9152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040248#comment-17040248
]
ASF subversion and git services commented on IMPALA-9152:
---------------------------------------------------------
Commit f403a96700e47df184ff782378342569be8f1c58 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f403a96 ]
IMPALA-9152: Make Impala ranger plugin singleton in FE Tests
Fix the flakiness of Ranger FE tests in AuthorizationStmtTest which is
caused by a row filter policy not being cleanly deleted. There is a bug
in Ranger that policies being deleted in Ranger Admin server but still
exist in Ranger plugins when there are concurrent create policy and get
policy requests (RANGER-2727). It's more possible to hit the bug if we
have more ranger plugins running, since each plugin instance will poll
policies in each 30s regularly.
Impalad and Catalogd servers only initialize one ImpalaRangerPlugin
instance. However, AuthorizationStmtTest has embedded Frontend and
CatalogServiceCatalog objects. It will initialize two ranger plugin
instances totally. What's worse, the JUnit testing framework makes a new
object for each test method run. Currently there are 29 test methods in
AuthorizationStmtTest, which means 29 AuthorizationStmtTest objects will
be created. So we finally have 58 ranger plugin instances running, which
makes RANGER-2727 easy to happen.
The failure can be reproduced by adding the following new test and run
it with all existing tests:
@Test
public void testRangerPolicyRepeatedly() throws Exception {
if (authzProvider_ == AuthorizationProvider.SENTRY) return;
for (int i = 0; i < 100; ++i) {
testRowFilterEnabled();
testColumnMaskEnabled();
}
}
We only explicitly create policies for column masking and row filtering
(other tests are using grant/revoke requests). This test increases the
number of CreatePolicy requests, so increases the possibility of
CreatePolicy requests running concurrently with GetPolicies requests
polling from other ranger plugin instances created by previous tests.
The fix is to make ImpalaRangerPlugin a singleton class so we will have
only one ranger plugin instance, which dramatically reduces the
possibility of hitting RANGER-2727. The thorough fix is bumping CDP
version after RANGER-2727 is resolved. Codes added in the previous patch
(c1244c2f04e629cc07b0830a597c70317be92768) are removed.
Tests:
- Ran AuthorizationStmtTest with the above new test.
Change-Id: I91f2bad1a9ce897b45cfc42f97b192fe2f8c7e06
Reviewed-on: http://gerrit.cloudera.org:8080/15235
Reviewed-by: Csaba Ringhofer <csringho...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> AuthorizationStmtTest.testColumnMaskEnabled failed in precommits.
> -----------------------------------------------------------------
>
> Key: IMPALA-9152
> URL: https://issues.apache.org/jira/browse/IMPALA-9152
> Project: IMPALA
> Issue Type: Bug
> Reporter: Anurag Mantripragada
> Assignee: Quanlong Huang
> Priority: Critical
>
> [~stigahuang] since you are going to work on this stuff, assigning it to you.
> Please feel free to reassign it. Thank you!
> Encountered here:
> [https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/8738]
> Looks like the test was expecting column masking error but encountered row
> filtering error.
> {code:java}
> got error:
> Impala does not support row filtering yet. Row filtering is enabled on table:
> functional.alltypes_view
> expected:
> Impala does not support column masking yet. Column masking is enabled on
> column: functional.alltypes_view.string_col {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
