----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71915/ -----------------------------------------------------------
(Updated Dec. 20, 2019, 8:16 p.m.) Review request for sentry, kalyan kumar kalvagadda and Vihang Karajgaonkar. Bugs: sentry-2359 https://issues.apache.org/jira/browse/sentry-2359 Repository: sentry Description ------- Right now, the PolicyEngine Interface only returns the list of privileges in the form of String. As a result, every authorization check has to convert the privilege string to privilege object even though the cached privilege objects are of the correct type already. We should add a new function that returns privilege object directly to avoid the overhead of conversion. Diffs (updated) ----- sentry-binding/sentry-binding-hive-common/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBinding.java 5c7f84f sentry-binding/sentry-binding-hive-conf/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java 90fcfc3 sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzBindingHookBase.java de88705 sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/MetastoreAuthzBindingBase.java 2940a1e sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/SentryMetaStoreFilterHook.java 8e09490 sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/privilege/hive/TestCommonPrivilegeForHive.java 6a8b871 sentry-binding/sentry-binding-kafka/src/test/java/org/apache/sentry/privilege/kafka/TestKafkaWildcardPrivilege.java 0a0e2f0 sentry-binding/sentry-binding-solr/src/test/java/org/apache/sentry/privilege/solr/TestCommonPrivilegeForSolr.java 6782089 sentry-binding/sentry-binding-sqoop/src/test/java/org/apache/sentry/privilege/sqoop/TestCommonPrivilegeForSqoop.java 94e9919 sentry-core/sentry-core-common/src/main/java/org/apache/sentry/core/common/utils/KeyValue.java b6a1faa sentry-core/sentry-core-model-db/src/main/java/org/apache/sentry/core/model/db/DBModelAuthorizables.java 7bc94c9 sentry-core/sentry-core-model-db/src/main/java/org/apache/sentry/core/model/db/validator/AbstractDBPrivilegeValidator.java fa28716 sentry-policy/sentry-policy-common/src/main/java/org/apache/sentry/policy/common/CommonPrivilege.java 5b261e3 sentry-policy/sentry-policy-common/src/main/java/org/apache/sentry/policy/common/PolicyEngine.java 504b5ea sentry-policy/sentry-policy-common/src/main/java/org/apache/sentry/policy/common/Privilege.java 6c2737a sentry-policy/sentry-policy-engine/src/main/java/org/apache/sentry/policy/engine/common/CommonPolicyEngine.java a819bb0 sentry-provider/sentry-provider-cache/src/main/java/org/apache/sentry/provider/cache/FilteredPrivilegeCache.java PRE-CREATION sentry-provider/sentry-provider-cache/src/main/java/org/apache/sentry/provider/cache/PrivilegeCache.java 4bb6d32 sentry-provider/sentry-provider-cache/src/main/java/org/apache/sentry/provider/cache/SimpleCacheProviderBackend.java ddb4ec5 sentry-provider/sentry-provider-cache/src/main/java/org/apache/sentry/provider/cache/SimpleFilteredPrivilegeCache.java PRE-CREATION sentry-provider/sentry-provider-cache/src/main/java/org/apache/sentry/provider/cache/SimplePrivilegeCache.java 5de3135 sentry-provider/sentry-provider-cache/src/main/java/org/apache/sentry/provider/cache/TreePrivilegeCache.java PRE-CREATION sentry-provider/sentry-provider-cache/src/main/java/org/apache/sentry/provider/cache/TreePrivilegeNode.java PRE-CREATION sentry-provider/sentry-provider-cache/src/test/java/org/apache/sentry/provider/cache/PrivilegeCacheTestImpl.java f2f735b sentry-provider/sentry-provider-cache/src/test/java/org/apache/sentry/provider/cache/TestSimpleFilteredPrivilegeCache.java PRE-CREATION sentry-provider/sentry-provider-cache/src/test/java/org/apache/sentry/provider/cache/TestSimplePrivilegeCache.java 891c1d9 sentry-provider/sentry-provider-cache/src/test/java/org/apache/sentry/provider/cache/TestTreePrivilegeCache.java PRE-CREATION sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/CacheProvider.java d50a0bc sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ProviderBackend.java b244dba sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/ResourceAuthorizationProvider.java 222b77a sentry-provider/sentry-provider-common/src/test/java/org/apache/sentry/provider/common/TestGetGroupMapping.java ccc505f sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/SimpleDBProviderBackend.java 277f6b3 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/SentryGenericProviderBackend.java f8dc211 sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestColumnEndToEnd.java 3881692 sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java 4c09e68 sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java cc0465a sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestMetastoreEndToEnd.java cb201bb Diff: https://reviews.apache.org/r/71915/diff/8/ Changes: https://reviews.apache.org/r/71915/diff/7-8/ Testing ------- Performance test in 1) Privilege: db[1000], table[10] Authorizable: db[12000], table[1] TreePrivilegeCache - total time on list string: 448 ms (TestTreePrivilegeCache.testListPrivilegesPerf) TreePrivilegeCache - total time on list obj: 365 ms (TestTreePrivilegeCache.testListPrivilegeObjectsPerf) SimplePrivilegeCache - total time on list string: 717 ms (TestSimplePrivilegeCache.testListPrivilegesPerf) SimplePrivilegeCache - total time on list obj: 794 ms (TestSimplePrivilegeCache.testListPrivilegeObjectsPerf) 2) Privilege: db[1000], table[10] Authorizable: db[12000], table[10] TreePrivilegeCache - total time on list string: 1302 ms (TestTreePrivilegeCache.testListPrivilegesPerf) TreePrivilegeCache - total time on list obj: 604 ms (TestTreePrivilegeCache.testListPrivilegeObjectsPerf) SimplePrivilegeCache - total time on list string: 4436 ms (TestSimplePrivilegeCache.testListPrivilegesPerf) SimplePrivilegeCache - total time on list obj: 3732 ms (TestSimplePrivilegeCache.testListPrivilegeObjectsPerf) Thanks, Na Li