Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13600 )
Change subject: wip: [sentry] use bulk Sentry API for ListTables ...................................................................... Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13600/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13600/2//COMMIT_MSG@10 PS2, Line 10: However, : testing with mini sentry which uses derby, the performance has not been : improved much. Without this patch: > It's also worth noting the tests you ran that got worse. The numbers here s Yeah, the above looks like an 'improvement' and sorry I forgot to paste the comparison for cases that got worse: Without my patch: ./bin/sentry_authz_provider-test --gtest_filter=*ListTablesBench* --num_databases=300 --num_tables_per_db=1 --has-db-privileges=true sentry_authz_provider-test.cc:413] Time spent Listing tables: real 24.290s user 0.121s sys 0.000s With my patch $ ./bin/sentry_authz_provider-test --gtest_filter=*ListTablesBench* --num_databases=300 --num_tables_per_db=1 --has-db-privileges=true sentry_client.cc:182] Time spent list Sentry privileges by authorizables and user: real 40.920s user 0.006s sys 0.000s sentry_authz_provider-test.cc:414] Time spent Listing tables: real 40.959s user 0.025s sys 0.001s So it looks like the more privileges that need to be returned from the bulk API the slower it gets. In cases I bump number of database/tables to larger number (e.g. 1000), the Sentry RPC can timeout. Given the above investigations, discussed with Andrew and Alexey offline, we decided to abandon this change. Because with existing approach, even ListTables can timeout for cases with many tables, it will warm the cache and successive ListTables from the same user should succeed. If we switch to use the bulk API, in the worst cases the Sentry RPC time out (when returning a large number of privileges), no results will be the cached and ListTables can fail consistently for the user. -- To view, visit http://gerrit.cloudera.org:8080/13600 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4c17befbcd423637f97971672fe8bca40311d214 Gerrit-Change-Number: 13600 Gerrit-PatchSet: 2 Gerrit-Owner: Hao Hao <hao....@cloudera.com> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Hao Hao <hao....@cloudera.com> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Fri, 14 Jun 2019 22:44:47 +0000 Gerrit-HasComments: Yes
