Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/20742 )
Change subject: IMPALA-11501: Add flag to allow catalog-cache operations on masked tables ...................................................................... Patch Set 1: (5 comments) Thanks for the help Quanlong! In this round of review I tried to familiarize myself with the behavior of Impala after the flag is introduced based on the test cases added in test_ranger.py and found some differences between 2 catalog modes. I will continue the review and let you know if there are additional questions or comments on other parts of the patch. http://gerrit.cloudera.org:8080/#/c/20742/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20742/1//COMMIT_MSG@40 PS1, Line 40: doens't nit: doesn't http://gerrit.cloudera.org:8080/#/c/20742/1//COMMIT_MSG@41 PS1, Line 41: can't nit: not able to http://gerrit.cloudera.org:8080/#/c/20742/1//COMMIT_MSG@44 PS1, Line 44: lookup nit: look up http://gerrit.cloudera.org:8080/#/c/20742/1/tests/authorization/test_ranger.py File tests/authorization/test_ranger.py: http://gerrit.cloudera.org:8080/#/c/20742/1/tests/authorization/test_ranger.py@1571 PS1, Line 1571: non_owner_client Should this be renamed to 'non_admin_client' or 'owner_client'? It seems in this test we use this client to submit queries as the user getuser(), which is the owner of the table involved in the test and is a non-administrative user. http://gerrit.cloudera.org:8080/#/c/20742/1/tests/authorization/test_ranger.py@1608 PS1, Line 1608: self.execute_query_expect_success( : non_owner_client, : "refresh functional.alltypestiny partition(year=2009, month=1)", user=user) I observed the following 2 differences between the legacy and the local catalog modes when '--allow_catalog_cache_op_from_masked_users=true' is added. Don't know if I missed something. Are those differences expected? The 1st difference: with respect to the REFRESH on a table. Specifically, in the legacy catalog mode, I found this REFRESH on partitions is already allowed even without 'user' executing "describe functional.alltypestiny", whereas REFRESH on the same table is denied. In addition, as shown in the following, in the legacy catalog mode, after a REFRESH on partitions is executed, the same "refresh functional.alltypestiny" will be allowed, which I think/guess is due to the table metadata being loaded (just like what happens after "describe functional.alltypestiny"). [localhost:21050] default> refresh functional.alltypestiny; Query: refresh functional.alltypestiny Query submitted at: 2023-12-03 11:38:56 (Coordinator: http://fangyu-upstream-dev.gce.cloudera.com:25000) ERROR: AuthorizationException: User 'fangyurao' does not have privileges to execute 'INVALIDATE METADATA/REFRESH' on: functional.alltypestiny [localhost:21050] default> refresh functional.alltypestiny partition(year=2009, month=1); Query: refresh functional.alltypestiny partition(year=2009, month=1) Query submitted at: 2023-12-03 11:39:04 (Coordinator: http://fangyu-upstream-dev.gce.cloudera.com:25000) Query state can be monitored at: http://fangyu-upstream-dev.gce.cloudera.com:25000/query_plan?query_id=ba4e95cf064405e3:22bc528e00000000 [localhost:21050] default> refresh functional.alltypestiny; Query: refresh functional.alltypestiny Query submitted at: 2023-12-03 11:39:10 (Coordinator: http://fangyu-upstream-dev.gce.cloudera.com:25000) Query state can be monitored at: http://fangyu-upstream-dev.gce.cloudera.com:25000/query_plan?query_id=7041cfffdf9bafac:0d1b547600000000 On the other hand, in the local catalog mode, both REFRESH on table and REFRESH on partitions are allowed. The 2nd difference: with respect to INVALIDATE METADATA on a table following a REFRESH on the same table. In the legacy catalog mode, a subsequent INVALIDATE METADATA on the same table is allowed once a REFRESH on partitions of this table is executed. However, in the local catalog mode, even we execute both a REFRESH on the table and a REFRESH on partitions of the same table as 'user' successfully, an INVALIDATE METADATA on the same table is not allowed. In this case 'user' has to execute DESCRIBE on the table to execute INVALIDATE METADATA on the table successfully. -- To view, visit http://gerrit.cloudera.org:8080/20742 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I45935654cbf05a55d740f1b04781022c271f7678 Gerrit-Change-Number: 20742 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Comment-Date: Sun, 03 Dec 2023 21:23:09 +0000 Gerrit-HasComments: Yes
