Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/11027 )
Change subject: IMPALA-7320. Avoid calling getFileStatus() for each partition when table is loaded ...................................................................... Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/11027/2/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/11027/2/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1557 PS2, Line 1557: preloadPermissionsCache(msPartitions, permCache); > Same question as above. Could this be an overkill if size(msPartitions) is I added a simple heuristic here that compares size(msPartitions) vs the number of existing partitions, and only uses this trick if the number of new partitions is 3x the number of existing partitions. That should catch RECOVER PARTITIONS use case when basically constructing the entire hierarchy from an existing table path, but not affect "alter table add" or RECOVER when it just adds a few. Let me know what you think -- To view, visit http://gerrit.cloudera.org:8080/11027 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I83e5ebc214d6620d165e13f8cc80f8fdda100734 Gerrit-Change-Number: 11027 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com> Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com> Gerrit-Comment-Date: Thu, 26 Jul 2018 01:42:03 +0000 Gerrit-HasComments: Yes
