Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-4943: Speed up block md loading for add/recover partition calls. ......................................................................
IMPALA-4943: Speed up block md loading for add/recover partition calls. This change makes alter table add/recover partitions calls use the per directory block metadata loading routines instead of doing it per file. This is done since these calls always load the entire partition directory from scratch and there is no advantage in loading them incrementally on a per-file basis. Tests: Ran core tests and the metadata benchmark tests. (I) Improvement: METADATA-BENCHMARKS() 100K-PARTITIONS-1M-FILES-03-RECOVER [text / none / none] (718.62s -> 549.91s [-23.48%]) (I) Improvement: METADATA-BENCHMARKS() 100K-PARTITIONS-1M-FILES-08-ADD-PARTITION [text / none / none] (46.92s -> 26.20s [-44.15%]) Change-Id: I331f1f090518f317bcd7df069e480edbd8f039f1 Reviewed-on: http://gerrit.cloudera.org:8080/6651 Reviewed-by: Bharath Vissapragada <bhara...@cloudera.com> Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java 1 file changed, 20 insertions(+), 5 deletions(-) Approvals: Impala Public Jenkins: Verified Bharath Vissapragada: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/6651 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I331f1f090518f317bcd7df069e480edbd8f039f1 Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins
