Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16392
Change subject: [WIP] MPALA-10075: Reuse unchanged partition instances ...................................................................... [WIP] MPALA-10075: Reuse unchanged partition instances Currently, we always update the partition instance when we reload a partition. If a partition remains the same after reloading, we should reuse the old partition instance. So we won't send redudant updates on these partitions. This reduces the size of the catalog topic update. When a huge table is REFRESHed, catalogd only propagates the changes partitions. Tests: - Manually verified that repeate REFERSHs won't trigger catalog updates on partitions. - (TODO) add e2e tests Change-Id: I2dd645c260d271291021e52fdac4b74924df1170 --- M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java 4 files changed, 30 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/16392/1 -- To view, visit http://gerrit.cloudera.org:8080/16392 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2dd645c260d271291021e52fdac4b74924df1170 Gerrit-Change-Number: 16392 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
