Quanlong Huang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18978 )
Change subject: IMPALA-11580: Fix memory leak in legacy catalog mode when applying incremental partition updates ...................................................................... IMPALA-11580: Fix memory leak in legacy catalog mode when applying incremental partition updates In the legacy catalog mode, catalogd propagates incremental metadata updates at the partition level. While applying the updates, impalad reuses the existing partition objects and moves them to a new HdfsTable object. However, the partition objects are immutable, which means their reference to the old table object remains unchanged. JVM GC cannot collect the stale table objects since they still have active reference from the partitions, which results in memory leak. This patch fixes the issue by recreating a new partition object based on the existing partition object with the new table field. Tests: - Verified locally that after applying the patch, I don’t see the number of live HdfsTable objects keeps bumping. Change-Id: Ie04ff243c6b82c1a06c489da74353f2d8afe423a Reviewed-on: http://gerrit.cloudera.org:8080/18978 Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Reviewed-by: Csaba Ringhofer <csringho...@cloudera.com> --- M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java 2 files changed, 11 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Verified Csaba Ringhofer: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/18978 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie04ff243c6b82c1a06c489da74353f2d8afe423a Gerrit-Change-Number: 18978 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
