Huaisi Xu has uploaded a new patch set (#2). Change subject: IMPALA-1702: Enforce unique table ID ......................................................................
IMPALA-1702: Enforce unique table ID The problem is that loaded tables can be sent together with INCOMPLETE tables after INVALIDATE METADATA. These tables may share same table ID with some of the tables previously because we reset() table ID counter when INVALIDATE METADATA. This can be problematic when this update goes to planner during analyzing: tables not yet analyzed have the same table ID as those already analyzed. This further causes problem in backend since we keep a map mapping from table ID to table metadata, and tables with same table ID overwrites each other in the map. This commit enforces unique table ID for the lifetime of catalog daemon by keeping a map recording already seen table and reuse their ID whenever possible. This also increases memory consumption upper bounded by the size of all database name + table name. Change-Id: Ifad648b72684ae495ec387590ab1bc58ce5b39e2 --- M fe/src/main/java/com/cloudera/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/com/cloudera/impala/catalog/IncompleteTable.java M fe/src/main/java/com/cloudera/impala/catalog/Table.java M fe/src/main/java/com/cloudera/impala/catalog/TableId.java M fe/src/main/java/com/cloudera/impala/catalog/TableLoader.java M fe/src/main/java/com/cloudera/impala/catalog/TableLoadingMgr.java M fe/src/main/java/com/cloudera/impala/common/Id.java M fe/src/main/java/com/cloudera/impala/service/CatalogOpExecutor.java 8 files changed, 53 insertions(+), 35 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/4349/2 -- To view, visit http://gerrit.cloudera.org:8080/4349 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifad648b72684ae495ec387590ab1bc58ce5b39e2 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Huaisi Xu <h...@cloudera.com> Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com> Gerrit-Reviewer: Huaisi Xu <h...@cloudera.com>
