Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13849
Change subject: IMPALA-8486: fix stale libCache entries in LocalCatalog mode ...................................................................... IMPALA-8486: fix stale libCache entries in LocalCatalog mode In LocalCatalog mode, after a function is dropped, statestored will broadcast the update to invalidate the cached CatalogObject in each coordinator (if they have). However, the current code path does not trigger libCache to remove the cached JAR/SO file. If we replace the function file in HDFS with a new one and create the function again using the same HDFS path, the SELECT statements in other coordinators won't trigger libCache to refresh the local cached file, so they still use the old cached file which causes errors. When a coordinator invalidates its cached CatalogObject of a function, it should also mark the corresponding libCache entry as "needs refresh". So the later usage of this function will check the last modified time of the HDFS file and refresh it in needs. To achieve this, we have to propagate the HDFS path of the function along with the full function name in the minimal topic, so libCache can target the cached entry. Tests - Re-enable test_udf_update_via_drop and test_udf_update_via_create for LocalCatalog mode. Change-Id: Ie4812fb8737de3ba6074ffeb9007927bfbbbaf9b --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M tests/common/skip.py M tests/query_test/test_udfs.py 4 files changed, 10 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/13849/3 -- To view, visit http://gerrit.cloudera.org:8080/13849 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie4812fb8737de3ba6074ffeb9007927bfbbbaf9b Gerrit-Change-Number: 13849 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
