Hello Tim Armstrong, Bikramjeet Vig, Dan Hecht, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/9089 to look at the new patch set (#6). Change subject: IMPALA-6215: Removes race when using LibCache. ...................................................................... IMPALA-6215: Removes race when using LibCache. LibCache's api to provide access to locally cached files has a race. Currently, the client of the cache accesses the locally cached path as a string, but nothing guarantees that the associated file is not removed before the client is done using it. This race is suspected as the root cause for the flakiness seen in IMPALA-6092. These tests fail once in a while with classloader errors unable to load java udf classes. In these tests, the lib cache makes no guarantee that the path to the jar will remain valid from the time the path is acquired through the time needed to fetch the jar and resolve the needed classes. LibCache offers liveness guarantees for shared objects via reference counting. The fix in this patch extends this API to also cover paths to locally cached files. Testing: - added a test to test_udfs.py that does many concurrent udf uses and removals. By increasing the concurrent operations to 100, the issue in IMPALA-6092 is locally reproducible on every run. With this fix, the problem is no longer reproducible with this test. Change-Id: I9175085201fe8b11424ab8f88d7b992cb7b0daea --- M be/src/codegen/llvm-codegen.cc M be/src/exec/external-data-source-executor.cc M be/src/exprs/hive-udf-call.cc M be/src/exprs/hive-udf-call.h M be/src/runtime/lib-cache.cc M be/src/runtime/lib-cache.h M be/src/service/fe-support.cc M tests/query_test/test_udfs.py 8 files changed, 166 insertions(+), 51 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/89/9089/6 -- To view, visit http://gerrit.cloudera.org:8080/9089 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9175085201fe8b11424ab8f88d7b992cb7b0daea Gerrit-Change-Number: 9089 Gerrit-PatchSet: 6 Gerrit-Owner: Vuk Ercegovac <vercego...@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com>