Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-5386: Fix ReopenCachedHdfsFileHandle failure case ......................................................................
IMPALA-5386: Fix ReopenCachedHdfsFileHandle failure case This fixes three issues with the file handle cache. The first issue is that ReopenCachedHdfsFileHandle can destroy the passed in file handle without removing the reference to it. The old file handle then refers to a piece of memory that is not a handle in the cache, so future use of the handle fails with an assert. The fix is to always overwrite the reference to the file handle when it has been destroyed. The second issue is that query_test/test_hdfs_fd_caching.py should run on anything that supports the hdfs commandline and tolerate query failure. It's logic is not specific to file handle caching, so it has been renamed to query_test/test_hdfs_file_mods.py. Finally, custom_cluster/test_hdfs_fd_caching.py should not be running on remote files (S3, ADLS, Isilon, remote clusters). The file handle cache semantics won't apply on those platforms. Change-Id: Iee982fa5e964f6c8969b2eb7e5f3eca89e793b3a Reviewed-on: http://gerrit.cloudera.org:8080/7020 Reviewed-by: Joe McDonnell <joemcdonn...@cloudera.com> Reviewed-by: Dan Hecht <dhe...@cloudera.com> Tested-by: Impala Public Jenkins --- M be/src/runtime/disk-io-mgr-handle-cache.inline.h M be/src/runtime/disk-io-mgr.cc M tests/common/skip.py M tests/custom_cluster/test_hdfs_fd_caching.py R tests/query_test/test_hdfs_file_mods.py 5 files changed, 28 insertions(+), 18 deletions(-) Approvals: Impala Public Jenkins: Verified Dan Hecht: Looks good to me, approved Joe McDonnell: Looks good to me, but someone else must approve -- To view, visit http://gerrit.cloudera.org:8080/7020 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iee982fa5e964f6c8969b2eb7e5f3eca89e793b3a Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com>
