Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/17054 )
Change subject: IMPALA-10497: Fix flakiness in test_no_fd_caching_on_cached_data. ...................................................................... Patch Set 1: (1 comment) Thanks Joe for the feedback. I will revert most of these and leave only the extra warm up part. http://gerrit.cloudera.org:8080/#/c/17054/1/tests/custom_cluster/test_hdfs_fd_caching.py File tests/custom_cluster/test_hdfs_fd_caching.py: http://gerrit.cloudera.org:8080/#/c/17054/1/tests/custom_cluster/test_hdfs_fd_caching.py@190 PS1, Line 190: cluster_size=1 > I think this should not impact the caching. Our scheduling algorithm maps f I will remove this so the cluster size back to default. Previously, when I kept cluster size as default (3), after several iteration of looping this test, one impalad failed with the following error. I0210 17:50:00.781529 10340 mem_tracker.cc:83] Creating tracker /tmp-sharded_lru_cache->root I0210 17:50:00.782321 10340 status.cc:129] Failed to delete old cache file /tmp/impala-cache-file-fc4060e3ab7e45b0:a680f0d3b05005a1: Not found: /tmp/impala-cache-file-fc4060e3ab7e45b0:a680f0d3b05005a1: No such file or directory (error 2) @ 0x2c008d7 _ZN6impala13GetStackTraceB5cxx11Ev @ 0x1c9003e impala::Status::Status() @ 0x36016f9 impala::io::DataCache::Partition::DeleteExistingFiles() @ 0x3601e22 impala::io::DataCache::Partition::Init() @ 0x360975e impala::io::DataCache::Init() @ 0x35b4db0 impala::io::DiskIoMgr::Init() @ 0x251b229 impala::ExecEnv::Init() @ 0x29a0412 ImpaladMain() @ 0x1bdf4d3 main @ 0x7faa8e3bebf7 __libc_start_main @ 0x1ae81da _start I0210 17:50:00.897989 10340 mem_tracker.cc:87] Destroying tracker /tmp-sharded_lru_cache->root I was guessing that since all three write their data cache to common /tmp/ dir, one might accidentally delete cache file that belong to other impalad. Just running the test one time doesn't reveal this error. So in this patchset 1, I follow examples from custom_cluster/test_data_cache.py to set custom_cluster=1. -- To view, visit http://gerrit.cloudera.org:8080/17054 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I774f9dfea7dcc107c3c7f2b76db3aaf4b2dd7952 Gerrit-Change-Number: 17054 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Comment-Date: Thu, 11 Feb 2021 02:18:19 +0000 Gerrit-HasComments: Yes
