Yida Wu has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17906 )
Change subject: IMPALA-10953 Fix impalad crashes due to TmpFileMgr initialization fails ...................................................................... IMPALA-10953 Fix impalad crashes due to TmpFileMgr initialization fails The impalad can start successfully with an uninitialized TmpFileMgr when abort_on_config_error is set to false. However in some cases, this could lead to the use of the uninitialized TmpFileMgr and then the crash. The purpose of keeping a failed TmpFileMgr is to allow the users to use impala even with minor configuration issues because only spilling the data needs the TmpFileMgr and queries might have a small chance to spill the data. Previously we have a DCHECK for the initialization status of the TmpFileMgr, however it doesn't work for the release version, and the way of using DCHECK to handle this situation is not good enough if we need to keep the impala running. Instead of putting verifications to every function of the TmpFileMgr, this patch fixes the bug by verifying the initialization status when creating the TmpFileGroup. It returns an error if TmpFileMgr fails to initialize. When the QueryState instance fails to create the TmpFileGroup, it stores a nullptr to the local temporary file group and fails the query only when it needs to spill the data. Moreover, TmpFileGroup can be only created by the TmpFileMgr after this change. Tests: Ran core tests and exhaustive e2e tests. Added a e2e test: test_scratch_dirs_spill_fails_due_to_init_failure. Added a unit test: BufferPoolTest::FailInitTmpFileMgr. Change-Id: I7852bf4206a603e6e2652ce9232077777a06ce67 --- M be/src/runtime/bufferpool/buffer-pool-test.cc M be/src/runtime/io/disk-io-mgr-test.cc M be/src/runtime/query-state.cc M be/src/runtime/test-env.cc M be/src/runtime/test-env.h M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc M be/src/runtime/tmp-file-mgr.h M tests/custom_cluster/test_scratch_disk.py 9 files changed, 336 insertions(+), 217 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/06/17906/2 -- To view, visit http://gerrit.cloudera.org:8080/17906 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7852bf4206a603e6e2652ce9232077777a06ce67 Gerrit-Change-Number: 17906 Gerrit-PatchSet: 2 Gerrit-Owner: Yida Wu <wydbaggio...@gmail.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
