Yida Wu created IMPALA-12720:
--------------------------------
Summary: Investigate reference deleted hdfs files in spilling tests
Key: IMPALA-12720
URL: https://issues.apache.org/jira/browse/IMPALA-12720
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.3.0
Reporter: Yida Wu
Assignee: Yida Wu
This is following jira for IMPALA-12698. While running the test
TestScratchDir.test_scratch_dirs_batch_reading, it was observed that certain
hdfs files in /proc/xxxx/fd appeared as deleted. This suggests a delay in the
immediate release of file handles. However, in local tests, it was noticed that
the files are eventually deleted from the fd path after a few minutes. Further
investigation may be worthwhile to understand the underlying cause of this
symptom.
{code:java}
lr-x------ 1 impala impala 64 Jan 9 23:40 750 ->
'/impala/Impala/testdata/cluster/cdh7/node-2/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920999_180368.meta
(deleted)'
lr-x------ 1 impala impala 64 Jan 9 23:40 751 ->
'/impala/Impala/testdata/cluster/cdh7/node-1/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920938
(deleted)'
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
