Yida Wu created IMPALA-12720: -------------------------------- Summary: Investigate reference deleted hdfs files in spilling tests Key: IMPALA-12720 URL: https://issues.apache.org/jira/browse/IMPALA-12720 Project: IMPALA Issue Type: Bug Components: Backend Affects Versions: Impala 4.3.0 Reporter: Yida Wu Assignee: Yida Wu
This is following jira for IMPALA-12698. While running the test TestScratchDir.test_scratch_dirs_batch_reading, it was observed that certain hdfs files in /proc/xxxx/fd appeared as deleted. This suggests a delay in the immediate release of file handles. However, in local tests, it was noticed that the files are eventually deleted from the fd path after a few minutes. Further investigation may be worthwhile to understand the underlying cause of this symptom. {code:java} lr-x------ 1 impala impala 64 Jan 9 23:40 750 -> '/impala/Impala/testdata/cluster/cdh7/node-2/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920999_180368.meta (deleted)' lr-x------ 1 impala impala 64 Jan 9 23:40 751 -> '/impala/Impala/testdata/cluster/cdh7/node-1/data/dfs/dn/current/BP-1389948453-127.0.0.1-1700061594492/current/finalized/subdir2/subdir27/blk_1073920938 (deleted)' {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)