[ https://issues.apache.org/jira/browse/IMPALA-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yida Wu resolved IMPALA-12681. ------------------------------ Fix Version/s: Impala 4.4.0 Target Version: Impala 4.4.0 Resolution: Fixed > Some local file descriptors not released when using remote spilling > ------------------------------------------------------------------- > > Key: IMPALA-12681 > URL: https://issues.apache.org/jira/browse/IMPALA-12681 > Project: IMPALA > Issue Type: Bug > Components: Backend > Affects Versions: Impala 4.0.0, Impala 4.1.0, Impala 4.2.0 > Reporter: Yida Wu > Assignee: Yida Wu > Priority: Major > Fix For: Impala 4.4.0 > > > The bug occurs during remote spilling when writing spilled data to local > buffers. In this process, if the files are not completely filled, such as > when no more data is incoming at the end of spilling, there is a possibility > of partial writes to the files, the files might be physically removed without > properly releasing the associated file descriptor. This issue can be observed > in cases like the one described below. > {code:java} > find /proc/*/fd -ls | grep '(deleted)' > 288574785 0 lrwx------ 1 impala impala 64 Jan 3 14:24 /proc/x/fd/xxxx -> > /opt/impala/scratch/impala-scratch/impala-scratch-xxxxxxxxx-xxxx-xxxx\ > (deleted) {code} > In such a scenario, the disk space occupied by the file may not be reclaimed > because the file descriptor still maintains a reference to the file. -- This message was sent by Atlassian Jira (v8.20.10#820010)