[
https://issues.apache.org/jira/browse/IMPALA-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yida Wu resolved IMPALA-12681.
------------------------------
Fix Version/s: Impala 4.4.0
Target Version: Impala 4.4.0
Resolution: Fixed
> Some local file descriptors not released when using remote spilling
> -------------------------------------------------------------------
>
> Key: IMPALA-12681
> URL: https://issues.apache.org/jira/browse/IMPALA-12681
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.0.0, Impala 4.1.0, Impala 4.2.0
> Reporter: Yida Wu
> Assignee: Yida Wu
> Priority: Major
> Fix For: Impala 4.4.0
>
>
> The bug occurs during remote spilling when writing spilled data to local
> buffers. In this process, if the files are not completely filled, such as
> when no more data is incoming at the end of spilling, there is a possibility
> of partial writes to the files, the files might be physically removed without
> properly releasing the associated file descriptor. This issue can be observed
> in cases like the one described below.
> {code:java}
> find /proc/*/fd -ls | grep '(deleted)'
> 288574785 0 lrwx------ 1 impala impala 64 Jan 3 14:24 /proc/x/fd/xxxx ->
> /opt/impala/scratch/impala-scratch/impala-scratch-xxxxxxxxx-xxxx-xxxx\
> (deleted) {code}
> In such a scenario, the disk space occupied by the file may not be reclaimed
> because the file descriptor still maintains a reference to the file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
