Yida Wu created IMPALA-12971:
--------------------------------
Summary: Incorrect capacity of remote scratch dir when both local
and remote scratch dir are configured
Key: IMPALA-12971
URL: https://issues.apache.org/jira/browse/IMPALA-12971
Project: IMPALA
Issue Type: Bug
Components: Backend
Reporter: Yida Wu
Assignee: Yida Wu
The current implementation in AllocateRemoteSpace()
(https://github.com/apache/impala/blob/4be5fd8896dcd445a6379bdcda4bdcf318f24511/be/src/runtime/tmp-file-mgr.cc#L1391)
has a bug when both local and remote scratch directories are configured.
Currently, the function utilizes current_bytes_allocated_ to determine whether
to allocate space based on the remote directory limit (capacity). However, this
approach is flawed when the system allows spilling to other local directories,
as current_bytes_allocated_ represents the total used scratch space rather than
the usage of the remote directory alone, potentially causing allocation
failures when the combined usage (local + remote) exceeds the remote limit.
This bug happens when both remote and local scratch space are configured, and a
limit is set to the remote directory (normally remote capacity is set to
unlimited)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
