Yida Wu created IMPALA-12971: -------------------------------- Summary: Incorrect capacity of remote scratch dir when both local and remote scratch dir are configured Key: IMPALA-12971 URL: https://issues.apache.org/jira/browse/IMPALA-12971 Project: IMPALA Issue Type: Bug Components: Backend Reporter: Yida Wu Assignee: Yida Wu
The current implementation in AllocateRemoteSpace() (https://github.com/apache/impala/blob/4be5fd8896dcd445a6379bdcda4bdcf318f24511/be/src/runtime/tmp-file-mgr.cc#L1391) has a bug when both local and remote scratch directories are configured. Currently, the function utilizes current_bytes_allocated_ to determine whether to allocate space based on the remote directory limit (capacity). However, this approach is flawed when the system allows spilling to other local directories, as current_bytes_allocated_ represents the total used scratch space rather than the usage of the remote directory alone, potentially causing allocation failures when the combined usage (local + remote) exceeds the remote limit. This bug happens when both remote and local scratch space are configured, and a limit is set to the remote directory (normally remote capacity is set to unlimited) -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org