This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new 476c200f00 refactor: Set expected runtime config in error message when
the used disk space during the spilling process has exceeded the allocation
limit (#20375)
476c200f00 is described below
commit 476c200f00cdd061d834fd09b73f5799c12ac252
Author: Eren Avsarogullari <[email protected]>
AuthorDate: Tue Mar 3 01:58:23 2026 -0800
refactor: Set expected runtime config in error message when the used disk
space during the spilling process has exceeded the allocation limit (#20375)
## Which issue does this PR close?
- Closes #20373.
## Rationale for this change
Minor refactoring on error message by exposing required config name for
the end user. This is follow-up PR to both PR: #20226 and #20372 by
runtime config: `datafusion.runtime.max_temp_directory_size`.
## What changes are included in this PR?
**Current:**
```
The used disk space during the spilling process has exceeded the allowable
limit of {}. Try increasing the `max_temp_directory_size` in the disk manager
configuration.
```
**New:**
```
The used disk space during the spilling process has exceeded the allowable
limit of {}. \
Please try increasing the config:
`datafusion.runtime.max_temp_directory_size`.",
```
## Are these changes tested?
Yes, legacy UT case has been updated by covering expected config name.
## Are there any user-facing changes?
Yes, error message has been updated which is exposed to end-users.
---
datafusion/core/tests/memory_limit/mod.rs | 15 ++++++++++-----
datafusion/execution/src/disk_manager.rs | 3 ++-
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/datafusion/core/tests/memory_limit/mod.rs
b/datafusion/core/tests/memory_limit/mod.rs
index c28d23ba06..ff8c512cbd 100644
--- a/datafusion/core/tests/memory_limit/mod.rs
+++ b/datafusion/core/tests/memory_limit/mod.rs
@@ -602,11 +602,16 @@ async fn test_disk_spill_limit_reached() -> Result<()> {
.await
.unwrap();
- let err = df.collect().await.unwrap_err();
- assert_contains!(
- err.to_string(),
- "The used disk space during the spilling process has exceeded the
allowable limit"
- );
+ let error_message = df.collect().await.unwrap_err().to_string();
+ for expected in [
+ "The used disk space during the spilling process has exceeded the
allowable limit",
+ "datafusion.runtime.max_temp_directory_size",
+ ] {
+ assert!(
+ error_message.contains(expected),
+ "'{expected}' is not contained by '{error_message}'"
+ );
+ }
Ok(())
}
diff --git a/datafusion/execution/src/disk_manager.rs
b/datafusion/execution/src/disk_manager.rs
index d878fdcf66..1a14bd239a 100644
--- a/datafusion/execution/src/disk_manager.rs
+++ b/datafusion/execution/src/disk_manager.rs
@@ -420,7 +420,8 @@ impl RefCountedTempFile {
let global_disk_usage =
self.disk_manager.used_disk_space.load(Ordering::Relaxed);
if global_disk_usage > self.disk_manager.max_temp_directory_size {
return resources_err!(
- "The used disk space during the spilling process has exceeded
the allowable limit of {}. Try increasing the `max_temp_directory_size` in the
disk manager configuration.",
+ "The used disk space during the spilling process has exceeded
the allowable limit of {}. \
+ Please try increasing the config:
`datafusion.runtime.max_temp_directory_size`.",
human_readable_size(self.disk_manager.max_temp_directory_size
as usize)
);
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
