LantaoJin opened a new pull request, #84:
URL: https://github.com/apache/datafusion-java/pull/84
## Which issue does this PR close?
- Closes #83 .
## Rationale for this change
PR #28 added `tempDirectory(String)` as the only Java surface for
DataFusion's `RuntimeEnvBuilder` disk-manager knobs. Two real gaps remain —
both reachable on the Rust side, neither reachable from Java today, and neither
reachable via `setOption(...)` because they live on `RuntimeEnv` construction
rather than the `datafusion.*` config namespace (the native side already
explicitly rejects `datafusion.runtime.*` keys, for the same reason that
motivated the existing typed `memoryLimit(...)` / `tempDirectory(...)` setters):
- **No way to disable spill entirely.** Useful for memory-only execution
profiles or environments without writable disk.
- **No way to cap the spill volume size.** A runaway sort or hash-aggregate
can fill the spill disk — on a multi-tenant node, a co-tenant outage rather
than just a query failure. This matches the `--disk-limit` flag that
`datafusion-cli` ships.
Issue #83 originally included a third setter,
`tempDirectories(List<String>)`, but a follow-up survey of the upstream code,
CLI, docs, examples, and Python binding showed multi-directory mode is a
programmatic-only escape hatch on `DiskManagerBuilder`: not exposed by
`RuntimeEnvBuilder` itself, not used by `datafusion-cli`, not in `configs.md`,
not in any example. It can ship later if a real user files an issue with a use
case; punting it keeps this PR focused on the two setters with concrete
motivation.
## What changes are included in this PR?
- New setters on `SessionContextBuilder`:
- `disableSpill()`.
- `maxTempDirectorySize(long)`
- Proto: new `DiskManagerOptions` message (optional `disabled`, optional
`max_temp_directory_size`) added to `session_options.proto`.
- Native: one new arm in `createSessionContextWithOptions` that decodes
`DiskManagerOptions` and routes through `with_disk_manager_builder(Disabled)` /
`with_max_temp_directory_size`.
## Are these changes tested?
Yes. 9 new tests in `SessionContextBuilderTest`
## Are there any user-facing changes?
Yes, additive only — no breaking changes.
- Two new public setters on `SessionContextBuilder`. Existing callers using
only `tempDirectory(String)` see no behaviour change and produce identical wire
bytes.
- The new mutually-exclusive combination throws `IllegalStateException` at
`build()` time (not at the setter call), matching the `setOption` runtime-key
pattern.
- No Javadoc / user-guide updates in this PR; the new setters' Javadoc
carries the v1 surface description, consistent with how recent additions (e.g.
`cacheManager`, `registerObjectStore`) shipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
