tanishqgandhi1908 opened a new pull request, #8000:
URL: https://github.com/apache/texera/pull/8000
### What changes were proposed in this PR?
Model uploads were capped at 20 MiB. The shared upload engine read the
dataset's `single_file_upload_max_size_mib` key literally, so both resource
types resolved the same site setting and models inherited the dataset ceiling.
**Feature.** A `model { … }` section in `default.conf` with
`model_single_file_upload_max_size_mib = 2048` (2 GiB) plus the three
concurrency/chunk keys, and `gui.tabs.models_enabled = false`. The leaves carry
a `model_` prefix because `site_settings` rows are keyed by a leaf's last path
segment and `DefaultsConfig` requires those to be unique across
sections.`ConfigResource` now derives its public whitelist from
`gui`/`dataset`/`model`, so the new keys reach `/config/settings/public` — the
anonymous-exposure surface, and the reason the pinned set in
`ConfigResourceSpec` was updated in this PR.
**Refactor.** New `UploadLimits` descriptor ties a resource type to the
site-setting keys that bound its uploads (key + default per limit) and exposes
`singleFileUploadMaxBytes`. `ResourceStorage` — the existing per-resource
storage
descriptor — gains an `uploadLimits` slot, so both call sites in
`ResourceUploadService` read the limit through the descriptor instead of naming
a key. No new registry is introduced; an existing one gains a slot.
`private def singleFileUploadMaxBytes` also existed in `DatasetResource` and
`ModelResource`, where it was dead — defined, never called. Both copies are
removed along with their now-unused `SiteSettings` imports.
`gui.tabs.models_enabled` defaults to `false` and is toggleable in Admin →
Settings. It hides the sidebar entry.
### Any related issues, documentation, discussions?
Backend prerequisite for the model dashboard UI — #6499 and #6501. No issue
filed
for the 20 MiB cap; it was found while scoping that work.
### How was this PR tested?
New `UploadLimitsSpec` (5 tests, no DB harness — `SiteSettings` falls back
to the declared default when `SqlServer` is uninitialised). It pins each
descriptor's keys and defaults against `default.conf`, asserts the dataset and
model key sets are
disjoint, asserts the model ceiling resolves to 2 GiB, and checks the
`ResourceStorage` wiring.
Existing suites, all passing unmodified except the two pinned config
assertions:
- `Config/testOnly *DefaultsConfigSpec` — 5/5
- `ConfigService/testOnly *ConfigResourceSpec` — 33/33
- `FileService/testOnly *UploadLimitsSpec` — 5/5
- `FileService/testOnly *ModelUploadResourceSpec` — 13/13
- `FileService/testOnly *DatasetResourceSpec` — 152/152 (covers the
max-upload path)
`scalafmtCheckAll` clean.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
