GitHub user mengw15 created a discussion: Per-user warehouses are
feature-complete behind a flag; proposing to turn it on
Per-user warehouses (#6870) now work end to end behind
`storage.warehouse.enabled`, which has been off since the first commit. Each
user owns their own warehouses — as many as they need — instead of everything
sharing one. Everything has landed in `main`: the storage foundation and
REST-catalog work, the warehouse REST endpoints and Lakekeeper client, the
dashboard tab (#8005), the on-canvas picker (#8551), and the backend
requirement that every execution name a warehouse (#8586).
I would like to propose turning the flag on, and to hear objections before
anyone does.
### Demo
*(silent recording — here is what it shows, in order)*
1. The **Warehouses** dashboard tab: creating a warehouse, the row it produces,
and the details dialog its name opens.
2. The **warehouse picker** on the canvas, next to the computing-unit selector,
choosing where the next run writes.
3. With no warehouse left, the **Run button becomes "Create Warehouse"** and
leads straight to the create dialog; once a warehouse exists, Run comes back.
4. A run started from the **form view** with no warehouse selected is refused
with a message, rather than quietly writing to shared storage.
5. A normal execution, writing into the selected warehouse.
### What changes for users when it is on
- A **Warehouses** tab appears in the dashboard: create, inspect and delete
your own warehouses.
- A **warehouse picker** appears on the canvas, choosing where the next run
writes.
- **Every execution must name a warehouse.** Until you have one, Run becomes
"Create Warehouse"; runs started elsewhere (form view, run-up-to-operator,
replay, and agent-driven runs) are refused with a message rather than silently
writing to shared storage.
- Nothing is provisioned automatically — each user creates their own.
### What it requires from a deployment
The switch only takes effect with the REST catalog: `StorageConfig` keeps the
feature off unless the Iceberg catalog type is `rest` (Lakekeeper), because
warehouses are Lakekeeper entities. Deployments on another catalog type are
unaffected by the flag entirely.
### What it does not change
- **Results written before the flip stay readable.** Their storage URIs carry
no warehouse segment and continue to resolve to the shared default warehouse.
- **The switch is reversible, and fails loudly rather than silently.** Turned
back off, a result that lives in a per-user warehouse is refused with an
explicit message ("this result is stored in warehouse X; per-user warehouses
are disabled in this deployment") instead of surfacing as "table not found",
which would be indistinguishable from data loss. Nothing is deleted;
re-enabling restores access.
- **Where the data lives.** Each warehouse is its own Lakekeeper catalog with
its own key prefix inside the deployment's existing storage. Isolation is at
the catalog level, not at the bucket level.
### The ask
Speak up if enabling this would break something on a deployment you care about
— particularly anything affecting existing users, or if the "every execution
needs a warehouse" requirement looks too strict as a Phase 0 default. Absent
objections, I will turn it on.
GitHub link: https://github.com/apache/texera/discussions/8633
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
