sadpandajoe opened a new pull request, #43313:
URL: https://github.com/apache/superset/pull/43313
### SUMMARY
When a virtual dataset is created from SQL Lab's "Save & Explore" action,
the saved
dataset's `schema` is always set to whatever schema is selected in the SQL
Lab
dropdown — even when the query itself explicitly references a different
schema. For
example, selecting `information_schema` in the dropdown and then running
`select * from public."Vehicle Sales"` creates a dataset with
`schema=information_schema`,
not `schema=public`. This has been reported since 2021 (#16791) and affects
dataset
metadata correctness, RLS resolution, and permission checks that read the
dataset's
stored `schema`.
This PR derives the schema from the query itself when it is unambiguous, and
falls
back to today's dropdown-based behavior in every other case:
- The query is parsed with the existing Jinja-aware SQL parser
(`process_jinja_sql`,
the same one already used by the access-control check on this path) — no
new SQL
parsing is introduced.
- The dropdown schema is replaced with the query's schema **only** when the
query
references exactly one `(catalog, schema)` location, every referenced
table is
explicitly schema-qualified (zero unqualified references), the statement
doesn't
mutate data, doesn't change the default schema mid-script, doesn't contain
a
metadata (`SHOW ...`) statement, and doesn't reference any quoted
catalog/schema
identifier (conservative, since quoting can carry case-sensitive identity
on some
engines).
- Any other case — ambiguous, multi-schema, or unparseable queries — keeps
today's
behavior unchanged. This is an intentional, documented limitation, not an
oversight.
- Any failure in the optional parsing/derivation step (including malformed
`template_params` or a Jinja rendering error) safely falls back to the
submitted
schema rather than blocking dataset creation.
Out of scope for this PR (see `RCA.md` for details): the broader
multi-schema dataset
model proposed in community discussion, and permission/RLS enforcement based
on the
derived schema — both would be separate, larger changes.
`RCA.md` in this PR documents the full investigation: root cause, why it
wasn't
caught, the exact safety conditions and why each is needed, a verified
matrix of
parser behavior across ~20 query shapes, and known limitations of this fix.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Backend-only change (dataset creation logic); no UI change. Behavior
before/after for
the reported repro:
- **Before:** dropdown schema `information_schema` + query `select * from
public."Vehicle Sales"` → saved dataset `schema` = `information_schema` (wrong)
- **After:** same inputs → saved dataset `schema` = `public` (matches the
query)
### TESTING INSTRUCTIONS
1. `pytest tests/unit_tests/commands/dataset/test_create.py` — covers the
primary
repro case plus every fallback condition (zero-schema, multi-schema,
unqualified
reference, mutating statement, metadata statement, quoted identifier,
unparseable/
template-error SQL, Jinja partition-macro tables,
template-parameter-driven schema
selection, and positive catalog+schema derivation).
2. Manual repro: in SQL Lab, select a database and a schema in the
dropdowns, run a
query that explicitly references a *different* schema (e.g.
`select * from <other_schema>.<table>`), click Explore → name the dataset
→ Save &
Explore. Confirm the saved dataset's schema matches the query, not the
dropdown.
### ADDITIONAL INFORMATION
- [x] Has associated issue: Fixes #16791
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
