mikebridge opened a new pull request, #45100:
URL: https://github.com/apache/superset/pull/45100
### SUMMARY
Saving semantic-layer configuration invalidates its metadata caches so a
subsequent page reload can fetch upstream changes. A small **Reload metadata**
action also clears caches for upstream-only changes: it confirms the clear and
closes the editor, then the user reloads the affected Explore or dashboard tab.
The host atomically increments a per-layer `cache_version` in the metadata
database, in the same transaction as a configuration Save. Description-only
edits do not increment it. The UUID-addressed `clear_cache` endpoint accepts
`{}`, requires connection-write, layer-access and connection-modification
authority, and performs no provider discovery. Captured workspace/layer/version
tokens partition result, annotation, compatibility and column-value caches
while retaining their existing authorization and query dimensions.
Annotation cache keys follow the datasource named by the chart's saved query
context, matching annotation execution. This also changes RLS-key sourcing for
table-backed annotation charts; malformed contexts still fail at execution.
The scope excludes passwords, query options and process secrets. Operators
can supply a stable, globally unique workspace namespace shared by web, Celery
and MCP when connection routing differs. Old entries expire normally.
Concurrent reads can still finish using old metadata; this is best-effort
invalidation with no coordinated refresh, snapshot store, leases or fencing.
The shared catalog store from #44835, refresh/inspection operations from
#44849 and sync-state UI from #44851 are replaced by a database generation, one
authorized clear endpoint and ordinary page reloads.
This replaces the coordinated-refresh design for
[SC-121047](https://app.shortcut.com/preset/story/121047) and **supersedes
#44835, #44849 and #44851**. Those PRs remain open pending the owner's decision.
A separate commit removes the unused, unreleased refresh/snapshot interfaces
from #44834 (**16 additions / 951 deletions**). The existing SDK
`SemanticView.metadata_cache_token: str | None` contract is preserved. An
optional `from_configuration_with_cache_token` hook keeps legacy providers
compatible. **Complete invalidation also requires providers with local metadata
caches to adopt this token before cache lookup/eager discovery; host deployment
alone cannot clear those dictionaries.** #42760 must include the host ORM view
token in containment keys, even when the provider token is `None`.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before: saving connection configuration did not rotate the host metadata
cache generation, and the editor had no upstream-only clear action.
After: configuration Save invalidates automatically. A saved, unmodified
editor exposes **Reload metadata**; success displays “Cache cleared; reload to
fetch metadata” and closes the editor. Dirty/pending forms disable the action.
No browser walkthrough or screenshot is claimed; the interaction is covered by
Jest.
### TESTING INSTRUCTIONS
1. Enable `SEMANTIC_LAYERS`, open an existing layer, change its
configuration and Save. Reload the affected Explore/dashboard tab and verify it
reads with the new cache generation.
2. For an upstream-only change, open the unchanged configuration editor,
choose **Reload metadata**, observe confirmation, then reload the affected tab.
Use a token-aware provider to verify its process cache misses too.
3. Verify a dirty form disables Reload, rejected clears preserve the editor,
and users without connection-write/modification authority cannot clear.
4. Run the Python unit suite, semantic-layer and SQL 100% coverage gates,
and `SemanticLayerModal.test.tsx`.
Local validation at `047d5d9045acdf3ce0829585726c6944eebfc27a`:
semantic-layer **742 passed / 100% coverage**; SQL **2,304 passed / 100%
coverage**; modal Jest **7 passed**. Whole suite: **22,320 passed, 40 skipped,
2 xfailed, 2 failed** in 785.05s under `TZ=UTC`. The only failures are the
known macOS ARM `np.longdouble` numeric-serialization cases
(`dataframe_test.py::test_df_to_records_preserves_finite_longdouble_and_nulls_nonfinite`
and
`common/test_query_context_processor.py::test_get_data_json_preserves_browser_numeric_contract`);
those tests and numeric methods are unchanged from master. The process
lingered after the complete summary and was stopped; this is a
baseline-exception result, not a zero-failure run. Red-first cache-scope and UI
regressions are recorded. SQLite tests cover upgrade/backfill/downgrade,
transactional rollback and successive stale-writer increments; PostgreSQL/MySQL
migration execution and live providers remain for CI/downstream verification.
Changed-file formatting/lint/MyPy hooks pass. The normal frontend type hook
hits stale declarations; the authorized declaration refresh fails with TS2742
portability errors through this machine's shared dependency symlink. The strict
source-resolved affected-file type check passes. POT extraction and OpenAPI
generation leave their checked-in outputs unchanged.
### ADDITIONAL INFORMATION
- [ ] Has associated issue: SC-121047 is downstream tracking; public
design/review context is #44834, #44835, #44849 and #44851 above
- [x] Required feature flags: `SEMANTIC_LAYERS`
- [x] Changes UI
- [x] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [x] Migration is atomic, supports rollback & is backwards-compatible
- [x] Confirm DB migration upgrade and downgrade tested (SQLite)
- [x] Runtime estimates and downtime expectations provided: one
default-zero integer column on the connection table; no provider I/O or Python
row loop. SQLite validation completes within the focused test run; production
DDL locking/runtime depends on backend and table size and has not been
benchmarked. Apply the migration before deploying code that reads the column.
- [x] Introduces new feature or API
- [x] Removes existing feature or API: unused, unreleased #44834 refresh
contracts
During a rolling deployment, old application instances cannot increment the
new generation; repeat Reload metadata after rollout if configuration was saved
by an old instance. Each clear can cold-start provider caches; it does not
clear vendor warehouse caches or automatically execute charts.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
