eschutho opened a new pull request, #44278: URL: https://github.com/apache/superset/pull/44278
### SUMMARY Part of epic [sc-120993](https://app.shortcut.com/preset/story/120993) (migrate DB connections to the new dynamic connection form). Story: https://app.shortcut.com/preset/story/120996 Superset renders the dynamic connection form when the engine spec resolved from a database's driver defines a `parameters_schema`. Databricks connections on the legacy `databricks+pyodbc` driver resolve to `DatabricksODBCEngineSpec`, which has **no** `parameters_schema`, so they only ever expose the raw "SQLAlchemy URI" text field. The `databricks+connector` driver resolves to `DatabricksNativeEngineSpec`, which **does** define a `parameters_schema` and therefore renders the dynamic form. This adds an Alembic migration that rewrites existing `databricks+pyodbc` databases to `databricks+connector` so they render the new form. Per the fleet audit only 11 connections across 6 teams use the legacy ODBC driver. **Why this is a safe, scheme-only swap.** Both drivers share the exact same URL shape: ``` databricks+<driver>://token:<access_token>@<host>:<port>/<database> ``` and both read `http_path`, `catalog` and `schema` from `extra.engine_params.connect_args`, **not** from the URL. So the migration only rewrites the scheme prefix (`databricks+pyodbc` → `databricks+connector`) and leaves the rest of the URI byte-for-byte identical. #### ODBC → connector field mapping | Field | ODBC (`databricks+pyodbc`) | connector (`databricks+connector`, `DatabricksNativeEngineSpec`) | Change | |---|---|---|---| | access token | URL password (`token:<pat>@`) | URL password | none | | host | URL host | URL host | none | | port | URL port | URL port | none | | database / catalog name | URL path (`/<database>`) | URL path | none | | http_path | `extra.engine_params.connect_args.http_path` | same | none | | catalog / schema | `extra.engine_params.connect_args` | same (dynamic schema/catalog) | none | | encryption | `?ssl=1` query param (if present) | same | none | | **driver** | `pyodbc` | `connector` | **rewritten** | `encrypted_extra` (access token storage, OAuth2 client info) is not touched. #### Rollback `upgrade()` writes a marker key (`migrated_from_databricks_odbc: true`) into each migrated row's `extra`. `downgrade()` reverts **only** rows carrying that marker (stripping it on the way back), so a `databricks+connector` connection a user created directly is never rewritten to `pyodbc`. The round trip is exact and both directions are idempotent (they filter on the driver prefix and no-op when nothing matches). ### TESTING INSTRUCTIONS - `pytest tests/unit_tests/migrations/test_migrate_databricks_odbc_to_connector.py` (9 tests: scheme-swap helper, upgrade touches only ODBC rows, native connector rows untouched, idempotency, downgrade marker protection, full round trip). - Manual: create a database with a `databricks+pyodbc://token:<pat>@host:443/db` URI and `http_path` in Extra → engine params → connect_args; run `superset db upgrade`; confirm the connection now resolves to the "Databricks" native connector spec and the edit modal shows the dynamic form; run `superset db downgrade` and confirm the URI reverts to `databricks+pyodbc` and the marker is gone. ### ADDITIONAL INFORMATION - [x] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [x] Migration is atomic, supports rollback & is backwards-compatible - [x] Confirm DB migration upgrade and downgrade tested - [x] Runtime estimates and downtime expectations provided (touches only `databricks+pyodbc` rows — ~11 fleet-wide; paginated update; no schema/DDL change, no downtime) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
