Hi community, I'd like to propose DSIP-110: Support Custom Datasource Types via External Plugins.
Motivation DolphinScheduler's datasource layer is pluginized in name but not in fact. The plugin contract is hard-wired to the compiled-in DbType enum — DataSourceProcessor#getDbType() returns a compile-time enum, the t_ds_datasource.type column stores the enum's integer code, and both frontend surfaces (the datasource creation dialog and the task-node datasource selector) hardcode the type lists. As a result, an externally compiled plugin jar can only override a built-in type; it can never introduce a new one. Users whose databases are not among the 28 built-in types have no supported extension path short of forking the project. Goal A deployment administrator can install an externally compiled datasource plugin package — without modifying or recompiling any DolphinScheduler code — and the new type becomes usable end-to-end: visible in the UI, full lifecycle (create / connectivity test / authorize / metadata browsing), and executable in SQL and Procedure tasks. Installation is dropping the plugin jar into the plugins directory and restarting the servers. Design summary (full detail in the GitHub issue) String-based type identity in the SPI contract (DataSourceProcessor / param DTOs / plugin manager registry), with fail-fast duplicate-name detection; DbType degrades to named constants for built-ins. Type registry + discovery API: GET /datasources/types returns installed types with metadata (label, default port, capability flags) — the single contract for both frontend surfaces. Persistence: t_ds_datasource.type converted from tinyint (enum code) to varchar (type name), with upgrade scripts for MySQL/PostgreSQL. api-server adaptation: string-based routing, unknown-type validation, capability-driven dialect defaults. Task execution path: master/worker contexts and SQL/Procedure/DataX/Sqoop call sites switch from DbType.valueOf to string resolution. Frontend: dynamic type lists from the API; custom types render a generic JDBC fallback form (host/port/user/password/database/other). Packaging & driver conventions for external plugin jars (single shaded jar bundling the JDBC driver, install on all api/master/worker nodes). Alignment with the project security model: plugin installation is an administrator/deployment action, inside the existing trust boundary. A reference H2 datasource plugin (DbType contains H2 but no plugin module exists today) is planned as the living example and the CI test vehicle for the whole path. Open design decisions I'd especially like feedback on D1 — Fate of DbType: recommended to keep it as a deprecated constants holder, with all runtime paths using string type names (mirrors how task plugins use String taskType). D2 — Storage: recommended in-place type column conversion tinyint → varchar(64) with data migration, over a side mapping table. D3 — UI for custom types: recommended a generic JDBC fallback form for v1; plugin-declared form schema deferred to a follow-up DSIP. D4 — v1 task coverage: SQL + Procedure only; DataX/Sqoop stay built-in-only via the existing whitelist mechanism. D5 — Exact response shape of GET /datasources/types. I already added a GitHub Issue for my proposal, which you could see in: https://github.com/apache/dolphinscheduler/issues/18600 Looking forward any feedback for this thread. Best regards, eye-gu
