morningman opened a new pull request, #61604: URL: https://github.com/apache/doris/pull/61604
### What problem does this PR solve? Issue Number: close #xxx Problem Summary: This PR introduces a Service Provider Interface (SPI) framework for external datasource catalogs, enabling dynamic loading and ClassLoader isolation for catalog plugins. ES (Elasticsearch) is migrated as the first pilot. **Phase 1 — SPI Framework:** - `CatalogProvider` SPI interface with methods for catalog lifecycle management - `CatalogProviderRegistry` thread-safe provider registry - `CatalogPluginLoader` with URLClassLoader isolation for plugin JARs - FE startup integration: `loadPlugins()` called before `loadImage()` **Phase 2 — ES Migration (Pilot):** - `EsCatalogProvider` implementing CatalogProvider SPI - `CatalogFactory`: SPI provider lookup before switch-case fallback - `ExternalCatalog.buildDbForInit`: SPI provider lookup before switch-case fallback - `ExternalCatalog`: Three abstract methods converted to concrete SPI-delegating defaults (`initLocalObjectsImpl`, `listTableNamesFromRemote`, `tableExist`) with transient `provider` field - `GsonUtils`: ES types changed from `registerSubtype` to `registerCompatibleSubtype` for plugin-agnostic persistence - `PhysicalPlanTranslator.visitPhysicalEsScan`: SPI-based ScanNode creation with fallback - `fe-catalogs/catalog-es/` Maven module with shade plugin for self-contained plugin JAR - ES source files migrated to independent module **Key Design Decisions:** - **Plugin-agnostic persistence**: All external catalogs serialize/deserialize as base types. Old EditLog entries (e.g. `"clazz":"EsExternalCatalog"`) are handled via `registerCompatibleSubtype` - **Lazy initialization**: Plugins only loaded when catalog is first accessed via `makeSureInitialized()` - **Graceful degradation**: Missing plugin marks catalog as unavailable without blocking FE startup - **ClassLoader isolation**: Each plugin gets isolated URLClassLoader preventing dependency conflicts ### Release note None ### Check List (For Author) - Test: No need to test - structural refactoring with backward-compatible SPI fallback - Behavior changed: No - Does this need documentation: No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
