morningman opened a new pull request, #70338:
URL: https://github.com/apache/doris/pull/70338
### Background
SHOW CREATE CATALOG, and the other places that print catalog properties,
replace a value with `*XXX` when `DatasourcePrintableMap.isSensitiveKey`
matches its key. The keys come from a list hard-coded in fe-core and from the
keys that filesystem plugins report at startup through
`FileSystemProvider.sensitivePropertyKeys()`. A connector plugin tells the
engine about its security-relevant properties through
`ConnectorProvider#securitySurface()`, which today lists only the SSRF endpoint
keys and the jdbc `driver_url` keys that the CREATE and ALTER CATALOG gates
check.
### What problem does this PR solve?
Issue Number: None
Related PR: #69588 (added the connector security surface this extends),
#69575 (master loads the Lance catalogs that 4.1.4 persisted), #65730
(branch-4.1's Lance catalog, whose two REST secrets 4.x masks in fe-core)
PR Type: Enhancement
Problem Summary:
A connector plugin parses its own catalog properties, but their secret
values are masked only if fe-core hard-codes the key names, which the connector
framework rules out. The Lance connector being ported from branch-4.x needs
this for `lance.rest.bearer-token` and `lance.rest.api-key`: 4.1.4 masks them,
and without a way to declare them master would print them in clear.
`ConnectorSecuritySurface` can now declare secret keys, and the engine adds an
admitted provider's keys to its masking list at FE startup. Nothing changes
after an upgrade until a connector declares keys, and none does in this PR.
### Details
- **Before and after:** Before, a property that only a connector knows, such
as `lance.rest.bearer-token`, showed its value in SHOW CREATE CATALOG. After, a
key the connector declares with
`ConnectorSecuritySurface.ofSensitiveKeys(...)`, or with
`.withSensitiveKeys(...)` on top of its other declarations, shows `*XXX`,
matched case-insensitively like the engine's own list; undeclared keys show as
before.
- **Configs and defaults:** None. The default surface declares no secret.
- **Impact on existing users:** None for SQL, metadata, or configs. As for
every change to the connector SPI surface, the plugin API major moves from 30.0
to 31.0, so a third-party connector plugin built against 30.x has to be
rebuilt; the in-tree plugins are built with the FE.
### Testing
All runs were local (macOS), on master `59c49656240` plus this change:
- `mvn -f fe/pom.xml -pl :fe-connector-spi -am test`: the module's 156 tests
pass. They include the new `ConnectorSecuritySurfaceTest`, which covers
combining secrets with the SSRF and driver_url declarations, appending, and
that no existing surface, `EMPTY` included, is changed. They also include
`ConnectorPluginSurfaceTest` with the refreshed baseline and 31.0.
- fe-core, 12 classes and 157 tests, all passing.
`ConnectorPluginManagerTest` has three new cases:
- an admitted provider's key is masked, case-insensitively, in the
rendering SHOW CREATE CATALOG uses, and undeclared keys stay visible;
- a refused provider adds nothing;
- the `registerProvider` test seam registers keys too.
- The other eleven classes register providers or check the security surface,
such as `PluginDrivenExternalCatalog{SsrfGate,DriverUrlGate}Test`,
`CatalogFactoryPluginRoutingTest` and `DatasourcePrintableMapTest`.
- Six mutations, each caught by the test written for it:
- not registering;
- registering before the admission checks;
- skipping the test seam;
- three in `withSensitiveKeys`: dropping the SSRF keys, replacing instead
of appending, and returning a mutable list.
No connector declares keys yet, so nothing changes end to end. The Lance
connector, ported in follow-up PRs, will be the first to declare them.
### Notes for Reviewers
- The keys are read when a provider is registered and added only after the
admission checks in `ConnectorPluginManager#registerDiscovered` pass, so a
refused provider adds nothing. Connector plugins load only at FE startup,
before any catalog property is shown, as filesystem plugins do.
`registerProvider`, the seam tests use to stand in for a plugin, registers the
keys too, like `FileSystemPluginManager#registerProvider`.
- A catalog whose connector plugin is not loaded (a degraded catalog) keeps
only the engine's own list.
- Why the surface rather than a provider method like
`FileSystemProvider.sensitivePropertyKeys()`: the connector SPI already gathers
its security-relevant property keys on the surface, and a connector with both
an endpoint and a secret declares them on one object.
### Release note
None
### Check List (For Author)
- Behavior changed: No
- Does this need documentation: No
- Pre-submission status: Ready
### AI Disclosure
- Tool: Claude Code (Claude Opus 5.5)
- Generated: the code, the tests, and this description
- Human review: not stated
- Prompts: The author asked for a generic connector SPI to mask a
connector's secret catalog properties instead of adding the Lance keys to
fe-core's list, approved the `ConnectorSecuritySurface` design, and asked for
it as a separate PR ahead of the Lance connector port.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
