[
https://issues.apache.org/jira/browse/KAFKA-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16486495#comment-16486495
]
ASF GitHub Bot commented on KAFKA-6886:
---------------------------------------
rayokota opened a new pull request #5068: KAFKA-6886 Externalize secrets from
Connect configs
URL: https://github.com/apache/kafka/pull/5068
This commit allows secrets in Connect configs to be externalized and
replaced with variable references of the form `${provider:[path:]key}`.
There are 2 main additions to `org.apache.kafka.common.config`: a
`ConfigProvider` and a `ConfigTransformer`. The `ConfigProvider` is an
interface that allows key-value pairs to be provided by an external source for
a given "path". An a TTL can be associated with the key-value pairs returned
from a "path". The `ConfigTransformer` will use instances of `ConfigProvider`
to replace variable references in a set of configuration values.
In the Connect framework, `ConfigProvider` classes can be specified in the
worker config, and then variable references can be used in the connector
config. In addition, the herder can be configured to restart connectors (or
not) based on the TTL returned from a `ConfigProvider`. The main class that
performs restarts and transformations is `WorkerConfigTransformer`.
Finally, a `configs()` method has been added to both `SourceTaskContext` and
`SinkTaskContext`. This allows connectors to get configs with variables
replaced by the latest values from instances of `ConfigProvider`.
Most of the other changes in the Connect framework are threading various
objects through classes to enable the above functionality.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Externalize Secrets for Kafka Connect Configurations
> ----------------------------------------------------
>
> Key: KAFKA-6886
> URL: https://issues.apache.org/jira/browse/KAFKA-6886
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect
> Reporter: Robert Yokota
> Assignee: Robert Yokota
> Priority: Major
> Fix For: 2.0.0
>
>
> Kafka Connect's connector configurations have plaintext passwords, and
> Connect stores these in cleartext either on the filesystem (for standalone
> mode) or in internal topics (for distributed mode).
> Connect should not store or transmit cleartext passwords in connector
> configurations. Secrets in stored connector configurations should be allowed
> to be replaced with references to values stored in external secret management
> systems. Connect should provide an extension point for adding customized
> integrations, as well as provide a file-based extension as an example.
> Second, a Connect runtime should be allowed to be configured to use one or
> more of these extensions, and allow connector configurations to use
> placeholders that will be resolved by the runtime before passing the complete
> connector configurations to connectors. This will allow existing connectors
> to not see any difference in the configurations that Connect provides to them
> at startup. And third, Connect's API should be changed to allow a connector
> to obtain the latest connector configuration at any time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
