JNSimba opened a new pull request, #100:
URL: https://github.com/apache/doris-kafka-connector/pull/100
## What this PR does
Adds opt-in one-way TLS support for Doris connections used by the Kafka
connector.
- Applies TLS consistently to Doris HTTP APIs, backend probes, Stream Load,
two-phase commit, schema operations, and MySQL/JDBC connections.
- Supports a worker-local PEM CA chain without changing JVM-global trust
settings.
- Verifies hostnames by default and prevents HTTPS-to-HTTP redirect
downgrades.
- Keeps Copy Into external-storage traffic on an independent system-trust
client.
- Adds unit tests plus HTTP and JDBC TLS integration test coverage.
- Allows existing sink E2E tests to use an externally supplied Doris
environment while retaining the local Kafka container.
## Configuration
| Property | Default | Meaning |
| --- | --- | --- |
| `doris.enable.tls` | `false` | Enables one-way TLS for Doris HTTP and
MySQL/JDBC connections. |
| `doris.tls.ca-certificate-path` | empty | Worker-local path to a PEM CA
certificate or certificate chain used to verify Doris. An empty value uses the
default Java trust store. |
| `doris.tls.skip-hostname-verification` | `false` | Skips hostname
verification while still validating the certificate chain against the
configured CA. |
| `doris.tls.excluded-protocols` | empty | Comma-separated Doris protocols
that remain plaintext. Supported values are `http` and `mysql`. |
TLS remains fully backward compatible because it is disabled by default.
When enabled for a protocol, the connector does not fall back to plaintext
after a TLS failure.
## Example
Save the following as `doris-tls-sink.json`. `doris.urls` contains hostnames
only; the CA file must be available at the same local path on every Kafka
Connect worker that can run the task.
```json
{
"name": "doris-tls-sink",
"config": {
"connector.class": "org.apache.doris.kafka.connector.DorisSinkConnector",
"topics": "orders",
"tasks.max": "1",
"doris.topic2table.map": "orders:orders",
"doris.urls": "doris-fe.example.com",
"doris.http.port": "8040",
"doris.query.port": "9030",
"doris.database": "sales",
"doris.user": "connector_user",
"doris.password": "replace-with-secret",
"doris.enable.tls": "true",
"doris.tls.ca-certificate-path": "/etc/kafka-connect/certs/doris-ca.pem",
"doris.tls.skip-hostname-verification": "false",
"doris.tls.excluded-protocols": "",
"load.model": "stream_load",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter": "org.apache.kafka.connect.storage.StringConverter"
}
}
```
Register it through the Kafka Connect REST API:
```bash
curl -X POST -H 'Content-Type: application/json' --data @doris-tls-sink.json
\
http://connect-worker:8083/connectors
```
For staged migrations, set `doris.tls.excluded-protocols=http` or `mysql` to
leave only that Doris protocol on plaintext. Client certificates and private
keys are not supported.
## Verification
- `mvn test`
- 124 tests run, 0 failures, 0 errors, 2 skipped
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
