srijan-singh opened a new pull request, #11198: URL: https://github.com/apache/seatunnel/pull/11198
### Purpose of this pull request https://github.com/apache/seatunnel/issues/10443#issuecomment-4796095891 This pull request adds a new **Couchbase sink connector** for Apache SeaTunnel (V2 API). The connector writes `SeaTunnelRow` data as JSON documents into a Couchbase bucket/scope/collection using the Couchbase Java SDK 3.5.1. It supports: - **Upsert mode** — insert or replace existing documents via `upsert-enable` - **Composite document keys** — built from configurable `primary-key` field list (joined with `_`); falls back to a random UUID when not set - **Batch flushing** — buffers rows in memory and flushes on size threshold (`buffer-flush.max-rows`) or time threshold (`buffer-flush.interval`) - **Retry with backoff** — transient write failures are retried up to `retry.max` times with `retry.interval` ms delay (multiplied by attempt number) - **Full type mapping** — all SeaTunnel primitive types mapped to Couchbase JSON equivalents; `BYTES` stored as Base64 string --- ### Does this PR introduce _any_ user-facing change? **Yes.** This is a new connector. Users can now configure a `Couchbase` sink in their SeaTunnel jobs: ```hocon sink { Couchbase { connection.string = "couchbase://localhost" username = "Administrator" password = "password" bucket = "my_bucket" collection = "my_collection" primary-key = ["id"] upsert-enable = true } } ``` There is no change to any existing connector or API — this is purely additive. --- ### How was this patch tested? **1. Live Docker integration test** A standalone integration test was run against a real `couchbase/server:community-7.1.1` Docker container, validating all core write behaviors: | Test | Scenario | Result | |------|----------|--------| | [1] | Plain document insert with UUID key | ✅ PASS | | [2] | Upsert — writing same key twice replaces the document | ✅ PASS | | [3] | Composite primary key: `user_id + order_id` joined as `u1_o42` | ✅ PASS | | [4] | N1QL `COUNT(*)` query returns expected document count after writes | ✅ PASS | **2. E2E test module** Added `CouchbaseIT` under `seatunnel-e2e/seatunnel-connector-v2-e2e/connector-couchbase-e2e/`. The test uses Testcontainers to start a real Couchbase container, bootstraps the cluster fully via REST (node config → services → credentials → bucket → collection → index storage mode), executes a `FakeSource → Couchbase` sink job, then verifies document count via N1QL. --- ### Check list * [x] No new Jar binary packages added without license notice — Couchbase Java SDK (`com.couchbase.client:java-client:3.5.1`) is Apache 2.0 licensed * [x] Documentation updated: `docs/en/connectors/sink/Couchbase.md` added with full options table, data type mapping, and task examples * [ ] `incompatible-changes.md` — not applicable (new connector, no breaking changes) * [x] [`plugin-mapping.properties`](plugin-mapping.properties) updated: `seatunnel.sink.Couchbase = connector-couchbase` * [x] [`seatunnel-dist/pom.xml`](seatunnel-dist/pom.xml) updated: `connector-couchbase` dependency added * [x] CI label added in [`.github/workflows/labeler/label-scope-conf.yml`](.github/workflows/labeler/label-scope-conf.yml): `couchbase:` block * [x] E2E test added: [`seatunnel-e2e/seatunnel-connector-v2-e2e/connector-couchbase-e2e/`](seatunnel-e2e/seatunnel-connector-v2-e2e/connector-couchbase-e2e/) * [x] [`config/plugin_config`](config/plugin_config) updated: `connector-couchbase` listed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
