atiaomar1978-hub commented on PR #25034: URL: https://github.com/apache/camel/pull/25034#issuecomment-5052244015
**Implementation summary** Added the new `camel-clickhouse` component (producer-only) using the official ClickHouse Java client (`com.clickhouse:client-v2`). Supported operations: - `insert` (default) — stream data in native formats (`JSONEachRow`, `RowBinary`, `CSV`, etc.) - `query` — run SQL and return results in the configured format - `ping` — health check **PR:** https://github.com/apache/camel/pull/25034 --- **Live validation (ClickHouse 24.8 + Camel routes)** Validated all use cases against a live ClickHouse server (Docker: `mirror.gcr.io/clickhouse/clickhouse-server:24.8`) using standalone Camel routes. No Apache Camel repo changes were made for this validation; a separate demo project was used (`C:\c\camel-clickhouse-demo`). **Environment** - ClickHouse: `http://localhost:8123` - Database/table: `camel_demo.events` - Schema: `(id UInt32, name String, source String DEFAULT 'test') ENGINE = MergeTree ORDER BY id` - Auth: `default` / empty password **Use cases tested — all 11 passed** | # | Use case | Route / config | DB validation | |---|----------|----------------|---------------| | 1 | ping | `operation=ping` | `CamelClickHousePingOk=true` | | 2 | insert String | `format=JSONEachRow`, String body | `id=1, source=string` | | 3 | insert byte[] | JSONEachRow bytes | `id=2, source=bytes` | | 4 | insert File | JSONEachRow from file | `id=3, source=carol/file` | | 5 | insert CSV | `format=CSV` | `id=4, source=csv` | | 6 | query count | SQL in body, `format=CSV` | count=4 at step 6 | | 7 | query select | `SELECT id, name WHERE id=1` | returns `1,"alice"` | | 8 | asyncInsert | `asyncInsert=true&waitForAsyncInsert=true` | `id=5, source=async` present in DB | | 9 | header overrides | `CamelClickHouseTable` + `CamelClickHouseFormat` headers | `id=6, source=headers` | | 10 | batchSize + List | `batchSize=2`, 5 registered POJOs → 3 inserts | 5 rows `id=10..14, source=batch` | | 11 | default operation | no `operation` param (defaults to insert) | `id=7, source=default-op` | **Final DB state (validated via `clickhouse-client`)** ``` 12 rows in camel_demo.events By source: string, bytes, file, csv, async, headers, default-op → 1 row each batch → 5 rows (batchSize=2 split 5 POJOs into 3 insert calls) ``` --- **Observations / notes for reviewers** 1. **`batchSize`** applies to **List** bodies only. Requires a shared `Client` with the POJO class registered via `client.register(Class, client.getTableSchema(table))`. Stream/String/byte[]/File inserts are unaffected. 2. **`asyncInsert=true`** — data is written correctly, but `CamelClickHouseWrittenRows` may be **0** until the server flushes async buffers. Validate with a query, not only the header. 3. **ClickHouse 24.8 Docker** — recent images require explicit auth config (`CLICKHOUSE_USER=default`, `CLICKHOUSE_PASSWORD=`, `CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1`) for programmatic clients. 4. **Review feedback addressed in PR** — `volatile` lazy client, `ssl=true` → `https://`, `skipITs.ppc64le`/`skipITs.s390x`, unrelated upgrade-guide section reverted, client-side batching implemented. --- **Tests** - Unit tests: 16 passed (AssertJ + Mockito) - Integration tests: 2 (Testcontainers, skipped without Docker in local run) - Live demo: 11/11 use cases passed with DB validation --- **Example routes** ```java // Insert JSONEachRow from("direct:insert") .to("clickhouse://camel_demo.events?operation=insert&format=JSONEachRow&serverUrl=http://localhost:8123";); // Query from("direct:query") .setBody(constant("SELECT count() FROM camel_demo.events")) .to("clickhouse://camel_demo?operation=query&format=CSV&serverUrl=http://localhost:8123";); // Ping from("direct:ping") .to("clickhouse://camel_demo?operation=ping&serverUrl=http://localhost:8123";); // Batch List insert (shared Client with registered POJO required) from("direct:insertBatch") .to("clickhouse://camel_demo.events?operation=insert&batchSize=2"); ``` --- _AI-generated comment prepared on behalf of the contributor._ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
