ekas-7 commented on issue #871:
URL: https://github.com/apache/wayang/issues/871#issuecomment-6857474931

   Hi @zkaoudi,
   
   Thank you for the +1 on this. The implementation is now up in #877, and I 
would be grateful if you could take a look whenever you have time.
   
   A short summary of what the PR contains:
   
   - `wayang-platforms/wayang-clickhouse`, following the Trino module layout: 
TableSource, Filter, Projection, Join, GlobalReduce, ReduceBy, Sort and 
TableSink.
   - TableSink overwrite mode emits `CREATE TABLE ... ENGINE = MergeTree ORDER 
BY tuple() AS SELECT ...`, with the engine overridable per sink.
   - 41 unit tests that need no server, plus `ClickHouseOperatorsIT` (11 
tests), which I ran against ClickHouse 24.12 and which skips when no server is 
reachable.
   - Module README, `ClickHouseDemo` and `clickhouse.md` in 
`wayang-applications`, and the docs index entry. No docker-compose or setup 
folders, per #796.
   
   One thing I should flag: in this issue I proposed splitting the work into 
several small PRs, but #877 combines PRs 1–4. The pieces turned out to be hard 
to test in isolation (the integration test needs the sink, and the sink needs 
the other operators), so I kept them together. I realise that makes it a larger 
review. If you would prefer it split into smaller PRs, I am very happy to do 
that; just let me know.
   
   The questions from the issue description are still open, and I will follow 
whatever you decide:
   
   1. Is the Trino/Presto/BigQuery module structure the right reference?
   2. Is a configurable sink engine acceptable, or should the sink only `INSERT 
INTO` pre-created tables?
   3. Is an integration test that skips without a server fine, or would you 
prefer Testcontainers?
   
   Cost calibration (`ClickHouseCostPilotIT`) is not included and can follow 
separately, as noted under "out of scope".
   
   It also looks like the CI workflows on the PR are waiting for approval to 
run. Could you approve them when convenient?
   
   Thank you for your time, and for any feedback.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to