corgy-w opened a new pull request, #12324:
URL: https://github.com/apache/seatunnel/pull/12324
### What
The ClickHouse streaming source reader (`StreamValueReader`, used when the
table has no usable sorting key or the query is complex) now reports a failed
streaming query to the job instead of silently ending the split.
### Why
When the streaming query fails, the failure is thrown **inside the
asynchronous reader thread** and nobody observes it. The thread's `finally`
sets `eos`, which is exactly what the consumer interprets as "the split ended
normally":
- `ClickhouseValueReader.java:477-486` — `catch (ClickHouseException e) {
throw new ClickhouseConnectorException(...) }` runs on `asyncReadThread`; the
only thing that ever sees it is the thread's uncaught-exception handler.
- `ClickhouseValueReader.java:491-520` — `hasNext()` drains `rowQueue`, sees
`eos == true`, and returns `false`.
Result: a job whose streaming query failed mid-split **succeeds with the
remaining rows missing** — no error, no log, no retry, and a checkpoint that
says the split is done.
### Root cause
`eos` carries two different meanings ("the producer stopped" and "the
producer stopped successfully") and the failure has no channel to the consumer.
Same defect class as the recently merged InfluxDB empty-result fix (#11966): a
reader-side failure the job never sees.
### How verified
- New test
`ClickhouseValueReaderTest#testStreamReaderSurfacesAsyncQueryFailure`: the
mocked query throws `ClickHouseException`; `hasNext()` must throw
`ClickhouseConnectorException` carrying that cause.
- With this change: `Tests run: 11, Failures: 0, Errors: 0, Skipped: 0`
(`mvn -o -pl seatunnel-connectors-v2/connector-clickhouse test
-Dtest=ClickhouseValueReaderTest`).
- Reverting only `ClickhouseValueReader.java` and re-running the new test:
`AssertionFailedError: Expected
org.apache.seatunnel.connectors.seatunnel.clickhouse.exception.ClickhouseConnectorException
to be thrown, but nothing was thrown.`
### Compatibility
No option name, default value, or public API change. Behaviour change is
intentional: a job that previously "succeeded" with silently truncated data now
fails. Rows already delivered before the failure are unaffected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
