DanielLeens opened a new pull request, #12136:
URL: https://github.com/apache/seatunnel/pull/12136
### Purpose of this pull request
Fix a flaky `doris-connector-it` failure that is unrelated to the PRs it
blocks.
`org.apache.seatunnel.e2e.connector.doris.DorisIT#testCustomSql`
intermittently fails on the tail engine variants —
`testCustomSql{TestContainer}[6]` / `[7]` — with two distinct symptoms,
observed across many PRs that never touch Doris (e.g. #12096, #10874, #12042,
#12024, #12023):
```
testCustomSql{TestContainer}[6] <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <101> but was: <1>
at ...DorisIT.testCustomSql(DorisIT.java:174)
testCustomSql{TestContainer}[7] <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <101> but was: <201>
at ...DorisIT.testCustomSql(DorisIT.java:174)
```
### Root cause
`testCustomSql` is a `@TestTemplate` that reruns against a single long-lived
`apache/doris:doris-all-in-one` container for every engine variant, and the old
body was:
```java
initializeJdbcTable();
execResult =
container.executeJob("/doris_source_and_sink_with_custom_sql.conf");
Assertions.assertEquals(0, execResult.getExitCode());
Assertions.assertEquals(101, tableCount(sinkDB, UNIQUE_TABLE)); // line 174
clearUniqueTable(); // only
reached on success
```
The expected `101` is `100` FakeSource rows plus the single `custom_sql`
`INSERT` into the unique-key sink table. Two independent races produce the two
symptoms:
- **`was: <1>`** — Doris publishes stream-load data asynchronously. The
count is read the instant `executeJob()` returns, so it can observe only the
`custom_sql` seed row before the 100 loaded rows become visible.
- **`was: <201>`** — `clearUniqueTable()` runs *after* the assertion, not in
a `finally`. When an earlier variant fails (for instance on the visibility lag
above), its cleanup is skipped and leaves 101 rows behind;
`initializeJdbcTable()` only does `CREATE TABLE IF NOT EXISTS`, so the next
variant's 100 new rows accumulate on top → ~201. This is exactly why the
failures cluster on the *last* variants and surface on unrelated PRs — one
flake early in the parametrized run poisons the rest.
### Does this PR introduce _any_ user-facing change?
No. Test-only change; no production code is touched, and the asserted value
is unchanged.
- The count assertion is wrapped in a bounded Awaitility poll (60s, 2s
interval) that still requires **exactly 101**, so async publish lag no longer
yields a transient under-count. This is not a weakened assertion — an endpoint
that never reaches 101 still fails.
- `clearUniqueTable()` is moved into a `finally` block so the shared table
is always reset even when the assertion fails, so one variant can no longer
poison later variants.
Awaitility is already used by the sibling Doris IT classes in this module
(`DorisTimerFlushIT`, `DorisSchemaChangeIT`, `DorisMultiReadIT`,
`DorisCDCSinkIT`), so no dependency change is required.
### How was this patch tested?
Verified by CI on this PR head (`doris-connector-it`). The change keeps the
existing exit-code and exact-count assertions; it only removes the two
timing/isolation races described above.
### Check list
* [x] If any new Jar binary package adding in your PR, please add License
Notice according [New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [x] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
