shashank created CAMEL-25511:
--------------------------------
Summary: camel-jooq - the consumer deletes an entity whose
exchange failed (consumeDelete=true, the default), so the row is lost
Key: CAMEL-25511
URL: https://issues.apache.org/jira/browse/CAMEL-25511
Project: Camel
Issue Type: Bug
Components: camel-jooq
Reporter: shashank
With {{consumeDelete=true}} (the default) {{JooqConsumer.poll()}} (lines 53-81
at main c578a42a776d) fetches all the entities of the table, hands them to
{{processBatch}} and then runs {{context.batchDelete(results).execute()}} on
*all* fetched entities, whatever happened to their exchanges:
{code:java}
int messagePolled = processBatch(CastUtils.cast(answer));
if (configuration.isConsumeDelete()) {
context.batchDelete(results).execute();
}
{code}
A failed route does not throw out of {{getProcessor().process(exchange)}}; the
error handler leaves the exception on the exchange. So a row whose processing
failed (or whose exchange was marked rollback only) is deleted and lost,
although the docs present the table as "a logical queue" and say the entity is
deleted "when it has been processed". When a graceful shutdown starts after the
entities were fetched, {{isBatchAllowed()}} is false and {{processBatch}}
processes none of them, but they are all deleted too.
The consumers of the other "database as a queue" components do not do this:
camel-jpa deletes an entity only when its exchange has no exception (otherwise
the transaction is rolled back), and camel-sql runs {{onConsume}} only for a
successful exchange ({{onConsumeFailed}} otherwise).
h3. Reproduction
New {{JooqConsumerDeleteFailedTest}} (HSQLDB as in the other tests; the route
uses {{startScheduler=false}} and the test calls {{poll()}} once, so it is
deterministic). Two authors are inserted; the route throws for one of them, or
marks its exchange rollback only, or the poll runs after
{{deferShutdown(CompleteCurrentTaskOnly)}} as the shutdown strategy calls it.
On main all three tests fail:
{noformat}
testFailedExchangeKeepsRow: only the successfully processed author must be
deleted ==> expected: <[2]> but was: <[]>
testRollbackOnlyExchangeKeepsRow: an author whose exchange was marked rollback
only must not be deleted ==> expected: <[2]> but was: <[]>
testNotProcessedEntitiesKeepRows: entities that were not processed must not be
deleted ==> expected: <2> but was: <0>
{noformat}
h3. Proposed fix
{{processBatch}} records, right after processing each exchange, whether it
completed successfully (not failed, not rollback only); {{poll}} deletes only
those entities. A failed entity stays in the table and is consumed again by the
next poll (an entity that always fails is processed at every poll, as with
camel-jpa). Docs (and the catalog copy) say so, and the 4.23 upgrade guide has
a note: a route that relied on failing entities being deleted now receives them
again and can use {{onException(...).handled(true)}}. camel-jooq tests with the
fix: 16, 0 failures.
Found with a Lean 4 model of the poll (fetched entities with the outcome of
their exchange: ok, failed, not processed): "an entity is deleted only if its
exchange succeeded" fails for one failed entity, main is proved to delete every
fetched entity and to agree with the property exactly when all exchanges
succeeded; the fix is proved to satisfy it and to delete the same entities as
main when all exchanges succeed. Confirmed with the real consumer.
Affected: main, camel-4.22.x, camel-4.18.x, camel-4.14.x (same code, GitHub
contents API; latest releases 4.22.1, 4.18.4, 4.14.9). The consumer has deleted
all fetched entities since it was added (CAMEL-13256, Camel 3.0).
Duplicate check (2026-10-09, repeated in review): JIRA text "jooq" (23, none
about deletion or failures; CAMEL-13261 added the queue use case),
"consumeDelete" (13, all camel-jpa or the jooq queue feature); GitHub pull
requests "jooq consumer delete", "jooq consumeDelete", "JooqConsumer", "jooq
failed": none. No open pull request changes {{JooqConsumer}}.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
