[
https://issues.apache.org/jira/browse/CAMEL-25511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25511 started by shashank.
----------------------------------------
> camel-jooq - the consumer deletes an entity whose exchange failed
> (consumeDelete=true, the default), so the row is lost
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25511
> URL: https://issues.apache.org/jira/browse/CAMEL-25511
> Project: Camel
> Issue Type: Bug
> Components: camel-jooq
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> With {{consumeDelete=true}} (the default) {{JooqConsumer.poll()}} (lines
> 53-81 at main c578a42a776d) fetches all the entities of the table, hands them
> to {{processBatch}} and then runs {{context.batchDelete(results).execute()}}
> on *all* fetched entities, whatever happened to their exchanges:
> {code:java}
> int messagePolled = processBatch(CastUtils.cast(answer));
> if (configuration.isConsumeDelete()) {
> context.batchDelete(results).execute();
> }
> {code}
> A failed route does not throw out of {{getProcessor().process(exchange)}};
> the error handler leaves the exception on the exchange. So a row whose
> processing failed (or whose exchange was marked rollback only) is deleted and
> lost, although the docs present the table as "a logical queue" and say the
> entity is deleted "when it has been processed". When a graceful shutdown
> starts after the entities were fetched, {{isBatchAllowed()}} is false and
> {{processBatch}} processes none of them, but they are all deleted too.
> The consumers of the other "database as a queue" components do not do this:
> camel-jpa deletes an entity only when its exchange has no exception
> (otherwise the transaction is rolled back), and camel-sql runs {{onConsume}}
> only for a successful exchange ({{onConsumeFailed}} otherwise).
> h3. Reproduction
> New {{JooqConsumerDeleteFailedTest}} (HSQLDB as in the other tests; the route
> uses {{startScheduler=false}} and the test calls {{poll()}} once, so it is
> deterministic). Two authors are inserted; the route throws for one of them,
> or marks its exchange rollback only, or the poll runs after
> {{deferShutdown(CompleteCurrentTaskOnly)}} as the shutdown strategy calls it.
> On main all three tests fail:
> {noformat}
> testFailedExchangeKeepsRow: only the successfully processed author must be
> deleted ==> expected: <[2]> but was: <[]>
> testRollbackOnlyExchangeKeepsRow: an author whose exchange was marked
> rollback only must not be deleted ==> expected: <[2]> but was: <[]>
> testNotProcessedEntitiesKeepRows: entities that were not processed must not
> be deleted ==> expected: <2> but was: <0>
> {noformat}
> h3. Proposed fix
> {{processBatch}} records, right after processing each exchange, whether it
> completed successfully (not failed, not rollback only); {{poll}} deletes only
> those entities. A failed entity stays in the table and is consumed again by
> the next poll (an entity that always fails is processed at every poll, as
> with camel-jpa). Docs (and the catalog copy) say so, and the 4.23 upgrade
> guide has a note: a route that relied on failing entities being deleted now
> receives them again and can use {{onException(...).handled(true)}}.
> camel-jooq tests with the fix: 16, 0 failures.
> Found with a Lean 4 model of the poll (fetched entities with the outcome of
> their exchange: ok, failed, not processed): "an entity is deleted only if its
> exchange succeeded" fails for one failed entity, main is proved to delete
> every fetched entity and to agree with the property exactly when all
> exchanges succeeded; the fix is proved to satisfy it and to delete the same
> entities as main when all exchanges succeed. Confirmed with the real consumer.
> Affected: main, camel-4.22.x, camel-4.18.x, camel-4.14.x (same code, GitHub
> contents API; latest releases 4.22.1, 4.18.4, 4.14.9). The consumer has
> deleted all fetched entities since it was added (CAMEL-13256, Camel 3.0).
> Duplicate check (2026-10-09, repeated in review): JIRA text "jooq" (23, none
> about deletion or failures; CAMEL-13261 added the queue use case),
> "consumeDelete" (13, all camel-jpa or the jooq queue feature); GitHub pull
> requests "jooq consumer delete", "jooq consumeDelete", "JooqConsumer", "jooq
> failed": none. No open pull request changes {{JooqConsumer}}.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
