allthingssecurity opened a new pull request, #27347: URL: https://github.com/apache/camel/pull/27347
# Description [CAMEL-25221](https://issues.apache.org/jira/browse/CAMEL-25221) With `consumerProcessedStrategy=delete`, `CouchbaseConsumer` removed every document of a poll in `pollWithSqlQuery` / `pollWithView`, while building the exchanges and before `processBatch` handed any of them to the route. So a document was lost: - when its exchange failed (the route never sees it again); - when it was never handed to the route at all: `processBatch` stops at `maxMessagesPerPoll` and when `isBatchAllowed()` turns false because the consumer is stopping, and the rows after that point had already been removed. @davsclaus raised this in the review of #27123 (CAMEL-25171), and @oscerd filed it as CAMEL-25221 with the follow-up options. This PR takes the first one, delete after the exchange has been processed successfully, which the ticket calls the real fix (the comparable camel-mongodb change was CAMEL-25024). This change: - the removal runs in an on-completion (`SynchronizationAdapter.onComplete`) added to the exchange when it is built, the delete-after-read pattern of other consumers (for example aws2-s3). It runs only for an exchange that went through the route's unit of work and completed: a failed exchange keeps its document for the next poll, and an exchange that was never handed to the route (released by the drain loop from #27123) never runs it. A handled exception (`onException().handled(true)`, dead letter channel) completes the exchange, so the document is removed, as for the file consumer; - a failure to remove the document is reported through the consumer's exception handler (the removal used to throw out of `poll()` and abort the rest of the batch); - the comments that pointed at CAMEL-25221 in `processBatch` / `processExchange` (added by #27123) are updated; - `filter` and `none` are unchanged (neither removes anything); - upgrade guide: the last sentence of the existing 4.23 `=== camel-couchbase` entry (from #27123) said the document "is removed during the poll, before the route runs"; it is replaced by a `==== consumerProcessedStrategy=delete` subsection that describes the new timing, including that a document whose route always fails is now consumed again on every poll. Tests: new `CouchbaseConsumerProcessedStrategyTest` runs the consumer in a real route (so the exchange gets a unit of work) with the scheduler not started, calls `poll()` against a mocked cluster, and verifies `Collection.remove`: - the document still exists while the route runs, and is removed after it (SQL++); - a failed exchange keeps its document, the next row of the same poll is removed (SQL++ and view); - `maxMessagesPerPoll=1` with three rows: only the first is removed; - the consumer told to complete only its current task (`deferShutdown(CompleteCurrentTaskOnly)`, as the shutdown strategy does) during the first exchange: only the first is removed; - a failing removal is reported to the exception handler; - `consumerProcessedStrategy=none` removes nothing (control). Without the change 6 of the 7 fail (twice, `NeverWantedButInvoked` for the failed/undelivered rows, the document already removed while routing, and the removal exception thrown out of `poll()`); the `none` control passes. With the change the camel-couchbase module passes: 49 unit tests, 0 failures (the 8 Docker IT tests are skipped locally). # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built and tested the affected module, including the formatter and import-sort plugins. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
