[
https://issues.apache.org/jira/browse/CAMEL-25710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25710 started by shashank.
----------------------------------------
> camel-mail - with an idempotentRepository, mails of a poll that are not
> routed (consumer stopped mid-batch, or the poll failed on a later mail) keep
> their key and are skipped as duplicates by every later poll
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25710
> URL: https://issues.apache.org/jira/browse/CAMEL-25710
> Project: Camel
> Issue Type: Bug
> Components: camel-mail
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> With an {{idempotentRepository}} (the documented way to let several consumers
> share one mailbox), {{MailConsumer.retrieveMessages}} adds the key of every
> mail it retrieves to the repository ({{isValidMessage}}, main 71633593ebdc
> {{MailConsumer.java}} 369-381, {{retrieveMessages}} 327), before any of them
> is routed. The key is removed (default
> {{idempotentRepositoryRemoveOnCommit=true}}) or confirmed when the exchange
> of the mail completes, and removed on rollback. Mails that the poll retrieved
> but did not route keep their key, and every later poll skips them as
> duplicates, also on the other consumers sharing the repository. They stay
> unread on the server and are never routed, without any log line; with a
> persistent repository this survives restarts.
> Two paths leave keys behind:
> * *Consumer stopped in the middle of a batch.* {{processBatch}} (224-270)
> stops at {{isBatchAllowed()}} when the route or the CamelContext is stopped
> gracefully (default {{ShutdownRunningTask.CompleteCurrentTaskOnly}}): the
> rest of the batch (up to {{maxMessagesPerPoll}} / {{fetchSize}}, unlimited by
> default) is neither routed nor given back. The exchanges are not released
> either.
> * *Poll failed while retrieving.* When creating the exchange of a mail throws
> (for example its attachments cannot be read, default {{mapMailMessage=true}},
> {{skipFailedMessage=false}}), the poll fails as documented and no mail of it
> is routed; but the keys of the mails retrieved before it, and its own key,
> stay in the repository. After the bad mail is removed, those mails are still
> skipped.
> h3. Reproduction
> New {{MailIdempotentRepositoryNotRoutedTest}} (camel-mail, GreenMail IMAP,
> {{MemoryIdempotentRepository}}):
> * 5 mails; the route is stopped while it routes the first one. On main 4 keys
> stay in the repository ({{expected: <0> but was: <4>}}), so a later poll
> skips these 4 mails. With the fix they are routed after the route is started
> again.
> * 3 mails and a multipart mail without its boundary; the poll fails on the
> last one. On main the 4 keys stay ({{expected: <0> but was: <4>}}). With the
> fix the 3 good mails are routed once the bad mail is deleted.
> h3. Proposed fix
> {{processBatch}} removes the keys of the exchanges it did not route and
> releases them, as the aws2-s3 consumer does with its in-progress keys since
> CAMEL-25521; {{retrieveMessages}} does the same (and removes the key of the
> failing mail) when creating an exchange fails, then rethrows. No option or
> default changes.
> Behaviour change (upgrade guide note added): with an {{idempotentRepository}}
> and the defaults {{skipFailedMessage=false}} / {{handleFailedMessage=false}},
> a mail whose exchange cannot be created now fails every poll until it is
> removed, as it already does without an {{idempotentRepository}} and as the
> option documentation describes ("the consumer throws an exception and no
> mails from the batch would be able to be routed"). Before, it failed only the
> first poll and was then skipped, together with the mails retrieved before it.
> {{skipFailedMessage}} or {{handleFailedMessage}} skip such a mail.
> Not changed: a mail skipped by {{skipFailedMessage}} /
> {{handleFailedMessage}} (or flagged DELETED) keeps its key, as before.
> How it was found: a sweep of the file-like consumers that commit or give back
> messages on the wrong outcome, after CAMEL-25521 / CAMEL-25551 fixed the
> in-progress keys of the object store consumers; the mail consumer's
> idempotent repository plays the same role and has no give-back path. The test
> confirmed it.
> Affected: main (71633593ebdc; tested). camel-4.23.x, camel-4.22.x,
> camel-4.18.x and camel-4.14.x have the same {{processBatch}} loop and
> {{isValidMessage}} add (read, not run).
> Duplicate check (2026-10-16): JIRA text "mail idempotentRepository",
> "MailConsumer", component camel-mail since 2025-06 (CAMEL-23046, CAMEL-24432,
> CAMEL-24434, CAMEL-24403, CAMEL-24567: none about keys of mails that were not
> routed); GitHub open pull requests "camel-mail" (none touching MailConsumer).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
