[
https://issues.apache.org/jira/browse/CAMEL-25551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-25551.
---------------------------------
Resolution: Fixed
Fixed by https://github.com/apache/camel/pull/27682
> camel-azure-storage-blob, camel-huaweicloud-obs, camel-alibaba-oss,
> camel-aws2-s3-vectors - do not consume an object again while its exchange is
> in flight
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25551
> URL: https://issues.apache.org/jira/browse/CAMEL-25551
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2, camel-azure
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.23.0
>
>
> These polling consumers list objects and route them asynchronously, and
> delete (or move) the object in the on completion of the exchange. They do not
> track which objects are in flight, so when routing takes longer than the poll
> delay (slow route, threads(), etc.) the next poll lists the same object again
> and routes it a second time. The second exchange then fails to delete/move it
> (404).
> * camel-azure-storage-blob BlobConsumer with deleteAfterRead=true or
> moveAfterRead=true. With fileDir it is worse: the blob is deleted and
> downloaded again to the same local file while the first exchange is still
> reading it.
> * camel-huaweicloud-obs OBSConsumer with deleteObject / moveObject.
> * camel-alibaba-oss OSSConsumer with deleteAfterRead=true.
> * camel-aws2-s3-vectors consumer with deleteAfterRead=true (the
> processedVectorIds dedup only covers the branch without delete).
> The fix is the same as camel-minio (CAMEL-25512) and the in-progress
> repository of camel-aws2-s3: remember the objects in flight, skip them on the
> next poll, and release them when the exchange completes or fails, when the
> exchange could not be created, and for exchanges that are not routed because
> the consumer is stopping (closing their streams).
> camel-azure-storage-blob also fails the whole poll when one listed blob is
> deleted before it is read (404), and drops the exchanges already created
> without closing their streams.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
