fanyang commented on PR #14241: URL: https://github.com/apache/camel/pull/14241#issuecomment-2144908086
Hi @davsclaus @oscerd Thank you for the comments. The purpose of this design is to ensure that the batch sent by the upstream sender is consistent with the batch sent by the camel kinesis producer. For example: Upstream sender sent batch 0: record 0 to 4 camel kinesis producer sent batch 0: record 0 to 4 Upstream sender marked record 0 to 4 were sent Upstream sender sent batch 1: record 5 to 9 camel kinesis producer sent batch 1: record 5 to 9 Upstream sender marked record 5 to 9 were sent Once camel kinesis producer failed to send batch 1, then upstream sender will resend record 5 to 9. If the batch boundary is inconsistent between camel kinesis producer and upstream send, the above commit mechanism will be invalid, and data loss may occur. So, we cannot let the camel kinesis producer control the boundaries of the batch to avoid inconsistent batches. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
