aaron-y-chen opened a new pull request, #71135: URL: https://github.com/apache/airflow/pull/71135
related: #52712 This is the first PR for Kinesis part of #52712, I will integrate the trigger with the common-messaging interface and add a system/e2e test. ## Summary Add an asynchronous Kinesis Data Streams trigger for consuming records without occupying worker slots. The trigger polls all shards, emits JSON-serializable record batches, handles iterator expiry, throttling, pagination, and in-run resharding, and checkpoints per-shard sequence numbers when an asset state store is available. New shards first discovered after a restart honor the configured initial position, so `LATEST` may skip records written during the downtime. This is the provider-side foundation for the common-messaging integration tracked in #52712. --- ##### Was generative AI tooling used to co-author this PR? <!-- If generative AI tooling has been used in the process of authoring this PR, please change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by". --> - [x] Yes (please specify the tool below) Generated-by: [GPT 5.6-sol] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
