Programmer-RD-AI opened a new pull request, #53356: URL: https://github.com/apache/airflow/pull/53356
Add Azure Service Bus Queue and Subscription triggers for async message processing This PR implements Azure Service Bus triggers to enable asynchronous message processing in Airflow, allowing DAGs to be triggered when messages arrive in Service Bus queues or topic subscriptions. ## Changes ### New Features - **AzureServiceBusQueueTrigger**: Monitors Azure Service Bus queues for incoming messages - **AzureServiceBusSubscriptionTrigger**: Monitors Azure Service Bus topic subscriptions for incoming messages - **BaseAzureServiceBusTrigger**: Common base class providing shared functionality for message processing ### Key Components - Async message processing with configurable polling intervals - Support for multiple queues/topics per trigger - Configurable message batch sizes and wait times - Integration with existing Azure Service Bus connections - Comprehensive error handling and timeout management ### Documentation - Complete RST documentation with examples for both triggers - Connection configuration guidance - Usage examples and URI format specifications ### Testing - Full test coverage including unit tests, integration scenarios, and edge cases - Mock-based testing for Azure Service Bus interactions - Async test patterns for trigger functionality ## Technical Details - Uses `asgiref.sync.sync_to_async` for async/sync bridge - Implements proper asyncio patterns for concurrent message processing - Leverages existing `MessageHook` for Azure Service Bus connectivity - Follows Airflow trigger patterns with proper serialization support ## Related related: AIP-82 --- This implementation enables event-driven DAG execution based on Azure Service Bus messages, supporting modern microservices architectures and real-time data processing workflows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
