[jira] [Assigned] (AIRFLOW-3379) Support for AWS triggers (S3, SNS, SQS, CloudWatch, etc.)
[ https://issues.apache.org/jira/browse/AIRFLOW-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anonymous reassigned AIRFLOW-3379: -- Assignee: Brylie Christopher Oxley (was: Rohan Bhanderi) > Support for AWS triggers (S3, SNS, SQS, CloudWatch, etc.) > - > > Key: AIRFLOW-3379 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3379 > Project: Apache Airflow > Issue Type: Improvement > Components: aws >Reporter: Brylie Christopher Oxley >Assignee: Brylie Christopher Oxley >Priority: Major > > We would like to build a 'reactive', or event-driven, data pipeline. From > what I can gather, the primary Airflow DAG trigger is a timer (cron). > However, it would be useful to trigger dags on external events, namely AWS > events like S3 file, SQS, SNS, and/or CloudWatch. > I note there is an experimental API, which could be triggered from an AWS > Lambda, but would add boilerplate and brittleness to the data pipeline. > What are our options for triggering Airflow DAGS from external AWS events? -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (AIRFLOW-3379) Support for AWS triggers (S3, SNS, SQS, CloudWatch, etc.)
[ https://issues.apache.org/jira/browse/AIRFLOW-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anonymous reassigned AIRFLOW-3379: -- Assignee: Rohan Bhanderi > Support for AWS triggers (S3, SNS, SQS, CloudWatch, etc.) > - > > Key: AIRFLOW-3379 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3379 > Project: Apache Airflow > Issue Type: Improvement > Components: aws >Reporter: Brylie Christopher Oxley >Assignee: Rohan Bhanderi >Priority: Major > > We would like to build a 'reactive', or event-driven, data pipeline. From > what I can gather, the primary Airflow DAG trigger is a timer (cron). > However, it would be useful to trigger dags on external events, namely AWS > events like S3 file, SQS, SNS, and/or CloudWatch. > I note there is an experimental API, which could be triggered from an AWS > Lambda, but would add boilerplate and brittleness to the data pipeline. > What are our options for triggering Airflow DAGS from external AWS events? -- This message was sent by Atlassian Jira (v8.20.1#820001)
