[ https://issues.apache.org/jira/browse/AIRFLOW-81?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Imberman closed AIRFLOW-81. ---------------------------------- Resolution: Auto Closed > Scheduler blackout time period > ------------------------------ > > Key: AIRFLOW-81 > URL: https://issues.apache.org/jira/browse/AIRFLOW-81 > Project: Apache Airflow > Issue Type: New Feature > Components: scheduler > Reporter: Sean McIntyre > Priority: Minor > Labels: features > > I have the need for a scheduler blackout time period in Airflow. > My team, which uses Airflow, has been asked to not query one of my company's > data sources between midnight and 7 AM. When we launch big backfills on this > data source, it would be nice to have the Scheduler not schedule some > TaskInstances during the blackout hours. > We (@r39132 and @ledsusop) brainstormed a few ideas on gitter on how to do > this... > (1) Put more state/logic in the TaskInstance and Scheduler like this: > my_task = PythonOperator( > task_id='my_task', > python_callable=my_command_that_access_the_datasource, > provide_context=True, > dag=dag, > blackout=my_blackout_logic_for_the_datasource # <--- > ) > where my_blackout_logic is some function I provide that the scheduler calls > to determine whether or not it is the blackout period. > (2) Pause DAGs on nightly basis. This can be done with the `pause_dag` CLI > command scheduled by cron / Jenkins. However could this be considered a core > feature to bring into the Airflow UI and scheduling system? -- This message was sent by Atlassian Jira (v8.3.4#803005)