kaxil commented on PR #73952: URL: https://github.com/apache/airflow/pull/73952#issuecomment-6018376625
I talked this through with @jedcunningham too apart from Ash's feedback above. We think this should be considered together with the draining work. `use_job_schedule = False` has blocked asset-triggered runs since assets (datasets back then) landed in 2.4, because the flag gates all of _create_dagruns_for_dags. The "cron intervals" description is older than assets and was never updated. best-practices.rst recommends this flag for maintenance windows, and users have counted on it stopping every run the scheduler creates by itself. With this change, an asset event that lands during maintenance would start a run. It also matches the draining state from #72407: no scheduled or asset-triggered runs, while manual triggers, backfills and materializations still go through. use_job_schedule = False is the instance-wide version of that rule. The use case in #62929 (timetables off, assets on in a dev environment) is still valid imo. It may need finer, runtime-settable control over which triggers the scheduler acts on, rather than a new meaning for this flag. @dheerajturaga, this overlaps with your drain work and your team's use cases. What do you think? @luc-pimentel, your view too, please. If we keep the current behaviour, the fix would be updating the `config.yml` description to say asset-triggered runs stop as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
