Skipped sensors and cascading skips

2020-07-28 Thread Aaron Niskode-Dossett
Hi friends, the Airflow documentation states: Skipped tasks will cascade through trigger rules all_success and all_failed but not all_done, one_failed, one_success, none_failed, none_failed_or_skipped, none_skipped and dummy.

Re: [PROPOSAL][AIP-36 DAG Versioning]

2020-07-28 Thread Dan Davydov
Strongly agree with Max's points, also I feel the right way to go about this is instead of Airflow schedulers/webservers/workers reading DAG Python files, they would instead read from serialized representations of the DAGs (e.g. json representation in the Airflow DB). Instead of DAG owners pushing

Re: [PROPOSAL][AIP-36 DAG Versioning]

2020-07-28 Thread Maxime Beauchemin
> "mixed version" Personally I vote for a DAG version to be pinned and consistent for the duration of the DAG run. Some of the reasons why: - it's easier to reason about, and therefore visualize and troubleshoot - it prevents some cases where dependencies are never met - it prevents the explosion

Re: [PROPOSAL][AIP-36 DAG Versioning]

2020-07-28 Thread Kaxil Naik
Thanks Max for your comments. *DAG Fingerprinting: *this can be tricky, especially in regards to dynamic > DAGs, where in some cases each parsing of the DAG can result in a different > fingerprint. I think DAG and tasks attributes are left out from the > proposal that should be considered as part