Re: [D] Constrain Task Callback to Specific Task Operator Types [airflow]
GitHub user GabrielEisenbergOlympus added a comment to the discussion: Constrain Task Callback to Specific Task Operator Types Sorry for the late reply @potiuk! In my case it turns out that I had to carefully apply the new callback to the various DAGs and I also included logic to be safe based on the name of the operator that was used in each task. Less ideal than I would have liked. When I have time, I'd really like to try your approach. GitHub link: https://github.com/apache/airflow/discussions/45476#discussioncomment-11928057 This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
Re: [D] Constrain Task Callback to Specific Task Operator Types [airflow]
GitHub user potiuk added a comment to the discussion: Constrain Task Callback to Specific Task Operator Types No. Not as a direct feature. But you could possibl do something with cluster policies https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/cluster-policies.html - you could likely analyse your task in task_policy, see if your callback defined is as expected (according to whatever criteria you choose) and either remove the callback or - possibly better - throw an exception which will fail the DAG to being created. GitHub link: https://github.com/apache/airflow/discussions/45476#discussioncomment-11775133 This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
Re: [D] Constrain Task Callback to Specific Task Operator Types [airflow]
GitHub user GabrielEisenbergOlympus edited a discussion: Constrain Task Callback to Specific Task Operator Types Hi all 😄 I'm working with Airflow 2.10.1 on MWAA and need to send results from specific tasks to an external service. Based on certain constraints, this will need to be done in a callback. For ease of other users, I want to generalise this into standard callbacks that we use. My question is whether there is a way to constrain these task callbacks to certain types of operators without including a logic check inside the callback itself? In this way Airflow would do the filtering and would only invoke the callback in certain cases and not for every single task. Thanks so much! GitHub link: https://github.com/apache/airflow/discussions/45476 This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
