Yup, I like it. Thanks for driving this one.
- ferruzzi ________________________________ From: Wei Lee <weilee...@gmail.com> Sent: Thursday, September 4, 2025 3:43 AM To: dev@airflow.apache.org Subject: RE: [EXT] [Discussion] Should We avoid using AirflowException direclty? CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque. I believe we've reached a certain level of consensus, but we still have some things to discuss. I'll break it into two parts. — ## What we have concluded on. 1. In most cases, we should prioritize using Python’s standard exceptions (e.g., `ValueError`, `TypeError`, `OSError`) instead of wrapping everything in `AirflowException`. 2. Within `airflow-core`, we should define and utilize more specific exception classes under `airflow.exceptions`. 3. For provider-specific implementations, exceptions should be defined within `airflow.providers.<provider>.exceptions`. The use of points 2 and 3 should only be considered when point 1 is inappropriate, which should be a rare occurrence. What we can do other than nail down these rules Add it to https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst Add a prek hook to detect `raise AirflowException` If no one objects, I'm planning on sending a [Lazy Consensus] sometime tomorrow. — ## What we haven't yet had a consensus Whether we really can/want to remove or depreciate AirflowException If we want, how are we going to achieve it Best, Wei > On Sep 2, 2025, at 3:36 PM, Amogh Desai <amoghde...@apache.org> wrote: > > Generally agree with this too. > > `AirflowException` is literally being overused almost to an extent that it > has > become like a dumping ground (along the lines of what `utils` has become) > > I would be in favour of removing it too and addressing backcompat concerns > for user facing / custom provider / hooks etc facing classes. > > This could be run as a community exercise too once we reach that stage to > clean > up providers from using / overusing it. > > Thanks & Regards, > Amogh Desai > > > On Tue, Sep 2, 2025 at 1:58 AM Jens Scheffler <j_scheff...@gmx.de.invalid> > wrote: > >> Also was a bit focussing on other stuff, late to the party. >> >> For deprecation we should consider RUFF rules as well. >> >> But on side of code changes I fear a lot of people wil have >> implementations based on AirflowException. I did not get the idea of >> backwards compatability. If we change Exception types then Operators >> implemented on top of Standard-Provider might fail if the receive >> different exceptions from inherited classes? >> >> Jens >> >> On 01.09.25 17:05, Buğra Öztürk wrote: >>> Great discussion! Joining delayed. We have this usage in airflowctl too >>> where AirflowException is the main one even though it isn't imported from >>> core. >>> I agree with the consensus. Great idea to make it backward compatible. >>> Maybe would be good to define a migration calendar roughly. >>> While we should prevent usage with some pre-commit hooks, we should also >>> document the practices we should follow on exception handling for these >>> changes. >>> Even though we can manage to migrate seamlessly for the user, I think we >>> still should warn them about the exception handling usage and give some >>> kind of deprecation warning if possible >>> >>> Bugra Ozturk >>> >>> On Mon, 1 Sept 2025, 16:04 Wei Lee, <weilee...@gmail.com> wrote: >>> >>>> Thanks everyone for joining the discussion! >>>> >>>> I think we have some consensus. These should at least apply to new code. >>>> >>>> 1. Don’t use AirflowException >>>> 2. Use Python native whenever possible >>>> 3. Create customized exceptions instead of AirflowException >>>> >>>> But we’re worried about how and whether we should remove >> AirflowException. >>>> What I am thinking is removing `raise AirflowException`. >>>> As for AirflowException itself, it looks somewhat challenging. >>>> What if we do something like `CustomException(AirflowException)`, >>>> so `except AirflowException` blocks still work, but we can gain the >>>> benefit of making it descriptive. >>>> >>>> >>>> Best, >>>> Wei >>>> >>>>> On Aug 30, 2025, at 11:08 PM, Jarek Potiuk <ja...@potiuk.com> wrote: >>>>> >>>>>> I would say we should only worry about backcompat when there's >> something >>>>> to >>>>> worry about. >>>>> >>>>> I think Daniel is quite right. In this case what **would** happen if >>>>> someone relies on `try AirflowException`, the issue would be immediate >> to >>>>> see and easy to fix (and we can describe it in release notes as >>>> significant >>>>> news. I can't easily think of a "reasonable" case where behaviour would >>>> be >>>>> changed without crashing. I think that would be a blocker if we can >> have >>>> a >>>>> reasonable way of using it that might change the behaviour in a way >> that >>>> is >>>>> difficult to notice by the user, but when I think of potential cases I >>>>> cannot imagine usage that could lead to it. >>>>> >>>>> J >>>>> >>>>> >>>>> On Wed, Aug 27, 2025 at 3:31 PM Daniel Standish >>>>> <daniel.stand...@astronomer.io.invalid> wrote: >>>>> >>>>>> Generally agree with this. >>>>>> >>>>>> I would say we should only worry about backcompat when there's >>>> something to >>>>>> worry about. >>>>>> >>>>>> Like, if there's just a remote possibility that a user is doing >>>> something >>>>>> weird (e.g. in some subclass they wrote of something) then, not really >>>> for >>>>>> us to worry about. Seems rather internal behavior generally. >>>>>> >>>>>> If we *know* for sure changing it will break airflow for certain >>>>>> combinations that should work, that's something to address. >>>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>> >>>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> For additional commands, e-mail: dev-h...@airflow.apache.org >> >>
