Re: Looking for a speaker on Airflow for workshop on workflow languages for particle physics analysis: 3-5 April, 2024 at CERN

2024-03-20 Thread Neil
Hello, I would reccomend referencing Airflow Summit speakers as well. https://airflowsummit.org/ On Tue, Mar 19, 2024 at 11:14 PM Clemens Lange wrote: > Dear Airflow developers, > > The first week of April (3-5, April) there will be a workshop at CERN on > the current and future use of

Re: [VOTE] Release Airflow 2.8.4 from 2.8.4rc1

2024-03-20 Thread Jarek Potiuk
+1 (binding) - tested / verified all changes I was involved (either as fixer, bug introducer or both, particularly when both), verified reproducibility, licences, checksums, signatures, run a few DAGs - all looks good. On Wed, Mar 20, 2024 at 4:56 PM Jed Cunningham wrote: > Hey fellow

Re: [VOTE] Remove experimental API

2024-03-20 Thread Andrey Anshin
There is no specific rush, in case it is considered as experimental feature, this vote shows that it is not, it might be removed in any minor release. Benefit: remove legacy/unsupported/unmaintained code from codebase, rather than move it into the separate component (if someone wanted they might

Re: [DISCUSS] Applying D105 rule for our codebase ("undocumented magic methods") ?

2024-03-20 Thread Oliveira, Niko
I'm -1 to enabling D105 I don't think it will lead to helpful documentation. I think for the rare cases it is required it can left up to the developer or caught in PR review. Cheers, Niko From: Vincent Beck Sent: Wednesday, March 20, 2024 5:51:43 AM To:

[VOTE] Release Airflow 2.8.4 from 2.8.4rc1

2024-03-20 Thread Jed Cunningham
Hey fellow Airflowers, I have cut Airflow 2.8.4rc1. This email is calling a vote on the release, which will last at least 72 hours, from Wednesday, March 20, 2024 at 4:00 pm UTC until Saturday, March 23, 2024 at 4:00 pm UTC, and until 3 binding +1 votes have been received.

Re: Bad mixing of decorated and classic operators (users shooting themselves in their foot)

2024-03-20 Thread Andrey Anshin
> It handles everything. Now if you want to send a Slack message from a > PythonOperator you need to initialize a hook, find the right function to invoke etc. > And to Elad point " "I know there is an operator that does X, so I will just use it inside the python function I invoke from the

Re: [DISCUSS] Applying D105 rule for our codebase ("undocumented magic methods") ?

2024-03-20 Thread Vincent Beck
+1 for not enforcing as well. Let's leave to maintainers the flexibility to chose whether a given method should be documented. On 2024/03/20 08:28:51 Ash Berlin-Taylor wrote: > I'm for not enforcing this rule - as others have said its very unlikely to > result in more useful docs for

Re: Bad mixing of decorated and classic operators (users shooting themselves in their foot)

2024-03-20 Thread Jarek Potiuk
I am really torn on that one to be honest. I am OK with the error (with the note that it will likely break a lot of workflows), I am ok with the warning as well as a softer way of letting the user know they are doing it wrong). But ultimately, I'd really want we (re) consider if we cannot make it

Re: Bad mixing of decorated and classic operators (users shooting themselves in their foot)

2024-03-20 Thread Ash Berlin-Taylor
The reason users are sure they can use operators like that is that it has worked for a long time - hell I even wrote a custom nested operator in the past (pre 2.0 admittedly). So this pr should only be a warning by default, or a config option to warn but not error Alternatively do we just

Re: [DISCUSS] Applying D105 rule for our codebase ("undocumented magic methods") ?

2024-03-20 Thread Ash Berlin-Taylor
I'm for not enforcing this rule - as others have said its very unlikely to result in more useful docs for developers or end users. -asg On 20 March 2024 08:12:40 GMT, Andrey Anshin wrote: >±0 from my side > >Maybe we have to review all current methods which do not follow this rule >to find a

Re: [DISCUSS] Applying D105 rule for our codebase ("undocumented magic methods") ?

2024-03-20 Thread Andrey Anshin
±0 from my side Maybe we have to review all current methods which do not follow this rule to find a really useful meaning, and do not enforce (disable it). So for avoid unnecessary changes we might close https://github.com/apache/airflow/issues/37523 and remove/mark completed into the

Re: Python 3.12 support is here (!)

2024-03-20 Thread Jarek Potiuk
FYI. Python 3.12 is now fully back - ready for 2.9.0. We even got some of the providers that were excluded, back in: * databricks is back in after the issue we had was determined as "Python coverage" tool not working well yet for Python 3.12 (we disabled coverage for Python 3.12 now) *

Re: Bad mixing of decorated and classic operators (users shooting themselves in their foot)

2024-03-20 Thread Jarek Potiuk
Just to add to the discussion - a discussion raised today https://github.com/apache/airflow/discussions/38311 where the user is sure that they can use operators in such a way as described above, and even used the term "nested operator". I think getting

Re: [DISCUSS] Applying D105 rule for our codebase ("undocumented magic methods") ?

2024-03-20 Thread Pankaj Koti
+1 to what Aritra is saying. Best regards, *Pankaj Koti* Senior Software Engineer (Airflow OSS Engineering team) Location: Pune, Maharashtra, India Timezone: Indian Standard Time (IST) Phone: +91 9730079985 On Wed, Mar 20, 2024 at 12:05 PM Aritra Basu wrote: > I'm in general not a huge fan

Re: [DISCUSS] Applying D105 rule for our codebase ("undocumented magic methods") ?

2024-03-20 Thread Aritra Basu
I'm in general not a huge fan of documenting for the sake of documenting, so I'd be in agreement of not enforcing it via code but rather be enforced by the reviewers in cases they believe certain methods need documenting. -- Regards, Aritra Basu On Wed, Mar 20, 2024, 9:39 AM Jarek Potiuk wrote: