[RESULT][VOTE] Release Airflow Python Client 2.3.0

2022-08-02 Thread Sumit Maheshwari
Hello Dev Community, The release of Apache Airflow Client 2.3.0 (based on RC1) has been accepted. 5 “+1” votes received: - Ash Berlin-Taylor (binding) - Jarek Potiuk (binding) - Pankaj Koti - Phani Kumar - Sumit Maheshwari (binding) No "-1" votes received Vote thread: https://lists.apache.org/t

Re: [DISCUSS] Trimming down Airlfow Versions in issues

2022-08-02 Thread Jarek Potiuk
To focus on a possible solution - I created a PR with the removal of all the versions but the latest from a given branch and updated the description - https://github.com/apache/airflow/pull/25480 Let me know what you think. J. On Wed, Jul 27, 2022 at 4:52 PM Jarek Potiuk wrote: > Other ideas/o

Re: [Discuss] AIP-44 Airflow Database API

2022-08-02 Thread Jarek Potiuk
Hello, I have updated AIP-44 with the findings from the POC. If there are any more questions/comments, please let me know, otherwise I plan to start voting tomorrow. https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-44+Airflow+Internal+API On Mon, Jul 25, 2022 at 10:18 PM Jarek Potiuk wr

Re: Auto-registering of DAGs in DAG file? (no `as dag` needed?)

2022-08-02 Thread Jarek Potiuk
Less is more. I like the lack of "as dag". I think it's not really a breaking change. We can easily argue we are adding functionality rather than introducing a breaking change. I think the only reason why someone would create a DAG in a function is to return it and eventually add it to globals().

Re: Auto-registering of DAGs in DAG file? (no `as dag` needed?)

2022-08-02 Thread Felix Uellendall
Ah I should have checked your PR, sorry. I was looking at the first example. In general I like the idea of removing the `as dag` in the context manager syntax. Best,Felix Sent with [Proton Mail](https://proton.me/) secure email. --- Original Message --- On Tuesday, August 2nd, 2022 at 1

Re: Auto-registering of DAGs in DAG file? (no `as dag` needed?)

2022-08-02 Thread Pankaj Koti
Will this impact DAG file processing time? If we consider to include the change, we might also need to consider informing the user that such functions need to be lightweight inline with what we've here for top-level-code best practices: https://airflow.apache.org/docs/apache-airflow/stable/best-pr

Re: Auto-registering of DAGs in DAG file? (no `as dag` needed?)

2022-08-02 Thread Bas Harenslak
For me having to add “as dag” is somewhat non-evident. Also, it’s a few less boilerplate characters you have to write which I think is a good thing in this case. Curious on the edge cases in DAG generation as mentioned by Felix, but I like the idea so +1 for me. Bas > On 2 Aug 2022, at 13:18,

Re: Auto-registering of DAGs in DAG file? (no `as dag` needed?)

2022-08-02 Thread Felix Uellendall
Hey Ash, I personally don't like it, because it is not obvious to me. Also what happens if you return the `dag_2` variable and set the return value in the global context to `dag_2` as well? This is how I used to do it when generating DAGs - and in my opinion this is pythonic way of doing it wit

Auto-registering of DAGs in DAG file? (no `as dag` needed?)

2022-08-02 Thread Ash Berlin-Taylor
Hello all, I'm on a bit of a kick thinking about developer (specifically DAG author) experience and if there is anything we can Some time ago there was a previous conversation about if we should/could "autoregister" DAGs, rather than just looking at the objects in the top level (globals())