Re: Request

2022-04-27 Thread lambzee gn
Sure thing. Regards Harsha On Thu, 28 Apr, 2022, 5:21 am Ashir Ali, wrote: > I am hereby writing this email to request about unsubscribing the emails > of community. > > Best Regards, > Ashir >

Request

2022-04-27 Thread Ashir Ali
I am hereby writing this email to request about unsubscribing the emails of community. Best Regards, Ashir

Re: task_id declarations

2022-04-27 Thread Ferruzzi, Dennis
Yeah, alright, I guess that makes sense. Thank you both for the replies. Maybe it's just confirmation bias in action, but it seems like I almost always see DAGs written like ``` with DAG(dag_id="simple", start_date=datetime(2022, 4, 1)) as dag: task1 = Operator1(task_id='task1')

[VOTE] Release Airflow 2.3.0 from 2.3.0rc2

2022-04-27 Thread Ephraim Anierobi
Hey fellow Airflowers, I have cut Airflow 2.3.0rc2. This email is calling a vote on the release, which will last for 72 hours, from Wednesday, April 27, 2022, at 08:48 pm UTC until Saturday, April 30, 2022, at 08:48 pm UTC

Re: task_id declarations

2022-04-27 Thread Jed Cunningham
Also keep in mind you don't even have to put tasks in a variable, for example: ``` with DAG(dag_id="simple", start_date=datetime(2022, 4, 1)) as dag: BashOperator(task_id="hello", bash_command="echo hello") BashOperator(task_id="world", bash_command="echo world") ```

Re: Implicit DAG registration

2022-04-27 Thread Jarek Potiuk
That looks like a nice improvement. J. On Wed, Apr 27, 2022 at 10:04 PM Malthe wrote: > > On Wed, 27 Apr 2022 at 15:27, Constance Martineau > wrote: > > Am intrigued. Curious about dynamic dag pattern, where you create the DAG > > object in a create_dag function and adding the DAG to globals.

Re: Implicit DAG registration

2022-04-27 Thread Malthe
On Wed, 27 Apr 2022 at 15:27, Constance Martineau wrote: > Am intrigued. Curious about dynamic dag pattern, where you create the DAG > object in a create_dag function and adding the DAG to globals. Would this new > way prevent someone from modifying the dag object within the function, or >

Re: [DISCUSSION] AIP-49 OpenTelemetry Support for Apache Airflow

2022-04-27 Thread Howard Yoo
Hi Elad, I have updated the AIP-49 with the appropriate changes to contain what you requested in your previous comments. I think the AIP is fairly comprehensive and ready to be voted, unless there is any objections. Sincerely, Howard On Mon, Apr 25, 2022 at 11:03 PM Howard Yoo wrote: > Hi

Re: [VOTE] Release Airflow 2.3.0 from 2.3.0rc1

2022-04-27 Thread Ephraim Anierobi
A bug was found in rc1 that'll necessitate an rc2. I'm canceling this vote and will create 2.3.0rc1 soon. On Wed, 27 Apr 2022 at 10:58, Ephraim Anierobi wrote: > Hey fellow Airflowers, > > I have cut Airflow 2.3.0rc1. This email is calling for a vote on the > release, > which will last for 72

Re: task_id declarations

2022-04-27 Thread Jarek Potiuk
I think the original reason is how Python parsing works. At the moment we create the task the variable name is not known. First task is created as an object and then the result of it is assigned to a variable. And I think we have no super-reliable way (unless there is some wild Python trickery) to

task_id declarations

2022-04-27 Thread Ferruzzi, Dennis
Hi folks, I'm hoping for a little history lesson. I'm idly wondering if there is a way to make a fairly big change (for me), but want to understand the reason it is the way it is now, before I go and put much time into "fixing" it. Every time I write a DAG it bugs me that we have to

Re: Implicit DAG registration

2022-04-27 Thread Constance Martineau
Am intrigued. Curious about dynamic dag pattern, where you create the DAG object in a a create_dag function and adding the DAG to globals. Would this new way prevent someone from modifying the dag object within the function, or returning it? > On Apr 27, 2022, at 11:20 AM, Ferruzzi, Dennis >

Re: Implicit DAG registration

2022-04-27 Thread Ferruzzi, Dennis
I don't know what it would take under the hood, but I'm intrigued. From a user perspective, anything to make the DAG DRYer is a win IMHO. From: Malthe Sent: Wednesday, April 27, 2022 1:49 AM To: dev@airflow.apache.org Subject: [EXTERNAL] Implicit DAG

Re: Implicit DAG registration

2022-04-27 Thread Ash Berlin-Taylor
This has often caught me out. And I think so long as there's an option to disable the auto-registration (for power users) this sounds like a great feature. Hash On 27 April 2022 09:49:50 BST, Malthe wrote: >DAGs must be registered at the module top-level. During dagbag >processing we have: >

[VOTE] Release Airflow 2.3.0 from 2.3.0rc1

2022-04-27 Thread Ephraim Anierobi
Hey fellow Airflowers, I have cut Airflow 2.3.0rc1. This email is calling for a vote on the release, which will last for 72 hours, from Wednesday, April 27, 2022, at 09:58 am UTC until Saturday, April 30, 2022, at 09:58 am UTC

Implicit DAG registration

2022-04-27 Thread Malthe
DAGs must be registered at the module top-level. During dagbag processing we have: top_level_dags = ((o, m) for m in mods for o in m.__dict__.values() if isinstance(o, DAG)) It makes sense of course – we can't have DAGs floating in space, they need an anchor. Or do they? It would be