Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-18 Thread Kaxil Naik
Hi Meghya, There is also “airflow task test” that can run a specific task for debugging. On Tue, 18 Jun 2024 at 21:04, Meygha Machado wrote: > Hi Jarek, > > Thank you for summarizing the discussion points and linking to the two PRs > that enhance the dag test feature. > > We (at Nielsen) are

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-18 Thread Meygha Machado
Hi Jarek, Thank you for summarizing the discussion points and linking to the two PRs that enhance the dag test feature. We (at Nielsen) are working on a PR to extend the `airflow dag test` to be able to reconstruct the context from an existing dag run. I think allowing the option to execute

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-17 Thread Amogh Desai
I agree with you Jarek. Every developer has their own way of debugging things and sharing those with the community including the best practices. There is always scope for improvement to the documentation! Thanks & Regards, Amogh Desai On Fri, Jun 14, 2024 at 12:17 PM Jarek Potiuk wrote: >

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-14 Thread Jarek Potiuk
Also I would like to drag attention of those who were interested in the subject on two related PRs: * https://github.com/apache/airflow/pull/40010 by jannisko (sorry if you see it - I do not know your real name :) ) -> where you can run a dag test while skipping (or actually mark as success) some

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-13 Thread Stefan Krawczyk
+1 for the recording please. On Thu, Jun 13, 2024, 1:26 PM Jarek Potiuk wrote: > Just summarizing the call: > > * we had demo from Nielsen showing their debugging feature > * It's based on environments created in the research environment where > users can run DAGs and debug individual tasks -

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-13 Thread Jarek Potiuk
Just summarizing the call: * we had demo from Nielsen showing their debugging feature * It's based on environments created in the research environment where users can run DAGs and debug individual tasks - basically replaying and debugging the tasks based on existing DAG runs but without saving

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-12 Thread Albert Okiri
Hi Jarek, I'm interested in joining this call. Regards, Albert. On Thu, 13 Jun 2024, 07:43 Poorvi Rohidekar, wrote: > Hi Jarek, > > I'd be interested in joining this call. > > Regards, > Poorvi > > On Tue, 11 Jun 2024 at 21:42, Jarek Potiuk wrote: > > > And yes. I will check with them about

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-12 Thread Poorvi Rohidekar
Hi Jarek, I'd be interested in joining this call. Regards, Poorvi On Tue, 11 Jun 2024 at 21:42, Jarek Potiuk wrote: > And yes. I will check with them about recording :) > > On Tue, Jun 11, 2024 at 4:41 PM Jarek Potiuk wrote: > > > I think I have to warn Nielsen team that we are going to have

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-11 Thread Jarek Potiuk
And yes. I will check with them about recording :) On Tue, Jun 11, 2024 at 4:41 PM Jarek Potiuk wrote: > I think I have to warn Nielsen team that we are going to have a big crowd > :) > > On Tue, Jun 11, 2024 at 4:40 PM Bishundeo, Rajeshwar > wrote: > >> This sounds exciting! I would like to

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-11 Thread Jarek Potiuk
I think I have to warn Nielsen team that we are going to have a big crowd :) On Tue, Jun 11, 2024 at 4:40 PM Bishundeo, Rajeshwar wrote: > This sounds exciting! I would like to join as well. > > -- Rajesh > > > > > > > On 2024-06-10, 11:12 PM, "Ephraim Anierobi"

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-11 Thread Bishundeo, Rajeshwar
This sounds exciting! I would like to join as well. -- Rajesh On 2024-06-10, 11:12 PM, "Ephraim Anierobi" mailto:ephraimanier...@gmail.com>> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-10 Thread Ephraim Anierobi
Hi Jarek, Awesome!! I’m interested to join too Ephraim On Mon, 10 Jun 2024 at 23:41, Mehta, Shubham wrote: > This is great, thanks for working with them to share this with the > community. Interested to join as well. > > Shubham > > On 2024-06-07, 11:56 PM, "Jarek Potiuk" ja...@potiuk.com>>

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-10 Thread Mehta, Shubham
This is great, thanks for working with them to share this with the community. Interested to join as well. Shubham On 2024-06-07, 11:56 PM, "Jarek Potiuk" mailto:ja...@potiuk.com>> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-10 Thread Wei Lee
Hi Jarek, Love to hear about it! But it’s way too late for my time.  Would it be possible for us to record it? Best, Wei > On Jun 11, 2024, at 7:12 AM, Julian LaNeve > wrote: > > +1 I'd love to hear about it but unfortunately can't make the meeting! > > -- > > *Julian LaNeve* > CTO > >

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-10 Thread Julian LaNeve
+1 I'd love to hear about it but unfortunately can't make the meeting! -- *Julian LaNeve* CTO Email: jul...@astronomer.io ( jul...@astronomer.io ) Mobile: 330 509 5792 On Mon, Jun 10, 2024 at 3:05 PM, Constance Martineau < consta...@astronomer.io.invalid > wrote: > > > > Hello again, > >

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-10 Thread Constance Martineau
Hello again, Given all the enthusiasm - assuming Nielsen is ok with this - what if someone recorded the meeting so that it could be shared with those that are interested? Constance On Mon, Jun 10, 2024 at 1:48 PM Constance Martineau wrote: > Hi Jarek, > > Same :) > > Thanks, > Constance > >

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-10 Thread Constance Martineau
Hi Jarek, Same :) Thanks, Constance On Mon, Jun 10, 2024 at 9:57 AM Amogh Desai wrote: > Hello Jarek, > > Please add me to the invite as well. > > Thanks & Regards, > Amogh Desai > > > On Mon, Jun 10, 2024 at 11:22 AM Abhishek Bhakat > wrote: > > > Hi Jarek, > > > > I would also like to join

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-10 Thread Amogh Desai
Hello Jarek, Please add me to the invite as well. Thanks & Regards, Amogh Desai On Mon, Jun 10, 2024 at 11:22 AM Abhishek Bhakat wrote: > Hi Jarek, > > I would also like to join as well, please. > > Thanks, > Avi > > On Sat, Jun 8, 2024 at 3:32 PM Buğra Öztürk > wrote: > > > Hello Jarek, >

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-09 Thread Abhishek Bhakat
Hi Jarek, I would also like to join as well, please. Thanks, Avi On Sat, Jun 8, 2024 at 3:32 PM Buğra Öztürk wrote: > Hello Jarek, > > Thanks for sharing! It sounds very interesting. I would like to join. Could > you please forward to me as well? > > Thanks! > > On Sat, 8 Jun 2024, 17:28 Jed

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-08 Thread Buğra Öztürk
Hello Jarek, Thanks for sharing! It sounds very interesting. I would like to join. Could you please forward to me as well? Thanks! On Sat, 8 Jun 2024, 17:28 Jed Cunningham, wrote: > Interesting. Can you forward to me as well Jarek? Thanks! >

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-08 Thread Jed Cunningham
Interesting. Can you forward to me as well Jarek? Thanks!

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-08 Thread Pankaj Koti
Hello Jarek, Thanks for sharing this. I would like to join as well, please! On Sat, 8 Jun 2024, 14:33 Sumit Maheshwari, wrote: > Same, add me as well. Thanks!! > > On Sat, Jun 8, 2024 at 2:27 PM Kaxil Naik wrote: > > > Same, would love to join > > > > On Sat, 8 Jun 2024 at 09:19, Aritra Basu

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-08 Thread Sumit Maheshwari
Same, add me as well. Thanks!! On Sat, Jun 8, 2024 at 2:27 PM Kaxil Naik wrote: > Same, would love to join > > On Sat, 8 Jun 2024 at 09:19, Aritra Basu wrote: > > > Hi Jarek, > > > > I'd be interested to join. > > > > -- > > Regards, > > Aritra Basu > > > > On Sat, Jun 8, 2024, 12:24 PM Jarek

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-08 Thread Kaxil Naik
Same, would love to join On Sat, 8 Jun 2024 at 09:19, Aritra Basu wrote: > Hi Jarek, > > I'd be interested to join. > > -- > Regards, > Aritra Basu > > On Sat, Jun 8, 2024, 12:24 PM Jarek Potiuk wrote: > > > Hello here, > > > > At PyCon US I met a few people from Nielsen who had developed

Re: Call with Nielsen team demoing their DAG debugging feature

2024-06-08 Thread Aritra Basu
Hi Jarek, I'd be interested to join. -- Regards, Aritra Basu On Sat, Jun 8, 2024, 12:24 PM Jarek Potiuk wrote: > Hello here, > > At PyCon US I met a few people from Nielsen who had developed internally > tooling for IDE/Python debugger integrated debugging of Airflow DAGs. > > They are

Call with Nielsen team demoing their DAG debugging feature

2024-06-08 Thread Jarek Potiuk
Hello here, At PyCon US I met a few people from Nielsen who had developed internally tooling for IDE/Python debugger integrated debugging of Airflow DAGs. They are thrilled with the opportunity of sharing what they've done and possibly maybe even bringing it to Airflow. As one of the Airflow 3