AI-Native Airflow - LLM-Driven Intelligence for Production Data Workflows

2025-09-30 Thread Pavankumar Gopidesu
Hi everyone, We're exploring adding LLM-powered SQL operators to Airflow and would love community input before writing an AIP. The idea: Let users write natural language prompts like "find customers with missing emails" and have Airflow generate safe SQL queries with full context about your datab

Re: [VOTE] September 2025 PR of the Month

2025-09-27 Thread Pavankumar Gopidesu
Big big +1 for 54383 :) Pavan On Thu, Sep 25, 2025 at 5:01 AM Rahul Vats wrote: > +1 for 54383 -- it was a huge change with almost no regressions, and the > quality of the PR is commendable. > > Regards, > Rahul Vats > > On Thu, 25 Sept 2025 at 09:13, Amogh Desai wrote: > > > Big +1 for 54383!

Re: Airflow website's build environment has been modernized

2025-09-25 Thread Pavankumar Gopidesu
Thats awesome, thanks kaxil for updating it, indeed we have been using very old versions :) Pavan On Thu, Sep 25, 2025 at 11:11 PM Kaxil Naik wrote: > Hi team, > > The build environment for airflow-site has finally been modernized !!! > > PR: https://github.com/apache/airflow-site/pull/1244 > >

Re: [ANNOUNCE] Apache Airflow 3.1.0 & Task SDK 1.1.0 Released

2025-09-25 Thread Pavankumar Gopidesu
Wooohoo Awesome. thanks kaxil, and thank you everyone one for all your contributions. Pavan On Thu, 25 Sep 2025 at 21:44, Vikram Koka via dev wrote: > Those numbers are awesome and scary at the same time! > > On Thu, Sep 25, 2025 at 1:43 PM Kaxil Naik wrote: > > > 163 contributors made this re

Re: [ACCELERATED VOTE] Release Airflow 3.1.0. from 3.1.0rc2 & Task SDK 1.1.0 from 1.1.0rc2

2025-09-25 Thread Pavankumar Gopidesu
+1 non-binding. Tested example dags, everything looks good. Great work everyone :) Regards, Pavan On Thu, Sep 25, 2025 at 9:54 AM Rahul Vats wrote: > +1 (non-binding). > > For both airflow-core 3.1.0 RC2 and task-sdk 1.1.0 RC2 > I have verified all regression tests with Celery and K8s executo

Re: [DISCUSS] Replace constraints with uv.lock mechanisms for dev env freeze

2025-08-19 Thread Pavankumar Gopidesu
I am aiming at in this "uv.lock" change is not even to use > > > dependabot for it, but to fulfill those two seemingly contradictory > > goals: > > > > > > * keep the seamless experience of upgrades in "canary" and allowing us &g

Re: [ACCELERATED VOTE] Release Airflow 3.0.5 from 3.0.5rc3 & Task SDK 1.0.5 from 1.0.5rc3

2025-08-19 Thread Pavankumar Gopidesu
+1 non-binding. Executed a few example dags all LGTM. Pavan On Tue, Aug 19, 2025 at 1:40 PM Ash Berlin-Taylor wrote: > +1 (binding) > > > On 19 Aug 2025, at 04:08, Wei Lee wrote: > > > > +1 (non-binding) > > > > Tested with a few dags and worked fine. > > > > Best, > > Wei > > > >> On Aug 19,

Re: [DISCUSS] Replace constraints with uv.lock mechanisms for dev env freeze

2025-08-19 Thread Pavankumar Gopidesu
Thanks Jarek for bringing this up, I really like this idea we should use uv.lock. For upgrading packages in the uv.lock file, dependaboat started adding support for this, but it seems there are quite a lot of issues. Some i have seen dependaboat updates lock file but not updating the package in to

Re: [ANNOUNCEMENT] Switched pre-commit to prek in our CI/Breeze

2025-08-17 Thread Pavankumar Gopidesu
Woohoo great work Jarek thank you :) Thank you Jo and ash. Pavan On Sun, Aug 17, 2025 at 9:27 AM Aritra Basu wrote: > This is a good change, looking forward to seeing it adopted more > -- > Regards, > Aritra Basu > > On Sun, 17 Aug 2025, 12:36 pm Jarek Potiuk, wrote: > > > Hello everyone, > >

Re: [VOTE] Airflow Providers prepared on August 12, 2025

2025-08-13 Thread Pavankumar Gopidesu
+1 non-binding Verified changes and triggered some examples all looks good. Pavan On Wed, Aug 13, 2025 at 12:12 PM Jarek Potiuk wrote: > +1 (binding): > * checked reproducibility > * checked file list > * checked installing in docker > * checked signatures, licences, checksums > > All good! >

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-12 Thread Pavankumar Gopidesu
ot;airlflow-preflight" package and use it in the docs everywhere - > >>>> that > >>>>>>> might be the "cleanest" solution eventually and prevent > >>>> typosquatting. > >>>>>>> On Fri, Aug 8, 2025 at 10:58 AM Ash Berlin-Taylor &

Re: [ANNOUNCE] SQLAlchemy 2 tests added

2025-08-11 Thread Pavankumar Gopidesu
Woohoo great work both. That's indeed a lot of effort :) Pavan On Mon, Aug 11, 2025 at 6:03 AM Dev iL wrote: > Thanks for the shout-out, and your patient guidance throughout this > process, Jarek :) > > At multiple steps along the way this PR grew so large that parts of it had > to be split ou

Re: [VOTE] Airflow Providers prepared on August 07, 2025

2025-08-08 Thread Pavankumar Gopidesu
Verified my changes and they are LGTM +1 non-binding for all providers except standard. -1 non-binding for standard provider, as it is failing for PythonVirtulaenvOperators. Pavan On Thu, Aug 7, 2025 at 3:39 PM Jarek Potiuk wrote: > +1 (binding) - everything minus standard provider > -1 (bin

Re: [DISCUSS] Upcoming pre-commit -> prefligit change

2025-08-08 Thread Pavankumar Gopidesu
Indeed nice tool. agree its bit hard to manipulate monorepo with breeze, i am fine with to drop breeze static-check. yeah naming bit confused me, i thought same preflight and searched in google, it got me flight related iternary and checks :) haha On Fri, 8 Aug 2025 at 07:28, Jarek Potiuk wr

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

2025-08-06 Thread Pavankumar Gopidesu
+1 non-bindig Ran few example dags working fine, there is one issue with the standard provider `PythonVirtualenvOperator` getting an unpack error. but that is not a blocker for release. https://github.com/apache/airflow/issues/54124#issuecomment-3161779344 Pavan On Wed, Aug 6, 2025 at 9:18 PM

Re: [VOTE] Airflow Providers prepared on July 29, 2025

2025-08-01 Thread Pavankumar Gopidesu
+1 non-binding, Verified all my changes LGTM, mostly mypy. triggered some example dags working fine. Pavan On Wed, Jul 30, 2025 at 1:27 PM Jarek Potiuk wrote: > +1 (binding). Checked: > > * SVN > * Docker installation > * reproducibility > * licences > * signatures > > I checked all my changes

Re: [VOTE] July 2025 PR of the Month

2025-07-29 Thread Pavankumar Gopidesu
+1 #49470 good work Jason. offocurse others great work all. agree with Jarek would be great if we expand and choose more :) On Tue, 29 Jul 2025 at 15:48, Vincent Beck wrote: > Many candidates this month :) My vote will go to #51657. This PR required > multiple iterations and change of directio

Re: [ANNOUNCE] Task SDK Integration Test Framework Merged

2025-07-29 Thread Pavankumar Gopidesu
cool. good work amogh :) On Fri, 25 Jul 2025 at 13:32, Aritra Basu wrote: > Nice! > -- > Regards, > Aritra Basu > > On Fri, 25 Jul 2025, 5:07 pm Jarek Potiuk, wrote: > > > Cool > > > > pt., 25 lip 2025, 13:07 użytkownik Amogh Desai > > napisał: > > > > > Hi All, > > > > > > I am excited to ann

Re: [DISCUSS] Removing dev dependencies from PyPI constraints ?

2025-07-25 Thread Pavankumar Gopidesu
Thanks Jarek, I am fine with option b as far as if we find anything while testing we can downgrade. Pavan On Thu, Jul 24, 2025 at 10:44 PM Jarek Potiuk wrote: > Would love to hear more if anyone has an opinion :). Yep. It's an advanced > topic and I can make arbitrary decision :) ... > > I am

Re: [VOTE] Airflow Providers prepared on July 17, 2025

2025-07-18 Thread Pavankumar Gopidesu
+1 non binding. Tested a few examples with ShortCircuitOperator , looks good to me. Pavan On Fri, Jul 18, 2025 at 7:40 AM Jarek Potiuk wrote: > +1 (binding) - checked: > > * svn > * installation > * reproducibility > * licences > * signatures > * checksums > > I installed the standard provider

Re: Python 3.13 is coming

2025-07-17 Thread Pavankumar Gopidesu
Verified the docs related changes, looking good :) Thanks Jarek, great work. Pavan On Thu, Jul 17, 2025 at 1:26 PM Jarek Potiuk wrote: > All right .. I addressed the next round of feedback and some investigations > where things were questioned led to interesting findings :) > > * interesting f

Re: [HELP NEEDED / Contribution Opportunity] Fix/Cleanup unreachable code mypy warnings

2025-07-16 Thread Pavankumar Gopidesu
gt; > > On Wed, Jul 16, 2025 at 8:29 PM Ankit Chaurasia > wrote: > > > Thank you for the detailed description on the issue. I have picked up a > few > > providers. > > > > *Ankit Chaurasia* > > > > > > > > > > > > > >

[HELP NEEDED / Contribution Opportunity] Fix/Cleanup unreachable code mypy warnings

2025-07-16 Thread Pavankumar Gopidesu
Hello All, I am back with one more kind request, This time fixing/cleaning up unreachable code. Jens and I were discussing this, we scanned the providers in our codebase by enabling the --warn-unreachable flag in mypy. It turns out we encountered around 120 warnings. Some of these warnings might

Re: [ANNOUNCE] Apache Airflow 3.0.3 Released

2025-07-15 Thread Pavankumar Gopidesu
woohoo, Thank you everyone. Pavan On Tue, Jul 15, 2025 at 7:17 AM Amogh Desai wrote: > That was a long one!! > > Thanks to everyone who contributed to this release > > Thanks & Regards, > Amogh Desai > > > On Tue, Jul 15, 2025 at 12:27 AM Kaxil Naik wrote: > > > Dear Airflow community, > > >

Re: [HELP NEEDED / Contribution Opportunity] Cleanup MyPy ignores

2025-07-14 Thread Pavankumar Gopidesu
| 16 | | @kyungjunleeme | 2| Looking forward to more collaboration. Regards, Pavan On Mon, Jul 14, 2025 at 5:49 PM Kaxil Naik wrote: > Nice 🎉 > > On Mon, 14 Jul 2025 at 20:29, Pavankumar Gopidesu > > wrote: > > > Thanks Sunak, yes we are almo

Re: [HELP NEEDED / Contribution Opportunity] Cleanup MyPy ignores

2025-07-14 Thread Pavankumar Gopidesu
to see the bump happening!! > > > > > > > > This is a great opportunity to learn how our static checks work too > :) > > > > > > > > Thanks & Regards, > > > > Amogh Desai > > > > > > > > > > > > On Sun

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc6 & Task SDK 1.0.3 from 1.0.3rc6

2025-07-14 Thread Pavankumar Gopidesu
+1 non-binding, Tested with a few example dags , LGTM. Pavan On Fri, Jul 11, 2025 at 6:51 PM Kaxil Naik wrote: > Hey fellow Airflowers, > > The release candidates for *Apache Airflow 3.0.3rc6 *and *Task SDK > 1.0.3rc6* are > now available for testing! > > This email is calling for a vote on

[HELP NEEDED / Contribution Opportunity] Cleanup MyPy ignores

2025-07-12 Thread Pavankumar Gopidesu
Hello All, I have a kind request to help clean up mypy ignores in the airflow code base. We have bumped mypy from 1.9.0 to the latest 1.16.1.The latest version has a good number of fixes, we have the opportunity now to remove mypy ignores where possible. Details are here https://github.com/apach

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc5 & Task SDK 1.0.3 from 1.0.3rc5

2025-07-11 Thread Pavankumar Gopidesu
+1 non-binding, Tested few example dags, Looking all good. Pavan On Thu, Jul 10, 2025 at 10:04 PM Jens Scheffler wrote: > Airflow-core: +1 (binding) - Checked SVN, Reproducible package build, > Licenses, Signatures > > Task-SDK: +1 (binding) - Checked SVN, Reproducible package build, > Licenses

Re: S3 Dag Bundle Versions and DB Manager

2025-07-09 Thread Pavankumar Gopidesu
Agree another s3 bucket also works here Another option also would be Using dynamodb table? that also supports snapshots and i feel it works very well with state management. Pavan On Wed, Jul 9, 2025 at 2:06 PM Jarek Potiuk wrote: > One of the options would be to use a similar approach as terr

Re: [VOTE] AIP-67 Mutlti-team deployment of Airflow Components (RELOADED)

2025-07-08 Thread Pavankumar Gopidesu
Thanks Jarek for updated work, Took a bit of time and re-read again, now it's more concise and nice to see the update compared to initial proposal. +1 binding. Pavan On Tue, Jul 8, 2025 at 8:24 PM Jens Scheffler wrote: > Thanks Jarek for the rework. I see added complexity for Multi-Team but

Re: [DISCUSS] Use ~= for python requires?

2025-07-07 Thread Pavankumar Gopidesu
Thanks Jarek, I am just catching up with this discussion. I agree that this is unilaterally forcing us to make changes, though the one (~=3.10) is also the standard one we have been using. I am in favour of using our existing convention ~=3.10. Pavan On Mon, Jul 7, 2025 at 10:12 AM Jarek Potiuk

Re: [HELP NEEDED / Contribution Opportunity] Migrate Provider Packages to use `BaseHook` via version_compat

2025-07-07 Thread Pavankumar Gopidesu
That's awesome , thank you everyone. Pavan On Mon, Jul 7, 2025 at 10:09 AM Amogh Desai wrote: > Hi All, > > Here are a few stats that highlight the strength and dedication of this > community: > > * ✅ ~62 PRs were merged in under 4 days > * 📆 Half of that time was over a weekend — including a l

Re: [VOTE] Release Airflow 3.0.3 from 3.0.3rc3 & Task SDK 1.0.3 from 1.0.3rc3

2025-07-06 Thread Pavankumar Gopidesu
+1 non-binding. Tested a few examples: dags, LGTM. There is one default example dag broken with import errors, I have added a fix here, https://github.com/apache/airflow/pull/52956. Pavan On Sat, Jul 5, 2025 at 11:09 PM Jarek Potiuk wrote: > I think yes - if it appeared in 3.0.3 and was not

Re: [VOTE] Airflow Providers prepared on July 03, 2025

2025-07-04 Thread Pavankumar Gopidesu
+1 non-binding. Tested with some sample dags LGTM. Pavan. On Thu, Jul 3, 2025 at 9:27 PM Jens Scheffler wrote: > +1 (binding) - Checked SVN, Check in Docker, Reproducible package build, > Licenses, Signatures > > Tested Docker, Standard and Edge Provider Packages with EdgeExecutor in > Airflow

Re: Code sharing between Airflow Core and Task SDK - how do we achieve it

2025-07-03 Thread Pavankumar Gopidesu
Thanks Ash Yes agree option 2 would be preferred for me. Making sure we have all the gaurdriles to protect any unwanted behaviour in code sharing and executing right of tests between the packages. Agree with others, option 2 would be On Thu, Jul 3, 2025 at 10:02 AM Amogh Desai wrote: > Thanks

Re: [ANNOUNCE] Python 3.9 support has been dropped on main branch

2025-06-27 Thread Pavankumar Gopidesu
Great work Elad. pavan On Fri, 27 Jun 2025 at 15:00, Amogh Desai wrote: > Good effort Elad. > > I am sure it wasn't easy to maintain a green PR with all the changes coming > in into the repo continuously. > > Well executed! > > Thanks & Regards, > Amogh Desai > > > On Fri, Jun 27, 2025 at 6:57 

Re: [VOTE] June 2025 PR of the Month

2025-06-27 Thread Pavankumar Gopidesu
Same for me vote goes to #51699. one of thing it solves triggers deadlock. I would like to shoutout aiologic maintainer, IIy Egorov, who helped on alot on the discussions continously on the locking approach and suggestions. Pavan On Fri, 27 Jun 2025 at 08:18, Shahar Epstein wrote: > I agree

Re: [ANNOUNCE] New PMC member: Amogh Desai

2025-06-26 Thread Pavankumar Gopidesu
Woohoo that's great news, well deserved. Congratulations Amogh... Pavan On Thu, Jun 26, 2025, 11:20 Kaxil Naik wrote: > Dear Airflow Community, > > The Project Management Committee (PMC) for Apache Airflow has invited > *Amogh > Desai *to become a PMC member and we are excited to announce that

Re: [DISCUSS] What should we cherry-pick ?

2025-06-23 Thread Pavankumar Gopidesu
Thanks Jarek, for starting this discussion, I agree with all the points. The real intention behind to backport https://github.com/apache/airflow/pull/51992 is , this area has a lot of ongoing development going and I felt it's worth porting to v3-0-test. I myself faced situations where I tried to

Re: [DISCUSS] Dropping Python 3.9 support

2025-06-22 Thread Pavankumar Gopidesu
Thanks elad, yes agree on this to start working on to drop as only few months left to Python 3.9 EOL. +1 Regards, Pavan On Sun, Jun 22, 2025 at 10:03 AM Elad Kalif wrote: > Hi everyone, > > I'd like to propose starting dropping support for Python 3.9. Our policy ( > > https://github.com/apache

Re: Discuss: AIP-67 (multi team) now that AIP-82 (External event driven dags) exists

2025-06-22 Thread Pavankumar Gopidesu
Thanks Jarek, that's a great update on this AIP, now it's much more slim down. left a minor comment. :) Overall looking great. Pavan On Sat, Jun 21, 2025 at 3:10 PM Jens Scheffler wrote: > Thanks for the rework/update of the AIP-72! > > Just a few small comments but overall I like it as it is

Re: [VOTE] Internationalization (i18n) policy

2025-06-05 Thread Pavankumar Gopidesu
+1 binding. Pavan On Thu, Jun 5, 2025, 20:28 Pierre Jeambrun wrote: > +1 binding, well prepared thanks > > On Thu 5 Jun 2025 at 15:16, Wei Lee wrote: > > > +1 binding, with one minor rewording, Taiwanese Mandarin is more > accurate. > > Thanks for all the hard work :) > > > > Best, > > Wei > >

Re: [VOTE] AIP-90 Human in the loop

2025-06-05 Thread Pavankumar Gopidesu
+1 binding.. Pavan On Thu, Jun 5, 2025, 21:08 Ash Berlin-Taylor wrote: > +1 (with it being “human-in-the-loop” still.) > > -ash > > > On 5 Jun 2025, at 11:34, Buğra Öztürk wrote: > > > > +1 binding, looks exciting > > > > On Thu, 5 Jun 2025, 12:29 Tamara Fingerlin, > > wrote: > > > >> +1 non-

Re: [ANNOUNCE] ti.run & task.test now uses Task SDK execution path

2025-05-23 Thread Pavankumar Gopidesu
Nice thanks kaxil. Pavan On Fri, May 23, 2025, 10:50 Amogh Desai wrote: > Damn nice! > > Lesser headache of fixing "two" ways of doing things! > > Thanks & Regards, > Amogh Desai > > > On Fri, May 23, 2025 at 2:22 AM Jarek Potiuk wrote: > > > Cool! > > > > On Thu, May 22, 2025 at 8:19 PM Kaxil

Re: [DISCUSS] Apache OpenDAL Provider (One Layer, All Storage)

2025-05-20 Thread Pavankumar Gopidesu
ed by Pavan is more of the"classic "Operator" use. > > I'd say there are a bit different users of both. > > J. > > > > On Mon, May 19, 2025 at 6:26 PM Pavankumar Gopidesu < > gopidesupa...@gmail.com> > wrote: > > > Yes kaxil, agree on so

Re: [DISCUSS] Apache OpenDAL Provider (One Layer, All Storage)

2025-05-19 Thread Pavankumar Gopidesu
V of what they should be using. > > On Mon, 19 May 2025 at 13:31, Pavankumar Gopidesu > > wrote: > > > Yes Vikram we can discuss. > > > > Pavan, > > > > On Mon, May 19, 2025 at 12:55 AM Vikram Koka > > > > > wrote: > > > > >

Re: [DISCUSS] Apache OpenDAL Provider (One Layer, All Storage)

2025-05-19 Thread Pavankumar Gopidesu
Yes Vikram we can discuss. Pavan, On Mon, May 19, 2025 at 12:55 AM Vikram Koka wrote: > Pavan, > > From a concept perspective and strategic direction, I am in 100% agreement. > I have also been thinking about this and submitted a talk for the Airflow > Summit on this topic. > > I am unsure of t

[DISCUSS] Apache OpenDAL Provider (One Layer, All Storage)

2025-05-17 Thread Pavankumar Gopidesu
I'd like to propose adding a new provider, Apache OpenDAL [1], to the collection of Apache providers in Airflow. OpenDAL (Open Data Access Layer) is a unified abstraction layer that simplifies interactions with various storage backends, including AWS S3, GCS, Azure, and key-value storage systems.

Re: [ANNOUNCE] Staging system for docs - including latest Provider wave

2025-05-15 Thread Pavankumar Gopidesu
Wooho, Thank you Jarek. This is really helpful to test docs and iterate changes. Regards, Pavan On Fri, May 16, 2025 at 7:09 AM Jarek Potiuk wrote: > For those who are still going to take a look, at providers I just completed > some testing and fixes and merged a few PRs of mine (few are still

Re: [ANNOUNCE] Task Execution API Versioning Strategy

2025-05-14 Thread Pavankumar Gopidesu
Great, Thank you kaxil :) Regards, Pavan On Wed, May 14, 2025, 16:12 Kaxil Naik wrote: > Hi team, > > Airflow 3.0+ uses Task Execution API, which uses Cadywn for versioning. To > ensure backwards compatibility is maintained between different Airflow > versioning with regards to this Execution A

Re: [ANNOUNCE] dag.test now uses Task SDK execution path

2025-05-09 Thread Pavankumar Gopidesu
Great, Thanks kaxil. Pavan On Sat, May 10, 2025, 02:52 Aritra Basu wrote: > 🙌 Woohoo! > -- > Regards, > Aritra Basu > > On Sat, 10 May 2025, 4:20 am Vikram Koka, > wrote: > > > Oh, excellent work! > > Excited to see this working. > > > > Thank you, > > Vikram > > > > On Fri, May 9, 2025 at 2:2

Re: [VOTE] April 2025 PR of the Month

2025-05-09 Thread Pavankumar Gopidesu
Indeed, it's a great effort. +1 to this https://github.com/apache/airflow/pull/49532 https://github.com/apache/airflow/pull/49468 On Fri, May 9, 2025 at 8:53 AM Amogh Desai wrote: > Great effort on the docs. > > +1 to https://github.com/apache/airflow/pull/49532 and > https://github.com/apache/

[ANNOUNCE] S3 Docs Publishing

2025-05-09 Thread Pavankumar Gopidesu
Hello All, We have successfully migrated our documentation publishing process from github to AWS S3. As many of you know, our previous publishing workflow was slow and made it difficult for Airflow site release managers and contributors to make changes efficiently. With this transition, the proce

Re: [ANNOUNCE] Speed-up of image releases

2025-05-04 Thread Pavankumar Gopidesu
Woohoo that's huge 👏, thank you Jarek. Pavan. On Sun, May 4, 2025, 21:52 Jarek Potiuk wrote: > Hello here, > > *TL;DR; Pushing release images to DockerHub is going to be more than 5x > faster (<15 minutes rather than way above 1hr).* > > I am not sure if you are aware but for the 3.0.0 release,

Re: [DISCUSS} Enabled experiment auto-merge feature

2025-04-30 Thread Pavankumar Gopidesu
I am also in line with Jens. The auto-merge feature works well in keeping the CI pipeline green before merging. However, there are situations where we need to merge changes quickly. Instead of using an escape route (which I’m not a fan of, as it carries risks if something goes wrong—even though we

Re: Marking issues & PRs for Airflow >3.0.0

2025-04-24 Thread Pavankumar Gopidesu
Yeah cool. Another thing I would like to mention. when PR cherry-picked using label backport-to-*. The default PR state is DRAFT on a backported branch. so please set it back to Read For Review. with this way the workflows will get triggered. Pavan On Thu, Apr 24, 2025 at 11:42 AM Amogh Desai w

Re: AW: [ANNOUNCE] Apache Airflow 3.0.0 Released

2025-04-22 Thread Pavankumar Gopidesu
Woohoo thank you everyone finally we made it. Pavan On Tue, Apr 22, 2025 at 5:49 PM Ankit Chaurasia wrote: > Congratulations everyone. Great work! > > *Ankit Chaurasia* > > > > > > > > On Tue, Apr 22, 2025 at 10:29 PM Avi wrote: > > > Finally... Start of a new era after 5 years of AF 1 and a l

Re: [VOTE] Release Airflow 3.0.0 from 3.0.0rc4 & Task SDK 1.0.0 from 1.0.0rc4

2025-04-22 Thread Pavankumar Gopidesu
+1 non-binding. Triggered example dags, looks good. Thank you everyone for all the hard work. Regards, Pavan On Tue, Apr 22, 2025, 10:32 Karthikeyan wrote: > +1 (non-binding) > > Thanks everyone for all the work throughout this significant milestone. > > Regards, > Karthikeyan S > > On Tue, A

Re: [VOTE] Airflow Providers prepared on April 16, 2025

2025-04-18 Thread Pavankumar Gopidesu
Also verified XcomBackend changes looks good. Regards, Pavan On Fri, Apr 18, 2025, 14:20 Pavankumar Gopidesu wrote: > +1 non-binding. > > Verified my changes related to the standard provider all look good. > > Pavan > > On Fri, Apr 18, 2025 at 12:59 PM Jarek Potiuk w

Re: [VOTE] Airflow Providers prepared on April 16, 2025

2025-04-18 Thread Pavankumar Gopidesu
+1 non-binding. Verified my changes related to the standard provider all look good. Pavan On Fri, Apr 18, 2025 at 12:59 PM Jarek Potiuk wrote: > +1 binding - checked reproducibility (FYI - due to later renaming of the > provider you need to remove generated/provider_dependencies.json manually

Re: Simple auth manager as default auth manager

2025-04-05 Thread Pavankumar Gopidesu
Great work Vincent. Pavan On Fri, Mar 21, 2025, 06:46 Shahar Epstein wrote: > Great job Vincent! > > On Thu, Mar 20, 2025 at 3:18 PM Beck, Vincent > > wrote: > > > Hi everyone, > > > > I just merged https://github.com/apache/airflow/pull/47691 which sets > > SimpleAuthManager as the default au

Re: [ANNOUNCE] UV mandatory tooling and (much nicer) doc building coming

2025-04-04 Thread Pavankumar Gopidesu
That's awesome Jarek, thank you for this :) Regards, Pavan On Wed, Apr 2, 2025 at 1:09 PM Jarek Potiuk wrote: > You might also want to do *docker system prune* or even *docker system > prune --all* or run *breeze doctor* to clean-up some stale cache, images, > docker volumes > > Also IntelliJ

Re: New committer: Rahul Vats

2025-03-31 Thread Pavankumar Gopidesu
Woohoo, Congratulations Rahul 🎉🎉 Pavan. On Mon, Mar 31, 2025 at 9:17 PM Pankaj Koti wrote: > Many congratulations Rahul! Very happy for you and well deserved 🎉🎉 > > On Tue, 1 Apr, 2025, 01:17 Buğra Öztürk, wrote: > > > Amazing news! Congratulations Rahul! Well deserved! > > > > On Mon, Mar 31,

Re: [DISCUSS] The `uv` as the only supported dev tool

2025-03-22 Thread Pavankumar Gopidesu
Yes, I am Ok with it and it is easy to use. Pavan. On Sat, Mar 22, 2025 at 11:58 AM Amogh Desai wrote: > I am OK with this decision too. > > I personally am finding `uv` to be nice and super user friendly. No need to > go through > the past list of commands to install some package, `uv sync`

Re: [ANNOUNCEMENT] airflow-core is there (finally!)

2025-03-21 Thread Pavankumar Gopidesu
Great work Jarek :) Pavan On Fri, Mar 21, 2025 at 5:42 PM Buğra Öztürk wrote: > Amazing news and effort! Thanks Jarek! > > On Fri, 21 Mar 2025, 16:18 Kaxil Naik, wrote: > > > Cool > > > > On Fri, 21 Mar 2025 at 20:46, Aritra Basu > > wrote: > > > > > Culmination of a great bit of effort Jarek

Re: 📢 Apache Airflow 3.0.0beta4 is available for testing! Final beta!! 🎉

2025-03-20 Thread Pavankumar Gopidesu
Whoohoo, Great work everyone :) Pavan On Thu, Mar 20, 2025 at 9:58 PM Buğra Öztürk wrote: > Amazing news! Kudos to everyone who contributed and made this happen! > > On Thu, Mar 20, 2025 at 9:52 PM Kaxil Naik wrote: > > > Docker image is published, use the following: > > > > docker pull apache

Re: New committers: Shubham Raj, Buğra Öztürk, LIU ZHE YOU, Kalyan Reddy !

2025-03-14 Thread Pavankumar Gopidesu
Woohoo. Congratulations Shubham, Buğra, LIU ZHE YOU, Kalyan. Pavan On Fri, Mar 14, 2025, 12:09 Rahul Vats wrote: > Congratulations, well deserved, everyone! > > Regards, > Rahul Vats > > > On Fri, 14 Mar 2025 at 17:35, Jarek Potiuk wrote: > > > Hello everyone, > > > > Wooo Hooo Hooo! > > > >

Re: 📢 Apache Airflow 3.0.0beta3 is available for testing! 🎉

2025-03-13 Thread Pavankumar Gopidesu
That's awesome... Thanks all. Pavan On Fri, Mar 14, 2025, 06:34 Jarek Potiuk wrote: > Wooohooo!!! > > On Fri, Mar 14, 2025 at 4:50 AM Vikram Koka > wrote: > > > Awesome! > > > > Thank you Kaxil, Jed, and all the contributors to this release. It is > great > > to see this coming together and we

Re: 📢 Apache Airflow 3.0.0beta1 is available for testing! 🎉

2025-02-28 Thread Pavankumar Gopidesu
Awesome, Thank you everyone 🎉 On Fri, Feb 28, 2025, 08:31 Jarek Potiuk wrote: > Awesome! > > On Fri, Feb 28, 2025 at 7:47 AM Amogh Desai > wrote: > > > Thank you Jed for getting this one through -- I am sure it wasn't an easy > > task! > > > > And kudos to all the contributors who have worked d

Re: [VOTE] February 2025 PR of the Month

2025-02-26 Thread Pavankumar Gopidesu
+1 #46942 On Wed, Feb 26, 2025 at 2:06 PM Ryan Hamilton wrote: > #46942 given how good it feels to see that much code deleted. > > On Wed, Feb 26, 2025 at 7:04 AM Rahul Vats wrote: > > > +1 for https://github.com/apache/airflow/pull/46939 > > > > Regards, > > Rahul Vats > > > > > > > > On Wed,

Re: Two Hard Things: Deadline Alerts Edition

2025-02-24 Thread Pavankumar Gopidesu
+1 `DeadlineReference` Regards Pavan Kumar On Tue, Feb 25, 2025, 04:11 Amogh Desai wrote: > Late to the party, but I'd vote for `DeadlineReference` too. Concise and > does the job well. > > Thanks & Regards, > Amogh Desai > > > On Tue, Feb 25, 2025 at 7:56 AM Wei Lee wrote: > > > A bit late t

Re: [DISCUSS] Time to say goodbye to the old UI?

2025-02-18 Thread Pavankumar Gopidesu
+1 to this, great step. Regards, Pavan On Tue, Feb 18, 2025, 16:26 Buğra Öztürk wrote: > +1. It would be a great step! It will enable us to clean more and reduce > duplicate code/usage and effort. > > On Tue, Feb 18, 2025 at 5:20 PM Brent Bovenzi > > wrote: > > > +1 > > > > - We absolutely don

Re: [ANNOUNCE] All providers moved to the new provider's structure !

2025-02-17 Thread Pavankumar Gopidesu
Fantastic work everyone! Pavan On Wed, Feb 12, 2025 at 10:40 PM Vikram Koka wrote: > Amazing work team! > Kudos to everyone who contributed to this effort. > > It was wonderful to see this coming together and I was amazed how quickly > all of you made this happen. > > Vikram > > > On Tue, Feb

Re: Very strange (AI generated) issues

2025-01-22 Thread Pavankumar Gopidesu
gt; > We got to the point that we are closing and reporting such > >> issues > >> > to > >> > > >>> > GitHub and we are blocking all such users without spending too > >> much > >> > > >>> time on > >> &g

Re: Very strange (AI generated) issues

2025-01-21 Thread Pavankumar Gopidesu
Yes, still issues are coming. Regards, Pavan On Wed, Jan 22, 2025 at 4:35 AM Amogh Desai wrote: > I saw a couple of such SPAM issues too. > > I also recall some SPAM comments on pull requests as well, so if any > contributor sees any such SPAM message, > please report it on Slack so that we can

Re: [VOTE] Amendment to AIP-83 re uniqueness of logical date

2025-01-21 Thread Pavankumar Gopidesu
+1 binding Regards, Pavan Kumar On Wed, Jan 22, 2025, 01:35 Wei Lee wrote: > +1 binding > > Best, > Wei > > > On Jan 22, 2025, at 9:12 AM, Phani Kumar > wrote: > > > > +1 binding > > > > On Tue, 21 Jan, 2025, 13:55 Vikram Koka, > > wrote: > > > >> +1 binding. > >> > >> Thank you Daniel Standi

Re: [ANNOUNCE] Upgraded Sphinx to 7

2025-01-13 Thread Pavankumar Gopidesu
Woohoo 🎉 great job Jarek, Shahar , Jens. We are on track :) Regards, Pavan On Mon, Jan 13, 2025, 06:52 Shahar Epstein wrote: > I was happy to take part in upgrading Sphinx! The last Dev call triggered > me to do something about it :) > > It was plenty of trial and error until we've figured ou

Re: [DISCUSS] Drop support for the DAG processor embedded in the scheduler

2025-01-10 Thread Pavankumar Gopidesu
+1 Regards, Pavan On Fri, Jan 10, 2025, 15:34 Vikram Koka wrote: > +1 on this > > For many reasons, which have already been brought up in the thread. > > > On Fri, Jan 10, 2025 at 7:30 AM Igor Kholopov > > wrote: > > > +1, there are a lot of old code paths that exist only because of the > > e

Re: [VOTE] December 2024 PR of the Month

2025-01-09 Thread Pavankumar Gopidesu
Everyone has done an excellent job. I would also vote for https://github.com/apache/airflow/pull/45266. as it addresses a long-standing issue related to pull_request_target and includes numerous improvements to the CI process. Great work Jarek. Regards, Pavan On Thu, Jan 9, 2025 at 8:33 PM Shah

Re: [ANNOUNCE] Simplified (and safer) CI workflows !

2024-12-29 Thread Pavankumar Gopidesu
Woohooo Thanks Jarek Great work :) Regards, Pavan On Sun, Dec 29, 2024 at 10:15 PM Jarek Potiuk wrote: > > Hello here, > > TL;DR; I just merged https://github.com/apache/airflow/pull/45266 - > which implemented a much simplified and nicer workflow for our CI. > > Rebase to the latest `main` and

Re: [VOTE] Airflow Providers prepared on December 22, 2024

2024-12-24 Thread Pavankumar Gopidesu
+1 non binding. Regards, Pavan Kumar On Mon, Dec 23, 2024, 06:30 Utkarsh Sharma wrote: > +1 Non-binding > > Thanks, > Utkarsh Sharma > > On Mon, Dec 23, 2024 at 11:12 AM Amogh Desai > wrote: > > > +1 non binding > > > > Checked one of my changes and ran a few example DAGs with CNCF provider, >

Re: [VOTE] Release Airflow 2.10.4 from 2.10.4rc1

2024-12-11 Thread Pavankumar Gopidesu
+1 non-binding, Tested few examples dags. Looks fine. Regards, Pavan Kumar On Thu, Dec 12, 2024, 05:20 Amogh Desai wrote: > +1 non binding. > > Tested my example DAG set and everything seems to work as expected. > > @Scheffler Jens (XC-DX/ETV5) what challenge > did you have with breeze? > Did

Re: [DISCUSS] Consistency of version checks in providers

2024-12-06 Thread Pavankumar Gopidesu
I agree with all the points. Yes, we definitely need a mechanism to verify imports and identify the source of imports. Validating these will help in the release process smoothly. Recently, we faced an issue, as Jarek mentioned, where Elad tried to release providers and suddenly there were mismatche

Re: [DISCUSS] Avoid Yoda conditions in our code

2024-11-28 Thread Pavankumar Gopidesu
gt; https://github.com/apache/airflow/pull/44466 -> Note it's not only for > asserts, there are a number of other places where comparison was not RIGHT > (pun intended).. > > > On Fri, Nov 29, 2024 at 7:50 AM Pavankumar Gopidesu > wrote: > > > +1 agree :) > >

Re: [DISCUSS] Avoid Yoda conditions in our code

2024-11-28 Thread Pavankumar Gopidesu
+1 agree :) Regards, Pavan On Fri, Nov 29, 2024, 06:37 Amogh Desai wrote: > Vote +1 for this, I do. (Sorry couldn't help it) > > It's nice that there is some automation to do this. It was always slightly > annoying to me > coming from a different programming language background > > Thanks & Re

Re: AIP-84 Public API Migration

2024-11-28 Thread Pavankumar Gopidesu
Woohoo 🙌 thank you everyone :) On Thu, Nov 28, 2024, 18:13 Vincent Beck wrote: > HUGE! > > On 2024/11/28 17:25:45 Jarek Potiuk wrote: > > WOOWOOWOW! > > > > On Thu, Nov 28, 2024 at 6:11 PM Pierre Jeambrun > > wrote: > > > > > Hello all, > > > > > > We just completed the migration of the pu

Re: [VOTE] November 2024 PR of the Month

2024-11-27 Thread Pavankumar Gopidesu
+1 for 41916, Great work Illumaria. Regards, Pavan Kumar On Wed, Nov 27, 2024 at 4:36 PM kalyan reddy wrote: > > +1 for https://github.com/apache/airflow/pull/43902 > > From: Kaxil Naik > Sent: 27 November 2024 20:14 > To: dev@airflow.apache.org > Subject: Re:

Re: Status on weekend Pydantic 2.10.0/ 2.10.1 issue ("Unable to evaluate type annotation")

2024-11-26 Thread Pavankumar Gopidesu
Yes indeed. Thanks for sharing the updates. Regards, Pavan Kumar On Tue, Nov 26, 2024, 13:32 Amogh Desai wrote: > Thanks for the update and following this issue closely, Jarek. > > Thanks & Regards, > Amogh Desai > > > On Tue, 26 Nov 2024 at 3:52 PM, Jarek Potiuk wrote: > > > *TL;DR; In case y

Re: [ANNOUNCE] Cherry Picker Automation Now in GitHub Actions!

2024-11-18 Thread Pavankumar Gopidesu
not be created automatically? (Or asked > another way, how do I know when I need to manually do the back port PR?) > > > > On 18 Nov 2024, at 20:35, Pavankumar Gopidesu wrote: > > > > Hello All, > > > > We have set up Cherry Picker Automation that supports bot

[ANNOUNCE] Cherry Picker Automation Now in GitHub Actions!

2024-11-18 Thread Pavankumar Gopidesu
Hello All, We have set up Cherry Picker Automation that supports both automated and manual backporting to any branch. 1. For automated backporting, simply add the required labels to your PR (eg, backport-to-v2-10-test, backport-to-v2-9-test) 2. Once the PR is merged, the changes will be automati

Re: [DISCUSSION] Replace Poke & Reschedule mode from Sensors for Airflow 3 in favor of Deferrable

2024-11-13 Thread Pavankumar Gopidesu
Great points. Maintaining both sync and async hooks is indeed challenging, and making the triggerer mandatory in Airflow 3 seems like a sensible move to simplify the system. However, for users heavily relying on the reschedule option, migrating to Airflow 3 could be difficult if this option isn’t

Re: [VOTE] AIP-86 Deadline Alerts (Formerly SLAs)

2024-11-05 Thread Pavankumar Gopidesu
+1 binding. Regards, Pavan On Tue, Nov 5, 2024 at 6:27 PM Jarek Potiuk wrote: > > +1 binding. I really like how it is now simple and straightforward and > stating "deadline" not SLAs. For reasons discussed before "deadline" is > soo much better. Seems like many of our AIPs are immediately ge

Re: New committer: Pavan Kuman Gopidesu (gopidesupavan)

2024-10-31 Thread Pavankumar Gopidesu
Thank you all for your support! I am excited to serve as a committer. Tha Airflow community has been welcoming since I started contributing, and I have acquired valuable insights that have significantly enhanced my development. I'll work hard to ensure this positive experience continues for everyon

Re: [ANNOUNCE] New PMC member: Vikram Koka

2024-10-28 Thread Pavankumar Gopidesu
Congrats Vikram! 🎉🎉. Regards, Pavan Kumar On Tue, Oct 22, 2024 at 11:30 AM Igor Kholopov wrote: > Congrats Vikram! > > -Igor > > On Mon, Oct 21, 2024 at 6:07 PM Oliveira, Niko > > wrote: > > > Congrats Vikram! Very well deserved indeed. I'm in the camp of folks who > > assumed you were already

Re: [VOTE] Airflow Providers prepared on October 10, 2024

2024-10-12 Thread Pavankumar Gopidesu
+1 non binding, tested my changes on a few dag examples, everything looks good! Regards, Pavan Kumar On Sat, Oct 12, 2024 at 4:43 AM Kaxil Naik wrote: > +1 binding on sigs, license & checksums > > On Fri, 11 Oct 2024 at 20:46, Jarek Potiuk wrote: > > > +1 (binding) - checked reproducibility, c

Re: [DISCUSS] pools and deferrables

2024-10-10 Thread Pavankumar Gopidesu
Apologies if my earlier message wasn't clear! What I was trying to convey is that, since we're discussing pool and deferrable tasks. Yes, there's also been some discussion around managing deferred tasks at the DAG level, which might be relevant here. The PR I mentioned (https://github.com/apache/ai

Re: [DISCUSS] pools and deferrables

2024-10-09 Thread Pavankumar Gopidesu
Thanks Daniel for putting this together. As we are discussing here about pool/slots occupancy, at present there is no way to control how many deferred task can be executed except pool configuration IMHO. There is issue created by Raphaelauv to control the number of deferred tasks on dag level. ht

Re: Re: [DISCUSS] Sensor Improvements With Tirggers

2024-10-07 Thread Pavankumar Gopidesu
msgraph example. The usage of > TimeDeltaTrigger in combination with retry_execute is very elegant. > > I'm happy to help in any way. > > Best, > > Alex > > On 2024/09/27 07:51:51 Pavankumar Gopidesu wrote: > > Thank you David, Yes this is not to block any

Re: [PROPOSAL] Pre-installling standard provider and releasing dev version manually as needed

2024-10-03 Thread Pavankumar Gopidesu
Thanks Jarek , I am updating the pr now. Regards, Pavan On Thu, Oct 3, 2024 at 8:26 AM Jarek Potiuk wrote: > I provisionally generated and uploaded the .dev0 package > https://pypi.org/project/apache-airflow-providers-standard/1.0.0.dev0/ > from > latest main - and we shall see if everything wo

  1   2   >