[VOTE] Airflow Providers prepared on August 07, 2025

Hey all, I have just cut the new wave Airflow Providers packages. This email is calling a vote on the release, which will last for 72 hours - which means that it will end on August 10, 2025 06:50 AM UTC and until 3 binding +1 votes have been received. Consider this my (binding) +1. Airflow Provi

Airflow Summit Workshops Are Live 🎟️ Secure Your Spot!

[image: Airflow Summit 2025] *Dear Airflow Community, * The wait is over! *Airflow Summit 2025 workshops are now open for registration.* Whether you're just getting started or looking to deepen your Apache Airflow expertise, our workshops will help you get hands-on w

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

> You're talking about https://github.com/apache/airflow/pull/53492/ right? Yes. > Where is the PR from @Christos? https://github.com/apache/airflow/pull/54103 On Wed, Aug 6, 2025 at 23:51 Daniel Standish wrote: > > > > IMO, the approach on the patch isn't easily maintainable. Most of the >

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

+1 non-bindig Ran few example dags working fine, there is one issue with the standard provider `PythonVirtualenvOperator` getting an unpack error. but that is not a blocker for release. https://github.com/apache/airflow/issues/54124#issuecomment-3161779344 Pavan On Wed, Aug 6, 2025 at 9:18 PM

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

> > IMO, the approach on the patch isn't easily maintainable. Most of the > calculations are performed by SQL in a huge query. > It would be my preference to have many smaller queries and do part of the > calculations in python. This will be easier to understand, maintain and > debug in the future.

Re: [DISCUSS] Unnecessary dag hash changes at dag run time

If dag hash update is blocking dag run table queries are we locking the dag table when we update the hash? Because, there is a lower level lock on postgres, SELECT FOR NO KEY UPDATE, which tells the DB, "i am not going to change the PK", and this reduces locking of things that have a FK refere

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

Oh, repeating on the "right" thread: Airflow Core 3.0.4rc2: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Note: Source Tar.GZ is building with a differecen but same like RC1 viewing the diff in `diffoscope` shows just a symlink packaged permission diff because

Re: [VOTE] Release Apache Airflow from 3.0.4rc1 and TaskSDK from 1.0.4rc1

Airflow Core: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures Note: Source Tar.GZ is building with a differecen but same like RC1 viewing the diff in `diffoscope` shows just a symlink packaged permission diff because I tested on Linux vs. it was built on MacOS. So

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

Hi, I was (until now) not be able to re-read all the Slack discussion and like to make this latest at the weekend. I also like Jarek fear that the optimization makes the Scheduler rather hard to maintain. We also had some points where we_thought_ we can contribute some optimizations especiall

Re: [VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

+1 (binding): Checked: airflow: 3.0.4rc2 * reproducibility (this time I run it on linux and reproduced the same issue as Jens with symlink permission - I will fix it shortly) * checksums * signatures * licences * docker installation Checked task-sdk 1.0.4rc1: * reproducibility * checksums * sign

[VOTE] Release Apache Airflow from 3.0.4rc2 and TaskSDK from 1.0.4rc1

Hello everyone, Time for the next 3.0.4 vote. As the TaskSDK code has not changed, that is still at rc1, but this is a combined vote for all three packages. This email is calling for a vote on the release, which will last at least until 9th August (*46* hours from now, this is a repost of the v

Re: [VOTE] Release Apache Airflow from 3.0.4rc1 and TaskSDK from 1.0.4rc1

I’m going to start a new thread on this with the correct subject, but the time and body on this is correct, so it’s not a new vote > On 6 Aug 2025, at 15:26, Ash Berlin-Taylor wrote: > > Hello everyone, > > Time for the next 3.0.4 vote. As the TaskSDK code has not changed, that is > still at

Re: [DISCUSS] Merging cherry-picked PRs when green (and actually doing it)

Bad choice of words. > the author feels confident enough to merge Was referring to committers, normal author's cannot merge. Thanks & Regards, Amogh Desai On Wed, Aug 6, 2025 at 3:58 PM Jarek Potiuk wrote: > > I believe Pierre suggested that the responsibility lies with the one who > merges

[VOTE] Release Apache Airflow from 3.0.4rc1 and TaskSDK from 1.0.4rc1

Hello everyone, Time for the next 3.0.4 vote. As the TaskSDK code has not changed, that is still at rc1, but this is a combined vote for all three packages. This email is calling for a vote on the release, which will last at least until 9th August (*48* hours from now) and until 3 binding +1 vo

Cancelled: [VOTE] Release Airflow 3.0.4 from 3.0.4rc1 & Task SDK 1.0.4 from 1.0.4rc1

We noticed a problem with the dependency specification in apache-airflow-core on this one (it was possible to have TaskSDk 1.0.3 installed still, which meant the API server failed to start) We will follow up shortly with RC2 with a 48hour vote window as the changes are very minimal between rc1

Re: [DISCUSS] Merging cherry-picked PRs when green (and actually doing it)

> I believe Pierre suggested that the responsibility lies with the one who merges it, rather than the author, which makes more sense to me. The author might not have the permission to merge the backport PR. The person merging the PR should either merge the backport PR or ping the author to resolve

Re: [DISCUSS] Merging cherry-picked PRs when green (and actually doing it)

I believe Pierre suggested that the responsibility lies with the one who merges it, rather than the author, which makes more sense to me. The author might not have the permission to merge the backport PR. The person merging the PR should either merge the backport PR or ping the author to resolve

Re: [DISCUSS] Unnecessary dag hash changes at dag run time

It’s not even a core change — give the bug only affects Kube operators the fix can be included in the next provider release. -ash > On 6 Aug 2025, at 08:59, Jarek Potiuk wrote: > > Yes. We merge about 20 PRs a day - that is 140 PRs a week. Probably ~10% of > those is some fix to some (more or

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

> My main issue and the topic of this thread, has been that the scheduler does unnecessary work that leads to decreased throughput. My solution has been to limit the results of the query to the dag cap of active tasks that the user has defined. Yes. I understand that. There are situations that cau

Re: [DISCUSS] Unnecessary dag hash changes at dag run time

Yes. We merge about 20 PRs a day - that is 140 PRs a week. Probably ~10% of those is some fix to some (more or less obscure) core logic change. You should take a look at all the PRs that are getting merged and reviewed. This is just one of them. There is no need for 1000s of people who are subscri

Re: [DISCUSS] Unnecessary dag hash changes at dag run time

Yep. Thanks for the heads up. We saw both PR and the issue and it is scheduled for 3.0.5 - it did not make it in 3.0.4. I think it would be good if you confirm in your PR that you applied the patch and show some evidences of what happened - before and after, not only "word" explanation - words and

Re: [DISCUSS] Unnecessary dag hash changes at dag run time

Well, my email is not about the single PR or a follow up on that PR. It is referring to an issue in the core logic that results into DAG hash change. Jigar > On Aug 6, 2025, at 12:20 AM, Jarek Potiuk wrote: > > Yep. Thanks for the heads up. > > We saw both PR and the issue and it is schedul

Re: join the project’s mailing list

Welcome! I would highly also recommend that you join the community Slack workspace. The steps are already present here: https://airflow.apache.org/community/. Slack is a space where you can collaborate with the community and get involved in a lot of discussions with like-minded people :) Thanks