Re: [DISCUSS] Dutch locale is being outdated for 2 consecutive releases

2025-08-04 Thread Jarek Potiuk
I can share mine here :). It was extremely simple: "Please translate the 'TODO translate:' entries into LANGUAGE, following the style of existing translations". The tool of ours that adds TODO: with `--add-missing` prepares you the file with "TODO: translate: Original phrase" - AI will just do wha

Re: [DISCUSS] Dutch locale is being outdated for 2 consecutive releases

2025-08-04 Thread Jarek Potiuk
+1 to waiting till 3.1.0 with the decision, but also great we regularly check and raise it now. Hopefully that might increase interest in getting it "complete" :). Maybe we have some Dutch speaking people who could join the efforts? This is a fantastic - and generally very easy and quick if you use

Re: [DISCUSS] Dutch locale is being outdated for 2 consecutive releases

2025-08-04 Thread Amogh Desai
+1 to waiting until 3.1. Jarek, Pierre, Elad, Shahar, and others who have used vibe-coding for this, if you have any prompts that could help people with only basic knowledge of the language contribute to the translations, it might be worth sharing those over Slack. Thanks & Regards, Amogh Desai

Re: [DISCUSS] Merging cherry-picked PRs when green (and actually doing it)

2025-08-04 Thread Amogh Desai
Good point Pierre, yes it will anyways be reviewed when it's going to be ported over to v3-0-stable. I think the bottom line here is that if it's a simple PR with no conflicts / the author feels confident enough to merge, they should. Thanks & Regards, Amogh Desai On Mon, Aug 4, 2025 at 5:31 PM

Re: [DISCUSS] Dutch locale is being outdated for 2 consecutive releases

2025-08-04 Thread Shahar Epstein
Thanks Pierre and Wei, it makes sense - let's all sync again before the release of 3.1.0, I'll contact the relevant people for this case and ask them to close the gaps before that. On Tue, Aug 5, 2025, 03:34 Pierre Jeambrun wrote: > I don’t think this should apply until 3.1.0 is out and interna

Re: [DISCUSS] Dutch locale is being outdated for 2 consecutive releases

2025-08-04 Thread Wei Lee
+1 to what Pierre suggested. It’s still great to have such a reminder. We probably will need to check all the translation progress late Aug or early Sep. Best, Wei > On Aug 5, 2025, at 8:34 AM, Pierre Jeambrun wrote: > > I don’t think this should apply until 3.1.0 is out and internationalisa

Re: [DISCUSS] Dutch locale is being outdated for 2 consecutive releases

2025-08-04 Thread Pierre Jeambrun
I don’t think this should apply until 3.1.0 is out and internationalisation is released. Basically it can be out of date as long as the translation gap is closed for 3.1.0. IMHO On Mon 4 Aug 2025 at 22:13, Shahar Epstein wrote: > Hey everyone, > > I'd like to point out that the Dutch locale has

Re: [VOTE] Release Airflow 3.0.4 from 3.0.4rc1 & Task SDK 1.0.4 from 1.0.4rc1

2025-08-04 Thread Jens Scheffler
Hi Ash! Tested 3.0.4RC1 with latest EdgeExecutor and Integration Test DAG and all looks good! Airflow-Core 3.0.4rc1: +1 (binding) - Checked SVN, Reproducible package build, Licenses, Signatures I am not sure why the source tarball is binary different when I build locally but inspecting the

[DISCUSS] Dutch locale is being outdated for 2 consecutive releases

2025-08-04 Thread Shahar Epstein
Hey everyone, I'd like to point out that the Dutch locale has been out of sync for quite a while - less than 14% coverage, which is fairly low. There has been a PR for fixing it, yet it progresses quite slow: https://github.com/apache/airflow/pull/51233 As we're getting closer to a 2nd consecutiv

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

2025-08-04 Thread Christos Bisias
I created a draft PR for anyone interested to take a look at the code https://github.com/apache/airflow/pull/54103 I was able to demonstrate the issue in the unit test with much fewer tasks. All we need is the tasks brought back by the db query to belong to the same dag_run or dag. This can happe

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

2025-08-04 Thread Daniel Standish
> > The configurability was my recommendation for > https://github.com/apache/airflow/pull/53492 > Given the fact that this change is at the heart of Airflow I think the > changes should be experimental where users can switch between different > strategies/modes of the scheduler. > If and when we h

[VOTE] Release Airflow 3.0.4 from 3.0.4rc1 & Task SDK 1.0.4 from 1.0.4rc1

2025-08-04 Thread Ash Berlin-Taylor
Hey fellow Airflowers, The release candidates for *Apache Airflow 3.0.4rc1 *and *Task SDK 1.0.4rc1* are now available for testing! This email is calling for a vote on the release, which will last at least until 7th August (72 hours from now) and until 3 binding +1 votes have been received. Con

Metrics PR review

2025-08-04 Thread Christos Bisias
Hello, Is there anyone familiar with metrics that might be interested to take a look at this PR? https://github.com/apache/airflow/pull/53722 Thank you in advance! Regards, Christos Bisias

Re: Proposal: Add Apache SeaTunnel Provider Integration

2025-08-04 Thread Jia Fan
Thanks Jarek for your response and kind words. We will add the integration tests soon. Jarek Potiuk 于2025年8月4日周一 20:23写道: > > I personally - generally - welcome all the "Apache" projects to contribute > providers. They almost automatically fulfill all the criteria we have for > new providers: > >

Re: Proposal: Add Apache SeaTunnel Provider Integration

2025-08-04 Thread Jarek Potiuk
I personally - generally - welcome all the "Apache" projects to contribute providers. They almost automatically fulfill all the criteria we have for new providers: * they are open-source (and truly open source) * they have dedicated team of maintainers who "know" how Open Source works and their in

Re: [DISCUSS] Merging cherry-picked PRs when green (and actually doing it)

2025-08-04 Thread Pierre Jeambrun
Hello, I also considered that this is the responsibility of the person merging the PR to make sure that it is backported correctly if applicable. (PR opened and merged). I always try to fill in when I see that some PR is missing the backport, but it's harder when this is a PR we didn't follow alon

Re: Proposal: Add Apache SeaTunnel Provider Integration

2025-08-04 Thread Jia Fan
Thanks haicheng. Here is github pr link: https://github.com/apache/airflow/pull/50226 and demo link: https://github.com/apache/seatunnel/issues/9178 haicheng zhang 于2025年8月4日周一 19:19写道: > > Dear Airflow Developer Community, > >Hello! > >We have completed the initial implementation

Proposal: Add Apache SeaTunnel Provider Integration

2025-08-04 Thread haicheng zhang
Dear Airflow Developer Community, Hello! We have completed the initial implementation of the Apache SeaTunnel Provider for Airflow, including: - Operator/Sensor components for seamless integration - Support for submission, monitoring, and log collection of SeaTunnel jobs - Compreh

Re: [DISCUSS] Best practices for initializing `ObjectStoragePath`

2025-08-04 Thread Josef Šimánek
po 4. 8. 2025 v 12:42 odesílatel Bolke de Bruin napsal: > > Josef is proposing to make ObjectStoragePath construction > environment-agnostic by storing provider and base path > in Airflow connections. So you just need to change the connection > configuration. I'm trying to make it actually provi

Re: [DISCUSS] Best practices for initializing `ObjectStoragePath`

2025-08-04 Thread Bolke de Bruin
Josef is proposing to make ObjectStoragePath construction environment-agnostic by storing provider and base path in Airflow connections. So you just need to change the connection configuration. This makes indeed a tighter coupling and makes me wonder about the deployment model. While creating Obj

Re: [DISCUSS] AIP-92 Isolate DAG parsing logic

2025-08-04 Thread Jarek Potiuk
> > My main concern with this right now is the serialisation format of DAGs — > it wasn’t really designed with remote submission in mind, so it need some > careful examination to see if it is fit for this purpose or not. I understand Ash's concerns - the format has not been designed with size/spee

Re: [DISCUSS] Merging cherry-picked PRs when green (and actually doing it)

2025-08-04 Thread Jarek Potiuk
> Just want to make sure. Should we merge our own backport PR when it’s ready? I feel a bit uncomfortable merging it, as it’s technically my code. I still requested others to review it again in the past few occurrences. As I see it - yes we can merge them, but I think (as usual) - it depends :).

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

2025-08-04 Thread Christos Bisias
> > So my question to you is: is it impossible, or just demanding or difficult > to split your Dags into smaller dags connected with asset aware scheduling? Jarek, I'm going to discuss this with the team and I will get you an answer on that. I've shared this again on the thread https://github.c

Re: [DISCUSS] AIP-92 Isolate DAG parsing logic

2025-08-04 Thread Sumit Maheshwari
> > My main concern with this right now is the serialisation format of DAGs — > it wasn’t really designed with remote submission in mind, so it need some > careful examination to see if it is fit for this purpose or not. > I'm not sure on this point, cause if we are able to convert a DAG into JSON

Re: [DISCUSS] Scheduler does unnecessary processing when there are very large scheduled dags

2025-08-04 Thread Elad Kalif
Let me add some details 1. The DagRuns starvation issue was reported in https://github.com/apache/airflow/issues/49508 but no one was able to reproduce it so far. If anyone has reproduced an example please share on the issue. 2. The task instance starvation issue was reported in