I would also like to see it deprecated.
That said, I am not convinced there is anything that we cannot encode using
URI though. I think the problem is just when one tries to use the same URI
to mean two different things, e.g. both airflow connection URI and
sqlalchemy URI. They are different. H
Greetings!
I want to propose to deprecate Airflow Connection URI representation and
remove it in Airflow 3 in favor of the already existing replacement - JSON
representation.
In the past URI representation helped to add one of the awesome features -
Alternative Secrets Backends: Environment Varia
I was asked to open the issue in GitHub to get more visibility by Dask
developers so here it is https://github.com/dask/community/issues/355
On Fri, Nov 17, 2023 at 1:21 PM Jarek Potiuk wrote:
> OK. Seeing that - I think I will do the next step - I pointed this
> discussion to at the discord of
Just to clarify I'd like us to consider the possibility that no new
pendulum would be released or released at the end of 2024, like a
pessimistic scenario:
- What should we do in this case?
- Work out a backup plan.
Best Wishes
*Andrey Anshin*
On Fri, 17 Nov 2023 at 16:33, Jarek Potiuk w
I agree that cases could be different:
- someone use _PIP_ADDITIONAL_REQUIREMENTS
- someone install packages without pinning Airflow version
- some others might use official images without pinning specific versions
of python
In all cases this would lead to unintentional/unpredictable results.
Cha
I added you, Yulei,
I have no time to look at details, but I have two big concerns about this -
regarding Audience and Security (first concern) and whether we want to do
it all (second concern).
First about security and audience:
This is against the current Security Model of Airflow:
https://ai
And yes - agree that the environmental effect is smaller than "bare" Python
benchmark in our case - but I think it is still there.
There are a number of (valid) cases where people use airflow not only to
purely orchestrate external services, and they are using it run
computationally or logic-inten
Yeah. I see the point of Andrey - indeed, we had - for quite some time -
Python 3.11 exclusion for HDFS providers - until it has been fixed. and we
already have a built-in mechanism to exclude providers from certain
versions of Python - it's part of provider.yaml definition and we can deal
with it
Also I think TP - had a document in the past (years ago) describing a
draft of a more complete alternative we can take to approach datetime vs.
pendulum dichotomy. I cannot easily find the document and discussion - but
I do remember it was proposing some interesting changes in the approach of
Air
OK. Seeing that - I think I will do the next step - I pointed this
discussion to at the discord of Dask and see if there is a volunteer there
who would like to take on fixing the "Quarantined" issue
https://github.com/apache/airflow/issues/32778 -> we have the flaky tests
currently marked as "Quara
Ah ... put it in a wrong thread, sorry :) ...
On Fri, Nov 17, 2023 at 12:39 PM Jarek Potiuk wrote:
> OK. Seeing that - I think I will do the next step - I will point this
> discussion to at the discord of Dask and see if there is a volunteer there
> who would like to take on fixing the "Quaranti
Hi,
I agree that the current speed of development of Pendulum leaves something
to be desired. However, I think we should not underestimate the effort of
replacing it. It is not just a matter of %s/pendulum/datetime/g so to say.
If we are *truly* thinking about moving to native datetime / zoneinfo
OK. Seeing that - I think I will do the next step - I will point this
discussion to at the discord of Dask and see if there is a volunteer there
who would like to take on fixing the "Quarantined" issue
https://github.com/apache/airflow/issues/32778 -> we have the flaky tests
currently marked as "Qu
+1 for this moving it. It gives us more flexibility on both the core and
provider sides.
Best,
Wei
> On Nov 17, 2023, at 9:15 AM, Jarek Potiuk wrote:
>
> I am all for it. As we saw already and we see it more in the future -
> moving code of out of Airflow core to provider and having separate
>
+1 for removal if there is no active maintainer on this one
Best,
Wei
> On Nov 17, 2023, at 6:00 PM, Andrey Anshin wrote:
>
> +1 for suspend and after a while remove
>
> There is a small chance that things would improve after we suspended this
> provider.
>
> However we do not have a lot of s
There is no changes in stable pendulum so let's try to continue this
discussion and start think about "Plan B"
Just a reminder:
- pendulum 2.1.2 released 3 years ago (at the time Airflow 1.10.x)
- pendulum 2 doesn't work well in Python 3.12, this is a showstopper for
the support Python 3.12
- pend
+1 for suspend and after a while remove
There is a small chance that things would improve after we suspended this
provider.
However we do not have a lot of statistics, AFAIK (correct me if I'm wrong)
it is only two cases:
* 1 provider suspended in the past and restored (Yandex), there is no
proce
+1 All for removing it if it's not in use and is giving us trouble with
maintaining.
--
Regards,
Aritra Basu
On Fri, Nov 17, 2023, 1:47 PM Amogh Desai wrote:
> Theres very little incentive in maintaining this if theres no one actively
> maintaining it.
>
> I am totally for the removal +1
>
>
>
Personally for me it is controversial change and tradeoff between Stability
vs Performance
Since Airflow + Providers have 400+ dependencies, using the lowest version
of python provides better stability and the reason for this is pretty
simple - time spent for maintainers of packages to make it mor
Theres very little incentive in maintaining this if theres no one actively
maintaining it.
I am totally for the removal +1
On Fri, 17 Nov 2023 at 1:21 AM, Collin McNulty
wrote:
> +1 for removal
>
> On Thu, Nov 16, 2023 at 1:43 PM Hussein Awala wrote:
>
> > > we would do it branching off at t
I also agree with this idea.
It is always a good idea to be up to date with the python dependencies as
they have fixes for performance, scalability among other things.
As long as users have a mechanism to go back to the python version of their
interest, I do not see any problem in proceeding with
21 matches
Mail list logo
