Re: Airflow Docs Development Issues

2023-10-26 Thread Bowrna Prabhakaran
I would also like to join in this efforts. On Fri, Oct 27, 2023 at 8:19 AM Ryan Hatter wrote: > I'm happy to work on this alongside Utkarsh, Amogh Desai, and Aritra Basu > :) > Some thoughts on Utkarsh's proposal (and what him and I have been > discussing offline): > >1. I think we should

Re: Airflow Docs Development Issues

2023-10-26 Thread Ryan Hatter
I'm happy to work on this alongside Utkarsh, Amogh Desai, and Aritra Basu :) Some thoughts on Utkarsh's proposal (and what him and I have been discussing offline): 1. I think we should start with enabling Hugo in the documentation build process for new releases 1. This may need to

Re: Airflow Docs Development Issues

2023-10-26 Thread utkarsh sharma
That sounds good, I'll start with creating smaller tickets for the above task, which I intend to do by the end of this week. Thanks, Utkarsh Sharma On Thu, Oct 26, 2023 at 4:16 PM Aritra Basu wrote: > Yup, sounds good to me let's go for it! > > -- > Regards, > Aritra Basu > > On Thu, Oct 26,

Re: [DISCUSS] Removing Qubole provider (and adding removal process)

2023-10-26 Thread Jarek Potiuk
> > > I suggest also removing it from pypi for security reasons. If there is a > security issue with it then the issue will remain with us. > > I am quite sure we still have to handle security issues if someone finds them. releasing such a provider will still be possible using the tag/branch and

Re: [DISCUSS] Removing Qubole provider (and adding removal process)

2023-10-26 Thread Andrey Anshin
I think in the case of Qubole it is pretty easy to remove it from the provider codebase. I'm pretty sure that almost no one even noticed this removal. Best Wishes *Andrey Anshin* On Thu, 26 Oct 2023 at 23:06, Bolke de Bruin wrote: > I suggest also removing it from pypi for security

Re: [DISCUSS] Removing Qubole provider (and adding removal process)

2023-10-26 Thread Bolke de Bruin
I suggest also removing it from pypi for security reasons. If there is a security issue with it then the issue will remain with us. B. Sent from my iPhone > On 26 Oct 2023, at 20:20, Jarek Potiuk wrote: > > Hello Airflow community, > > How do we feel about removing the Qubole provider

[DISCUSS] Removing Qubole provider (and adding removal process)

2023-10-26 Thread Jarek Potiuk
Hello Airflow community, How do we feel about removing the Qubole provider completely (leaving only old releases in PyPI? On September 1 2023 ( https://lists.apache.org/thread/p394d7w7gc7lz61g7qdthl96bc9kprxh) the Qubole operator ws suspended. Due to the reasons described in the thread (Qubole

Re: [PROPOSAL] (likely) significantly shorten CI test time: splitting to db/non-db tests

2023-10-26 Thread Jarek Potiuk
BTW. I also plan to add short "best practices" chapter to our TESTING.rst to deal with some of those "special" cases based on the learning from that whole experience. On Thu, Oct 26, 2023 at 3:46 PM Jarek Potiuk wrote: > All right. > > I think I am getting closer to getting it ready and there

Re: [PROPOSAL] (likely) significantly shorten CI test time: splitting to db/non-db tests

2023-10-26 Thread Jarek Potiuk
All right. I think I am getting closer to getting it ready and there are indeed significant savings it seems. I believe we can achieve some ~40/50% improvement overall for the CPU build time needed to run the tests and likely some 30% - 40% improvement in waiting for the build to complete (note

Re: [Discussion] Requesting Feedback on Airflow Survey

2023-10-26 Thread Amogh Desai
Thanks for sharing, Briana. I got a chance to go through the doc and added my comments directly in the document as suggestions. Summary: * I think we should also have questions around the operators being used by the community * We should ask questions regarding the security aspect too as Jarek

[RESULT] [VOTE] AIP-58 Airflow ObjectStore

2023-10-26 Thread Bolke de Bruin
Hello, AIP-58 is accepted. 4 +1 binding votes have been cast: Bolke Jarek Kaxil Hussein 4 +1 non binding Jens Avi Dennis Igor Vote thread: https://lists.apache.org/thread/wokt58k15g81cjnsytq9k1ofvspb4d5c The pr is almost done and I intend to finalize it by the end of this week. Please

Re: Airflow Docs Development Issues

2023-10-26 Thread Aritra Basu
Yup, sounds good to me let's go for it! -- Regards, Aritra Basu On Thu, Oct 26, 2023, 1:47 PM Amogh Desai wrote: > Go ahead Utkarsh. It would be nice to work with you along this. > > Thanks, > Amogh Desai > > On Wed, Oct 25, 2023 at 10:02 PM Jarek Potiuk wrote: > > > +1. I think no-one will

Re: Limiting (or errorring out) Airflow for Python 3.12 until our dependencies/we catch up

2023-10-26 Thread Jarek Potiuk
Unfortunately not - once release is done in PyPI we cannot do anything (except yanking the release which means "mark it as a buggy release so that it does not get installed by default". PyPI releases are "write-once-only" - basically immutable. When you delete a package, you cannot upload a new

Re: Airflow Docs Development Issues

2023-10-26 Thread Amogh Desai
Go ahead Utkarsh. It would be nice to work with you along this. Thanks, Amogh Desai On Wed, Oct 25, 2023 at 10:02 PM Jarek Potiuk wrote: > +1. I think no-one will object to improve the current situation :) > > On Wed, Oct 25, 2023 at 5:02 PM utkarsh sharma > wrote: > > > Hey everyone, > > > >

Re: Limiting (or errorring out) Airflow for Python 3.12 until our dependencies/we catch up

2023-10-26 Thread Amogh Desai
Hi Jarek, Thanks for this initiative. It is always a good idea to let the end users know that "oh the software does not work with your current set of dependencies, do to handle it" I like the idea of limiting the versions to < 3.12 and am for it. While we do this, we should not spend a lot