Re: [VOTE] AIP-58 Airflow ObjectStore

2023-10-23 Thread Bolke de Bruin
The foundational implementation is in core as is suggested in the AIP. The Operator(s) are in common.io to allow for faster iterations. The challenge with fsspec is not in fsspec itself which brings very little additional dependencies on its own. It is in s3fs which relies on aiobotocore which rel

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

2023-10-23 Thread Amogh Desai
+1 from me as well. Looks like something that the dev community would be interested in using and hence contributing as well! Thanks & Regards, Amogh Desai On Tue, Oct 24, 2023 at 2:37 AM utkarsh sharma wrote: > +1 From my side as well. Also, having a dashboard to publish system test > results f

Re: Airflow Docs Development Issues

2023-10-23 Thread Amogh Desai
+1 (non binding) from me on the thought of moving the older docs (~18 months seems ok) to an archive instead of the repository. Coming to the other problem of copying the built docs into airflow-site for releases, maybe we can fix that using a script? Open for thoughts here. I would be very happy

Re: [VOTE] AIP-58 Airflow ObjectStore

2023-10-23 Thread Amogh Desai
Thanks for bringing this up, Bolke. I generally like the idea of having AS and I like where the discussions here are going. Just one qn I have regarding where this will fit into the wider ecosystem is that, should we integrate this into core rather than a provider? Meaning, it makes more sense to

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

2023-10-23 Thread utkarsh sharma
+1 From my side as well. Also, having a dashboard to publish system test results for the providers will ensure the expected working in the long run. Thanks, Utkarsh Sharma On Sun, Oct 22, 2023 at 4:44 PM Hussein Awala wrote: > +1 Many people will be interested in contributing to these providers

Keep Mssql support

2023-10-23 Thread agateaaa
Hi All: Mssql support was voted to be dropped. https://lists.apache.org/thread/r06j306hldg03g2my1pd4nyjxg78b3h4 One of our product requirements is that we can only use the Mssql database. The product that uses airflow is installed with a suite of 8-10 other products that all use Mssql database as

RE: Limiting (or errorring out) Airflow for Python 3.12 until our dependencies/we catch up

2023-10-23 Thread Damian Shaw
I agree, has there been a new release of Python that has worked with Airflow without at least some fixing? I understand that Airflow is both a library and an application and I do agree with not putting upper bounds on libraries unless required, but it seems like allows an arbitrary upper bound

Re: Limiting (or errorring out) Airflow for Python 3.12 until our dependencies/we catch up

2023-10-23 Thread Pierre Jeambrun
I think that limiting to <3.12 makes sense. 2.7.2 is already out so I'm not sure we can do anything for users trying to install 2.7.2 on python 3.12. I believe there is no such thing as a python minor that is out of the box working well for airflow. It seems that we always need extra efforts to br

Re: Limiting (or errorring out) Airflow for Python 3.12 until our dependencies/we catch up

2023-10-23 Thread Aritra Basu
I think I'm on the side of giving an error message saying 3.12 is not yet supported in 2.7.3, I would assume anyone seeing that would understand the implication that neither does 2.7.2 and thus they wouldn't try installing it. Though I would also think they would have the same understanding that i

Limiting (or errorring out) Airflow for Python 3.12 until our dependencies/we catch up

2023-10-23 Thread Jarek Potiuk
Hey everyone, I've opened a PR https://github.com/apache/airflow/pull/35123 to limit Airflow to Python < 3.12 though I am not sure if this is the best idea so I seek devlist wisdom to decide whether we should do this, or maybe something else like allowing airflow to be installed but produce a cle