Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

Thanks everyone for the discussion, I will create a voting thread and lazy consensus for the open-sources ones. Regards, Kaxil On Tue, 24 Oct 2023 at 18:19, Pankaj Koti wrote: > Hi Andrey, > > We discussed internally (at Astronomer) regarding your > suggestion on the PRs and we will

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

Hi Andrey, We discussed internally (at Astronomer) regarding your suggestion on the PRs and we will incorporate the suggestion in our PRs to remove the callable approach and include examples in the docs to achieve the same using TaskFlow and using hook methods in those. Thank you for the

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

+1 from me as well, this should enable a lot of new usecases -- Regards, Aritra Basu On Tue, Oct 24, 2023, 11:47 AM Amogh Desai wrote: > +1 from me as well. Looks like something that the dev community would be > interested in using and hence contributing as well! > > Thanks & Regards, > Amogh

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

+1 from me as well. Looks like something that the dev community would be interested in using and hence contributing as well! Thanks & Regards, Amogh Desai On Tue, Oct 24, 2023 at 2:37 AM utkarsh sharma wrote: > +1 From my side as well. Also, having a dashboard to publish system test > results

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

+1 From my side as well. Also, having a dashboard to publish system test results for the providers will ensure the expected working in the long run. Thanks, Utkarsh Sharma On Sun, Oct 22, 2023 at 4:44 PM Hussein Awala wrote: > +1 Many people will be interested in contributing to these

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

+1 Many people will be interested in contributing to these providers in the future; I believe maintaining them will not be a challenge. On Sat, Oct 21, 2023 at 4:25 PM Pierre Jeambrun wrote: > +1 from me as well. > > Indeed LLMs for Apache Airflow is a no brainer, this will open a lot of >

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

+1 from me as well. Indeed LLMs for Apache Airflow is a no brainer, this will open a lot of opportunities for our users. Having Astronomer supporting this initiative is great. I like the idea of system tests dashboard, especially for those that are not open source/easily testable without a paid

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

+1 from me I think the dashboard idea is great! On Thu, Oct 19, 2023 at 7:05 PM Andrey Anshin wrote: > Because 4 out 5 new providers have a draft PR I would like to raise a > question about which related to all new providers. Just to avoid the same > question in all PRs. > > Do we actually

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

Because 4 out 5 new providers have a draft PR I would like to raise a question about which related to all new providers. Just to avoid the same question in all PRs. Do we actually want to make new operators kindish of like "PythonOperator"? Maybe I miss some important thing and can't see why it

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

Note: Weaviate is also an open-source project with over 470k+ downloads last month . On Thu, 19 Oct 2023 at 12:40, Kaxil Naik wrote: > Absolutely, we will publish the results of test runs somewhere, we would

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

Absolutely, we will publish the results of test runs somewhere, we would probably start with dumping them in a publicly-accessible S3 bucket / Github issue and then move to a Dashboard. > Re 2) I think this is a great opportunity for Astronomers to take the > "3rd-party maintenance" role to

Re: [DISCUSSION] Add 5 new Providers to enable first-class LLMOps

I thought a bit about it, and I think the way we have "Astronomer" behind it, it checks all the boxes - providing that we will also have some (super simple) dashboard similar to the MWAA one https://aws-mwaa.github.io/open-source/system-tests/dashboard.html . From

[DISCUSSION] Add 5 new Providers to enable first-class LLMOps

Hey Everyone, As a follow-up to my Keynote talk, Building and deploying LLM applications with Apache Airflow , I am formally proposing the addition of these 5 providers to the Apache Airflow repo: - PgVector