Re: Thoughts on Adding Weaviate Provider?

2023-01-30 Thread Jarek Potiuk
Cool. Glad I could help. On Sun, Jan 29, 2023 at 10:12 PM Marcus Eagan wrote: > > J, > > Not discouraging at all. I reached out in search of clarity that I now have. > > Thanks for the detailed response. I think all of this makes a ton of sense. > Upon first sight of how the provider ecosystem h

Re: Thoughts on Adding Weaviate Provider?

2023-01-29 Thread Marcus Eagan
J, Not discouraging at all. I reached out in search of clarity that I now have. Thanks for the detailed response. I think all of this makes a ton of sense. Upon first sight of how the provider ecosystem had grown, I was shocked. I will talk with the Weaviate team and community about Airflow's pos

Re: Thoughts on Adding Weaviate Provider?

2023-01-29 Thread Jarek Potiuk
TL;DR; Generally we are very cautious about adding new "external service" providers to the community and it's highly unlikely we would want Weaviate in, but you are absolutely free (and encouraged) to release the provider on your own. Even if there is an open-source engine (like yours), when there

Thoughts on Adding Weaviate Provider?

2023-01-29 Thread Marcus Eagan
Hi Devs, In keeping with the open source ethos and the need for DAG workflows in Neural Search pipelines, I welcome feedback on the idea of adding a Weaviate provider to Airflow. It's the best open source neural search engine. I see the need for a test an