+1 non binding On Sat, 11 Nov 2023, 20:30 Kaxil Naik, <kaxiln...@gmail.com> wrote:
> Changing my vote to: +1 binding > > Forgot that common.io will be one of the dependency of Airflow similar to > common.sql > < > https://github.com/apache/airflow/blob/9cfb1a82f3b00fa70a7b5ce6818a9e618512de63/setup.py#L829-L830 > > > > > > On Sat, 11 Nov 2023 at 14:58, Kaxil Naik <kaxiln...@gmail.com> wrote: > > > Aah, completely forgot Airflow will actually depend on common.io -- that > > makes sense > > > > Yes. The main thing there is that this is a chicken-egg problem. WHEN we > > release 2.8.0 this provider MUST be already released (airflow will not > > install otherwise). So we MUST release the common.io provider before > > Airflow 2.8.0. Actually even long before - when we release alpha or > beta, > > common.io will have to be released already. otherwise it will be > > impossible > > to test alpha/beta etc. > > > > On Sat, 11 Nov 2023 at 14:57, Kaxil Naik <kaxiln...@gmail.com> wrote: > > > >> Yeah, I hadn't realized that 1.0 was already cut for common.io and the > >> openlineage provider situation. > >> > >> We can discuss the Providers before Airflow release separately so as to > >> not de-rail this vote. My main concern is the unreleased code can > obviously > >> change and if it is the first release of the providers, users won't be > able > >> to use it anyways. > >> > >> On Sat, 11 Nov 2023 at 14:32, Elad Kalif <elad...@apache.org> wrote: > >> > >>> Hi Kaxil > >>> > >>> We release providers when they are ready. This is no different than the > >>> openlienage provider that we needed to release before 2.7 and no one > >>> could > >>> use it before 2.7 is out. > >>> From providers release process point of view we are ready with AIP-58 - > >>> thus all left is for core release manager to choose when to cut 2.8 > >>> > >>> On Sat, Nov 11, 2023 at 3:46 PM Kaxil Naik <kaxiln...@gmail.com> > wrote: > >>> > >>> > Hmm interesting we already had 1.0 release for that provider so > >>> releasing > >>> > 1.0.1 is no different :shrug: > >>> > > >>> > I rescind my -1 (binding), to +0 (binding) --- because I don't see > how > >>> that > >>> > provider is going to be used without Airflow 2.8 and the code is > still > >>> > getting worked out > >>> > > >>> > On Sat, 11 Nov 2023 at 13:44, Kaxil Naik <kaxiln...@gmail.com> > wrote: > >>> > > >>> > > FYI: It contains Airflow 2.8 as requirement which is good but > really > >>> this > >>> > > provider right now, doesn't make much sense as the code path can > >>> change > >>> > > until Airflow 2.8 is cut > >>> > > > >>> > > On Sat, 11 Nov 2023 at 13:42, Kaxil Naik <kaxiln...@gmail.com> > >>> wrote: > >>> > > > >>> > >> -1 (binding) on common.io Provider > >>> > >> > >>> https://pypi.org/project/apache-airflow-providers-common-io/1.0.1rc1/ > >>> > >> > >>> > >> It isn't going to work with Airflow 2.7.*, it needs code that is > >>> main > >>> > >> (airflow.io.store.*) > >>> > >> > >>> > >> > >>> > >> > >>> > >> On Sat, 11 Nov 2023 at 12:57, Jarek Potiuk <ja...@potiuk.com> > >>> wrote: > >>> > >> > >>> > >>> One thing to add to my +1. > >>> > >>> > >>> > >>> I have not mentioned it before, but I promised to the team > >>> contributing > >>> > >>> the new LLM providers to review the double check licences / > >>> > dependencies > >>> > >>> of > >>> > >>> our new LLM providers. > >>> > >>> > >>> > >>> Just to confirm, it all looks good. > >>> > >>> > >>> > >>> We do not have a strict requirement to adhere to a permissive > >>> licence > >>> > for > >>> > >>> providers, because they are "optional" components of Airflow but > >>> we are > >>> > >>> better than this: > >>> > >>> > >>> > >>> * All the direct libraries we use belong to "Category A" > >>> > >>> https://www.apache.org/legal/resolved.html#category-a. T > >>> > >>> * There are is one transitive dependency (*tqdm*) that is in > >>> "Category > >>> > B" > >>> > >>> https://www.apache.org/legal/resolved.html#category-b -> this i > >>> > >>> "weak-copyleft" but allowed by the ASF - Mozilla Public licence. > >>> It has > >>> > >>> proper attribution in the "openai" client, so we do not have to > >>> mention > >>> > >>> them (because they do). > >>> > >>> > >>> > >>> Also as a side comment: > >>> > >>> > >>> > >>> Currently our automated licence check only checks if our sources > >>> have > >>> > >>> appropriate licensing information. But in the future, when we > >>> complete > >>> > >>> SBOM > >>> > >>> implementation (part of our Security Improvements work) - we will > >>> be > >>> > able > >>> > >>> to automate those licence checks, because the tool we are using > >>> > (cdxgen) > >>> > >>> to > >>> > >>> generate SBOM has also all the information about licences used by > >>> the > >>> > >>> dependencies of Airflow, so we will be able to verify compliance > >>> with > >>> > >>> licences in automated way for any new dependencies we will be > >>> using. > >>> > >>> > >>> > >>> This is a nice side effect of the security-related improvements > we > >>> work > >>> > >>> on > >>> > >>> :). Something not immediately visible for users and even > >>> contributors, > >>> > >>> but > >>> > >>> it puts very strong foundations to Airflow's stability and > >>> "enterprise" > >>> > >>> readiness - both security processes and licensing compliance > being > >>> > >>> expected > >>> > >>> from Airflow by the enterprise users. > >>> > >>> > >>> > >>> J. > >>> > >>> > >>> > >>> On Fri, Nov 10, 2023 at 6:27 PM Aritra Basu < > >>> aritrabasu1...@gmail.com> > >>> > >>> wrote: > >>> > >>> > >>> > >>> > +1 (non binding) > >>> > >>> > Tested out against some test dags > >>> > >>> > > >>> > >>> > -- > >>> > >>> > Regards, > >>> > >>> > Aritra Basu > >>> > >>> > > >>> > >>> > On Fri, 10 Nov 2023, 6:35 pm Pankaj Koti, < > >>> pankaj.k...@astronomer.io > >>> > >>> > .invalid> > >>> > >>> > wrote: > >>> > >>> > > >>> > >>> > > +1 (non-binding) > >>> > >>> > > > >>> > >>> > > Concurring with Rahul Vats on the testing of the mentioned > >>> > providers. > >>> > >>> > > > >>> > >>> > > A big thank you to Elad for skillfully overseeing and > >>> synchronizing > >>> > >>> this > >>> > >>> > > release > >>> > >>> > > with impeccable timing. We (Astronomer) truly value this > >>> effort and > >>> > >>> are > >>> > >>> > > enthusiastic > >>> > >>> > > about the establishment of a placeholder and initial space in > >>> > >>> Airflow > >>> > >>> > for > >>> > >>> > > the new > >>> > >>> > > LLM providers (Cohere, OpenAI, Pinecone, pgvector and > >>> Weaviate). > >>> > >>> > > We anticipate that this development will be advantageous for > >>> the > >>> > >>> > community > >>> > >>> > > and > >>> > >>> > > eagerly invite contributions from members of the Airflow + > LLM > >>> > >>> community > >>> > >>> > to > >>> > >>> > > enhance the robustness and feature completeness of these > >>> providers. > >>> > >>> > > > >>> > >>> > > Best regards, > >>> > >>> > > > >>> > >>> > > *Pankaj Koti* > >>> > >>> > > Senior Software Engineer (Airflow OSS Engineering team) > >>> > >>> > > Location: Pune, Maharashtra, India > >>> > >>> > > Timezone: Indian Standard Time (IST) > >>> > >>> > > Phone: +91 9730079985 > >>> > >>> > > > >>> > >>> > > > >>> > >>> > > On Fri, Nov 10, 2023 at 5:53 PM Hussein Awala < > >>> huss...@awala.fr> > >>> > >>> wrote: > >>> > >>> > > > >>> > >>> > > > +1 (binding) I checked the checksums, the signatures, the > >>> > licences > >>> > >>> and > >>> > >>> > > the > >>> > >>> > > > sources, all looks good. > >>> > >>> > > > > >>> > >>> > > > On Wed 8 Nov 2023 at 22:10, Elad Kalif <elad...@apache.org > > > >>> > wrote: > >>> > >>> > > > > >>> > >>> > > > > Hey all, > >>> > >>> > > > > > >>> > >>> > > > > I have just cut the new wave Airflow Providers packages. > >>> This > >>> > >>> email > >>> > >>> > is > >>> > >>> > > > > calling a vote on the release, > >>> > >>> > > > > which will last for 72 hours - which means that it will > >>> end on > >>> > >>> > November > >>> > >>> > > > 11, > >>> > >>> > > > > 2023 21:10 PM UTC and until 3 binding +1 votes have been > >>> > >>> received. > >>> > >>> > > > > > >>> > >>> > > > > > >>> > >>> > > > > Consider this my (binding) +1. > >>> > >>> > > > > > >>> > >>> > > > > Airflow Providers are available at: > >>> > >>> > > > > > https://dist.apache.org/repos/dist/dev/airflow/providers/ > >>> > >>> > > > > > >>> > >>> > > > > *apache-airflow-providers-<PROVIDER>-*.tar.gz* are the > >>> binary > >>> > >>> > > > > Python "sdist" release - they are also official > "sources" > >>> for > >>> > >>> the > >>> > >>> > > > provider > >>> > >>> > > > > packages. > >>> > >>> > > > > > >>> > >>> > > > > *apache_airflow_providers_<PROVIDER>-*.whl are the binary > >>> > >>> > > > > Python "wheel" release. > >>> > >>> > > > > > >>> > >>> > > > > The test procedure for PMC members is described in > >>> > >>> > > > > > >>> > >>> > > > > > >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > > >>> > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-pmc-members > >>> > >>> > > > > > >>> > >>> > > > > The test procedure for and Contributors who would like to > >>> test > >>> > >>> this > >>> > >>> > RC > >>> > >>> > > is > >>> > >>> > > > > described in: > >>> > >>> > > > > > >>> > >>> > > > > > >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > > >>> > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-contributors > >>> > >>> > > > > > >>> > >>> > > > > > >>> > >>> > > > > Public keys are available at: > >>> > >>> > > > > https://dist.apache.org/repos/dist/release/airflow/KEYS > >>> > >>> > > > > > >>> > >>> > > > > Please vote accordingly: > >>> > >>> > > > > > >>> > >>> > > > > [ ] +1 approve > >>> > >>> > > > > [ ] +0 no opinion > >>> > >>> > > > > [ ] -1 disapprove with the reason > >>> > >>> > > > > > >>> > >>> > > > > > >>> > >>> > > > > Only votes from PMC members are binding, but members of > the > >>> > >>> community > >>> > >>> > > are > >>> > >>> > > > > encouraged to test the release and vote with > >>> "(non-binding)". > >>> > >>> > > > > > >>> > >>> > > > > Please note that the version number excludes the 'rcX' > >>> string. > >>> > >>> > > > > This will allow us to rename the artifact without > modifying > >>> > >>> > > > > the artifact checksums when we actually release. > >>> > >>> > > > > > >>> > >>> > > > > The status of testing the providers by the community is > >>> kept > >>> > >>> here: > >>> > >>> > > > > https://github.com/apache/airflow/issues/35540 > >>> > >>> > > > > > >>> > >>> > > > > You can find packages as well as detailed changelog > >>> following > >>> > the > >>> > >>> > below > >>> > >>> > > > > links: > >>> > >>> > > > > > >>> > >>> > > > > > >>> > >>> > >>> https://pypi.org/project/apache-airflow-providers-amazon/8.11.0rc1/ > >>> > >>> > > > > > >>> > >>> > > > >>> > >>> > >>> > > >>> > https://pypi.org/project/apache-airflow-providers-apache-spark/4.4.0rc1/ > >>> > >>> > > > > > >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > > >>> > https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/7.9.0rc1/ > >>> > >>> > > > > > >>> > >>> > https://pypi.org/project/apache-airflow-providers-cohere/1.0.0rc1/ > >>> > >>> > > > > > >>> > >>> > > >>> > > https://pypi.org/project/apache-airflow-providers-common-io/1.0.1rc1/ > >>> > >>> > > > > > >>> > >>> > > >>> > > https://pypi.org/project/apache-airflow-providers-databricks/5.0.0rc1/ > >>> > >>> > > > > > >>> > >>> > >>> https://pypi.org/project/apache-airflow-providers-discord/3.4.1rc1/ > >>> > >>> > > > > > >>> > >>> > https://pypi.org/project/apache-airflow-providers-docker/3.8.1rc1/ > >>> > >>> > > > > > >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > > >>> > https://pypi.org/project/apache-airflow-providers-elasticsearch/5.1.1rc1/ > >>> > >>> > > > > > >>> > https://pypi.org/project/apache-airflow-providers-ftp/3.6.1rc1/ > >>> > >>> > > > > > >>> > >>> > >>> https://pypi.org/project/apache-airflow-providers-google/10.11.1rc1/ > >>> > >>> > > > > > >>> > https://pypi.org/project/apache-airflow-providers-http/4.7.0rc1/ > >>> > >>> > > > > > >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > > >>> > https://pypi.org/project/apache-airflow-providers-microsoft-azure/8.2.0rc1/ > >>> > >>> > > > > > >>> > >>> > https://pypi.org/project/apache-airflow-providers-openai/1.0.0rc1/ > >>> > >>> > > > > > >>> > >>> > > > >>> > >>> > >>> > > >>> > https://pypi.org/project/apache-airflow-providers-openlineage/1.2.1rc1/ > >>> > >>> > > > > > >>> > >>> > >>> https://pypi.org/project/apache-airflow-providers-pgvector/1.0.0rc1/ > >>> > >>> > > > > > >>> > >>> > >>> https://pypi.org/project/apache-airflow-providers-pinecone/1.0.0rc1/ > >>> > >>> > > > > > >>> > >>> > >>> https://pypi.org/project/apache-airflow-providers-postgres/5.8.0rc1/ > >>> > >>> > > > > > >>> > >>> > https://pypi.org/project/apache-airflow-providers-qubole/3.4.3rc1/ > >>> > >>> > > > > > >>> > >>> > > >>> > > https://pypi.org/project/apache-airflow-providers-salesforce/5.5.1rc1/ > >>> > >>> > > > > > >>> > >>> > https://pypi.org/project/apache-airflow-providers-slack/8.4.0rc1/ > >>> > >>> > > > > > >>> > >>> > > >>> > > https://pypi.org/project/apache-airflow-providers-snowflake/5.1.1rc1/ > >>> > >>> > > > > > >>> > >>> > >>> https://pypi.org/project/apache-airflow-providers-weaviate/1.0.0rc1/ > >>> > >>> > > > > > >>> > >>> > https://pypi.org/project/apache-airflow-providers-yandex/3.6.0rc1/ > >>> > >>> > > > > > >>> > >>> > > > > Cheers, > >>> > >>> > > > > Elad Kalif > >>> > >>> > > > > > >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > >> > >>> > > >>> > >> >
