Changing my vote to: +1 binding Forgot that common.io will be one of the dependency of Airflow similar to common.sql <https://github.com/apache/airflow/blob/9cfb1a82f3b00fa70a7b5ce6818a9e618512de63/setup.py#L829-L830>
On Sat, 11 Nov 2023 at 14:58, Kaxil Naik <kaxiln...@gmail.com> wrote: > Aah, completely forgot Airflow will actually depend on common.io -- that > makes sense > > Yes. The main thing there is that this is a chicken-egg problem. WHEN we > release 2.8.0 this provider MUST be already released (airflow will not > install otherwise). So we MUST release the common.io provider before > Airflow 2.8.0. Actually even long before - when we release alpha or beta, > common.io will have to be released already. otherwise it will be > impossible > to test alpha/beta etc. > > On Sat, 11 Nov 2023 at 14:57, Kaxil Naik <kaxiln...@gmail.com> wrote: > >> Yeah, I hadn't realized that 1.0 was already cut for common.io and the >> openlineage provider situation. >> >> We can discuss the Providers before Airflow release separately so as to >> not de-rail this vote. My main concern is the unreleased code can obviously >> change and if it is the first release of the providers, users won't be able >> to use it anyways. >> >> On Sat, 11 Nov 2023 at 14:32, Elad Kalif <elad...@apache.org> wrote: >> >>> Hi Kaxil >>> >>> We release providers when they are ready. This is no different than the >>> openlienage provider that we needed to release before 2.7 and no one >>> could >>> use it before 2.7 is out. >>> From providers release process point of view we are ready with AIP-58 - >>> thus all left is for core release manager to choose when to cut 2.8 >>> >>> On Sat, Nov 11, 2023 at 3:46 PM Kaxil Naik <kaxiln...@gmail.com> wrote: >>> >>> > Hmm interesting we already had 1.0 release for that provider so >>> releasing >>> > 1.0.1 is no different :shrug: >>> > >>> > I rescind my -1 (binding), to +0 (binding) --- because I don't see how >>> that >>> > provider is going to be used without Airflow 2.8 and the code is still >>> > getting worked out >>> > >>> > On Sat, 11 Nov 2023 at 13:44, Kaxil Naik <kaxiln...@gmail.com> wrote: >>> > >>> > > FYI: It contains Airflow 2.8 as requirement which is good but really >>> this >>> > > provider right now, doesn't make much sense as the code path can >>> change >>> > > until Airflow 2.8 is cut >>> > > >>> > > On Sat, 11 Nov 2023 at 13:42, Kaxil Naik <kaxiln...@gmail.com> >>> wrote: >>> > > >>> > >> -1 (binding) on common.io Provider >>> > >> >>> https://pypi.org/project/apache-airflow-providers-common-io/1.0.1rc1/ >>> > >> >>> > >> It isn't going to work with Airflow 2.7.*, it needs code that is >>> main >>> > >> (airflow.io.store.*) >>> > >> >>> > >> >>> > >> >>> > >> On Sat, 11 Nov 2023 at 12:57, Jarek Potiuk <ja...@potiuk.com> >>> wrote: >>> > >> >>> > >>> One thing to add to my +1. >>> > >>> >>> > >>> I have not mentioned it before, but I promised to the team >>> contributing >>> > >>> the new LLM providers to review the double check licences / >>> > dependencies >>> > >>> of >>> > >>> our new LLM providers. >>> > >>> >>> > >>> Just to confirm, it all looks good. >>> > >>> >>> > >>> We do not have a strict requirement to adhere to a permissive >>> licence >>> > for >>> > >>> providers, because they are "optional" components of Airflow but >>> we are >>> > >>> better than this: >>> > >>> >>> > >>> * All the direct libraries we use belong to "Category A" >>> > >>> https://www.apache.org/legal/resolved.html#category-a. T >>> > >>> * There are is one transitive dependency (*tqdm*) that is in >>> "Category >>> > B" >>> > >>> https://www.apache.org/legal/resolved.html#category-b -> this i >>> > >>> "weak-copyleft" but allowed by the ASF - Mozilla Public licence. >>> It has >>> > >>> proper attribution in the "openai" client, so we do not have to >>> mention >>> > >>> them (because they do). >>> > >>> >>> > >>> Also as a side comment: >>> > >>> >>> > >>> Currently our automated licence check only checks if our sources >>> have >>> > >>> appropriate licensing information. But in the future, when we >>> complete >>> > >>> SBOM >>> > >>> implementation (part of our Security Improvements work) - we will >>> be >>> > able >>> > >>> to automate those licence checks, because the tool we are using >>> > (cdxgen) >>> > >>> to >>> > >>> generate SBOM has also all the information about licences used by >>> the >>> > >>> dependencies of Airflow, so we will be able to verify compliance >>> with >>> > >>> licences in automated way for any new dependencies we will be >>> using. >>> > >>> >>> > >>> This is a nice side effect of the security-related improvements we >>> work >>> > >>> on >>> > >>> :). Something not immediately visible for users and even >>> contributors, >>> > >>> but >>> > >>> it puts very strong foundations to Airflow's stability and >>> "enterprise" >>> > >>> readiness - both security processes and licensing compliance being >>> > >>> expected >>> > >>> from Airflow by the enterprise users. >>> > >>> >>> > >>> J. >>> > >>> >>> > >>> On Fri, Nov 10, 2023 at 6:27 PM Aritra Basu < >>> aritrabasu1...@gmail.com> >>> > >>> wrote: >>> > >>> >>> > >>> > +1 (non binding) >>> > >>> > Tested out against some test dags >>> > >>> > >>> > >>> > -- >>> > >>> > Regards, >>> > >>> > Aritra Basu >>> > >>> > >>> > >>> > On Fri, 10 Nov 2023, 6:35 pm Pankaj Koti, < >>> pankaj.k...@astronomer.io >>> > >>> > .invalid> >>> > >>> > wrote: >>> > >>> > >>> > >>> > > +1 (non-binding) >>> > >>> > > >>> > >>> > > Concurring with Rahul Vats on the testing of the mentioned >>> > providers. >>> > >>> > > >>> > >>> > > A big thank you to Elad for skillfully overseeing and >>> synchronizing >>> > >>> this >>> > >>> > > release >>> > >>> > > with impeccable timing. We (Astronomer) truly value this >>> effort and >>> > >>> are >>> > >>> > > enthusiastic >>> > >>> > > about the establishment of a placeholder and initial space in >>> > >>> Airflow >>> > >>> > for >>> > >>> > > the new >>> > >>> > > LLM providers (Cohere, OpenAI, Pinecone, pgvector and >>> Weaviate). >>> > >>> > > We anticipate that this development will be advantageous for >>> the >>> > >>> > community >>> > >>> > > and >>> > >>> > > eagerly invite contributions from members of the Airflow + LLM >>> > >>> community >>> > >>> > to >>> > >>> > > enhance the robustness and feature completeness of these >>> providers. >>> > >>> > > >>> > >>> > > Best regards, >>> > >>> > > >>> > >>> > > *Pankaj Koti* >>> > >>> > > Senior Software Engineer (Airflow OSS Engineering team) >>> > >>> > > Location: Pune, Maharashtra, India >>> > >>> > > Timezone: Indian Standard Time (IST) >>> > >>> > > Phone: +91 9730079985 >>> > >>> > > >>> > >>> > > >>> > >>> > > On Fri, Nov 10, 2023 at 5:53 PM Hussein Awala < >>> huss...@awala.fr> >>> > >>> wrote: >>> > >>> > > >>> > >>> > > > +1 (binding) I checked the checksums, the signatures, the >>> > licences >>> > >>> and >>> > >>> > > the >>> > >>> > > > sources, all looks good. >>> > >>> > > > >>> > >>> > > > On Wed 8 Nov 2023 at 22:10, Elad Kalif <elad...@apache.org> >>> > wrote: >>> > >>> > > > >>> > >>> > > > > Hey all, >>> > >>> > > > > >>> > >>> > > > > I have just cut the new wave Airflow Providers packages. >>> This >>> > >>> email >>> > >>> > is >>> > >>> > > > > calling a vote on the release, >>> > >>> > > > > which will last for 72 hours - which means that it will >>> end on >>> > >>> > November >>> > >>> > > > 11, >>> > >>> > > > > 2023 21:10 PM UTC and until 3 binding +1 votes have been >>> > >>> received. >>> > >>> > > > > >>> > >>> > > > > >>> > >>> > > > > Consider this my (binding) +1. >>> > >>> > > > > >>> > >>> > > > > Airflow Providers are available at: >>> > >>> > > > > https://dist.apache.org/repos/dist/dev/airflow/providers/ >>> > >>> > > > > >>> > >>> > > > > *apache-airflow-providers-<PROVIDER>-*.tar.gz* are the >>> binary >>> > >>> > > > > Python "sdist" release - they are also official "sources" >>> for >>> > >>> the >>> > >>> > > > provider >>> > >>> > > > > packages. >>> > >>> > > > > >>> > >>> > > > > *apache_airflow_providers_<PROVIDER>-*.whl are the binary >>> > >>> > > > > Python "wheel" release. >>> > >>> > > > > >>> > >>> > > > > The test procedure for PMC members is described in >>> > >>> > > > > >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > >>> >>> > >>> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-pmc-members >>> > >>> > > > > >>> > >>> > > > > The test procedure for and Contributors who would like to >>> test >>> > >>> this >>> > >>> > RC >>> > >>> > > is >>> > >>> > > > > described in: >>> > >>> > > > > >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > >>> >>> > >>> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-contributors >>> > >>> > > > > >>> > >>> > > > > >>> > >>> > > > > Public keys are available at: >>> > >>> > > > > https://dist.apache.org/repos/dist/release/airflow/KEYS >>> > >>> > > > > >>> > >>> > > > > Please vote accordingly: >>> > >>> > > > > >>> > >>> > > > > [ ] +1 approve >>> > >>> > > > > [ ] +0 no opinion >>> > >>> > > > > [ ] -1 disapprove with the reason >>> > >>> > > > > >>> > >>> > > > > >>> > >>> > > > > Only votes from PMC members are binding, but members of the >>> > >>> community >>> > >>> > > are >>> > >>> > > > > encouraged to test the release and vote with >>> "(non-binding)". >>> > >>> > > > > >>> > >>> > > > > Please note that the version number excludes the 'rcX' >>> string. >>> > >>> > > > > This will allow us to rename the artifact without modifying >>> > >>> > > > > the artifact checksums when we actually release. >>> > >>> > > > > >>> > >>> > > > > The status of testing the providers by the community is >>> kept >>> > >>> here: >>> > >>> > > > > https://github.com/apache/airflow/issues/35540 >>> > >>> > > > > >>> > >>> > > > > You can find packages as well as detailed changelog >>> following >>> > the >>> > >>> > below >>> > >>> > > > > links: >>> > >>> > > > > >>> > >>> > > > > >>> > >>> >>> https://pypi.org/project/apache-airflow-providers-amazon/8.11.0rc1/ >>> > >>> > > > > >>> > >>> > > >>> > >>> >>> > >>> https://pypi.org/project/apache-airflow-providers-apache-spark/4.4.0rc1/ >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > >>> >>> > >>> https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/7.9.0rc1/ >>> > >>> > > > > >>> > >>> https://pypi.org/project/apache-airflow-providers-cohere/1.0.0rc1/ >>> > >>> > > > > >>> > >>> > >>> > https://pypi.org/project/apache-airflow-providers-common-io/1.0.1rc1/ >>> > >>> > > > > >>> > >>> > >>> > https://pypi.org/project/apache-airflow-providers-databricks/5.0.0rc1/ >>> > >>> > > > > >>> > >>> >>> https://pypi.org/project/apache-airflow-providers-discord/3.4.1rc1/ >>> > >>> > > > > >>> > >>> https://pypi.org/project/apache-airflow-providers-docker/3.8.1rc1/ >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > >>> >>> > >>> https://pypi.org/project/apache-airflow-providers-elasticsearch/5.1.1rc1/ >>> > >>> > > > > >>> > https://pypi.org/project/apache-airflow-providers-ftp/3.6.1rc1/ >>> > >>> > > > > >>> > >>> >>> https://pypi.org/project/apache-airflow-providers-google/10.11.1rc1/ >>> > >>> > > > > >>> > https://pypi.org/project/apache-airflow-providers-http/4.7.0rc1/ >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > >>> >>> > >>> https://pypi.org/project/apache-airflow-providers-microsoft-azure/8.2.0rc1/ >>> > >>> > > > > >>> > >>> https://pypi.org/project/apache-airflow-providers-openai/1.0.0rc1/ >>> > >>> > > > > >>> > >>> > > >>> > >>> >>> > >>> https://pypi.org/project/apache-airflow-providers-openlineage/1.2.1rc1/ >>> > >>> > > > > >>> > >>> >>> https://pypi.org/project/apache-airflow-providers-pgvector/1.0.0rc1/ >>> > >>> > > > > >>> > >>> >>> https://pypi.org/project/apache-airflow-providers-pinecone/1.0.0rc1/ >>> > >>> > > > > >>> > >>> >>> https://pypi.org/project/apache-airflow-providers-postgres/5.8.0rc1/ >>> > >>> > > > > >>> > >>> https://pypi.org/project/apache-airflow-providers-qubole/3.4.3rc1/ >>> > >>> > > > > >>> > >>> > >>> > https://pypi.org/project/apache-airflow-providers-salesforce/5.5.1rc1/ >>> > >>> > > > > >>> > >>> https://pypi.org/project/apache-airflow-providers-slack/8.4.0rc1/ >>> > >>> > > > > >>> > >>> > >>> > https://pypi.org/project/apache-airflow-providers-snowflake/5.1.1rc1/ >>> > >>> > > > > >>> > >>> >>> https://pypi.org/project/apache-airflow-providers-weaviate/1.0.0rc1/ >>> > >>> > > > > >>> > >>> https://pypi.org/project/apache-airflow-providers-yandex/3.6.0rc1/ >>> > >>> > > > > >>> > >>> > > > > Cheers, >>> > >>> > > > > Elad Kalif >>> > >>> > > > > >>> > >>> > > > >>> > >>> > > >>> > >>> > >>> > >>> >>> > >> >>> > >>> >>
