It looks good to me, sounds like well established, open-source - does not seem to have too complex dependencies (all of them we already have) https://github.com/ydb-platform/ydb-python-sdk . As long as we have integration test for it, it should be pretty easy to maintain (see https://github.com/apache/airflow/pull/39986 where Shahar improve integration testing docs explaining how to do it step-by-step (cool) .
J. On Wed, May 29, 2024 at 8:12 PM horror <[email protected]> wrote: > Hi Apache Airflow Community, > > I hope this message finds you well. > > I am writing to propose the addition of a new provider to Apache Airflow > for YDB, an open-source distributed SQL database. As a versatile and > scalable database, YDB offers a range of features that can greatly benefit > the Airflow ecosystem and its users. More information about YDB can be > found on their official website: ydb.tech (https://ydb.tech/). > > YDB’s open-source nature aligns perfectly with the principles of > collaboration and transparency that underpin the Apache Airflow project. By > integrating YDB as a provider in Apache Airflow, we can leverage its robust > capabilities to enhance workflow management for a variety of use cases. > Here are some of the potential benefits: > > 1. Scalability and Performance: YDB is designed to handle large-scale data > loads efficiently. Its integration into Airflow would provide users with a > high-performance option for managing complex data workflows. > 2. SQL Compatibility: YDB supports standard SQL queries, making it a > familiar and accessible choice for users transitioning from other SQL > databases. > 3. Open-Source Community: As an open-source project, YDB benefits from > continuous improvements and contributions from its user base. Integrating > it into Airflow encourages further collaboration and shared innovations. > 4. Enhanced Flexibility: Adding YDB as a provider would offer Airflow users > more flexibility in choosing the right database solution for their specific > requirements, thereby broadening the scope and appeal of Apache Airflow. > > Moreover, I would like to offer our commitment to not only write this YDB > provider but also to maintain and support it over the long term. Ensuring > the reliability and enhancement of the provider will be a priority for us, > and we are dedicated to contributing to the Apache Airflow community > through this effort. > > The addition of a YDB provider will also foster a stronger connection > between our communities, enabling knowledge exchange and potential > collaborative developments. Open-source projects thrive on diversity and > inclusivity, and this addition would support that dynamic ecosystem. > > I am willing to contribute to the development and integration effort to > ensure a smooth and effective implementation. Please let me know if there > are any specific guidelines or processes that I should follow to initiate > this proposal. > > Thank you for considering this suggestion. I look forward to your feedback > and the possibility of enriching Apache Airflow with YDB support. > > Best Regards, > Sergey >
