Hello. I've prepared a draft PR with a new YDB provider: https://github.com/apache/airflow/pull/39996 . Any comments about the idea to add this provider are welcome. Comments about the code are better to put inside PR rather than this mailing thread I suppose=) I would create a LAZY VOTE in a while. Thank you in advance!
сб, 1 июн. 2024 г. в 13:23, Jarek Potiuk <ja...@potiuk.com>: > It looks good to me, sounds like well established, open-source - does not > seem to have too complex dependencies (all of them we already have) > https://github.com/ydb-platform/ydb-python-sdk . As long as we have > integration test for it, it should be pretty easy to maintain (see > https://github.com/apache/airflow/pull/39986 where Shahar improve > integration testing docs explaining how to do it step-by-step (cool) . > > J. > > On Wed, May 29, 2024 at 8:12 PM horror <uzhas...@gmail.com> wrote: > > > Hi Apache Airflow Community, > > > > I hope this message finds you well. > > > > I am writing to propose the addition of a new provider to Apache Airflow > > for YDB, an open-source distributed SQL database. As a versatile and > > scalable database, YDB offers a range of features that can greatly > benefit > > the Airflow ecosystem and its users. More information about YDB can be > > found on their official website: ydb.tech (https://ydb.tech/). > > > > YDB’s open-source nature aligns perfectly with the principles of > > collaboration and transparency that underpin the Apache Airflow project. > By > > integrating YDB as a provider in Apache Airflow, we can leverage its > robust > > capabilities to enhance workflow management for a variety of use cases. > > Here are some of the potential benefits: > > > > 1. Scalability and Performance: YDB is designed to handle large-scale > data > > loads efficiently. Its integration into Airflow would provide users with > a > > high-performance option for managing complex data workflows. > > 2. SQL Compatibility: YDB supports standard SQL queries, making it a > > familiar and accessible choice for users transitioning from other SQL > > databases. > > 3. Open-Source Community: As an open-source project, YDB benefits from > > continuous improvements and contributions from its user base. Integrating > > it into Airflow encourages further collaboration and shared innovations. > > 4. Enhanced Flexibility: Adding YDB as a provider would offer Airflow > users > > more flexibility in choosing the right database solution for their > specific > > requirements, thereby broadening the scope and appeal of Apache Airflow. > > > > Moreover, I would like to offer our commitment to not only write this YDB > > provider but also to maintain and support it over the long term. Ensuring > > the reliability and enhancement of the provider will be a priority for > us, > > and we are dedicated to contributing to the Apache Airflow community > > through this effort. > > > > The addition of a YDB provider will also foster a stronger connection > > between our communities, enabling knowledge exchange and potential > > collaborative developments. Open-source projects thrive on diversity and > > inclusivity, and this addition would support that dynamic ecosystem. > > > > I am willing to contribute to the development and integration effort to > > ensure a smooth and effective implementation. Please let me know if there > > are any specific guidelines or processes that I should follow to initiate > > this proposal. > > > > Thank you for considering this suggestion. I look forward to your > feedback > > and the possibility of enriching Apache Airflow with YDB support. > > > > Best Regards, > > Sergey > > >
