This is fantastic! I love to see the testing and dashboards being invested in from the beginning, never gets old :)
It shows that you really took the time to read the past discussions and documentation we have. I won't be able to look at the code in detail until the new year, but so far this is a great submission and a +1 from me! Cheers, Niko ________________________________ From: K Mallam, Sunil <sunil.kmal...@teradata.com.INVALID> Sent: Sunday, December 24, 2023 7:36:02 AM To: dev@airflow.apache.org Cc: Tworkiewicz, Adam; Mishra, Harishankar; Chinthanippu, Satish Subject: RE: [EXTERNAL] [COURRIEL EXTERNE] [Discuss] New Airflow Community Provider: Teradata CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque. Thank you Jarek and Elad, we appreciate you recognizing the efforts we put in. Sunil K Mallam Staff Product Manager – 3rd Party Developer Tools [A black and orange logo Description automatically generated]<https://www.teradata.com/> 17095 Via Del Campo Ct. San Diego, CA 92127 teradata.com<https://www.teradata.com/> This e-mail is from Teradata Corporation and may contain information that is confidential or proprietary. If you are not the intended recipient, do not read, copy, or distribute the e-mail or any attachments. Instead, please notify the sender and delete the e-mail and any attachments. Thank you. Please consider the environment before printing. From: Elad Kalif <elad...@apache.org> Date: Sunday, 24 December 2023 at 12:30 PM To: dev@airflow.apache.org <dev@airflow.apache.org> Cc: Tworkiewicz, Adam <adam.tworkiew...@teradata.com>, Mishra, Harishankar <harishankar.mis...@teradata.com>, Chinthanippu, Satish <satish.chinthani...@teradata.com> Subject: [EXTERNAL] Re: [Discuss] New Airflow Community Provider: Teradata [CAUTION: External Email] Hi Sunil, Proposal looks very solid and I do see the value in having a Teradata provider. I agree with Jarek and I am +1 for it. On Fri, Dec 22, 2023 at 9:27 PM K Mallam, Sunil <sunil.kmal...@teradata.com.invalid> wrote: > Hi, > > > > I’m Sunil Mallam, a Staff Product Manager with Teradata, owning the > Third-Party Connectors and Integrations. > > > > Here’s our proposal to be a Community Provider, as we see Airflow being a > key integration for Teradata’s cloud customers in terms of *orchestrating > data pipelines and ML workflows *- > > > > *Why Teradata wants to be a community provider?* > > - Teradata has over 700 customers (most of them are from Fortune 1000 > list) who run enterprise workloads. > - Being on *Airflow’s documentation index provides* *better visibility > for our user*s and they can easily find Teradata’s provider package, > without Teradata having to put additional effort into promoting it. > - Also, our customers have confidence in the quality and stability of > the provider package if it’s managed and maintained by Teradata, while > being validated/approved by Airflow. > > > > *How is it a positive for the Airflow project/community and why should it > be accepted?* > > - We have a dedicated team to manage and maintain the provider package. > - Teradata will take responsibility for - > - end-to-end testing efforts, running system tests periodically in > their environment, and making the status available to Airflow. > - being available and responsive to any communication from the > community members. > - To support our dedication toward contributions, Teradata is a verified > adapter with dbt > > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.getdbt.com%2Fdocs%2Fsupported-data-platforms&data=05%7C02%7CSunil.KMallam%40teradata.com%7C2a3782d11fce4531fd9708dc044df8d8%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638389980147820074%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=frYiNGXU6w7mPhRY2PK7DM7Y%2Bzo9a4zYkrwHTITatxg%3D&reserved=0<https://docs.getdbt.com/docs/supported-data-platforms>> > and we’re > diligent in matching up to its releases (major and minor). > > > > *What is developed as part of the initial implementation (phase 1) and > what’s next (phase 2)?* > > *Phase 1 * > > 1. Building a Hook, Base Operator, and Transfer Operator (Teradata to > Teradata). > 2. Creating DAGs for a basic use case (data movement). > 3. Creating Tests and match Test Coverage. > 4. Create supporting documentation for users. > 5. Create a dashboard with system tests status history. > > > > *Phase 2* > > 1. Building a Transfer Operator (Cloud to Teradata and Teradata to > Cloud). > 2. Add SSL Support > 3. Enhancements to match Airflow releases. > 4. Enhancements to support Teradata VantageCloud use production use > cases. > 5. Support specific production use cases for Teradata Customers. > > > > *Initial Implementation Links* > > Implementation: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTeradata%2Fairflow%2Ftree%2Ftd_develop%2Fairflow%2Fproviders%2Fteradata&data=05%7C02%7CSunil.KMallam%40teradata.com%7C2a3782d11fce4531fd9708dc044df8d8%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638389980147820074%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=noCZSfUrWFPcksWvXe9QSu0xcdM0Y8vF8Etuq69Q2BM%3D&reserved=0<https://github.com/Teradata/airflow/tree/td_develop/airflow/providers/teradata> > > Documentation: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTeradata%2Fairflow%2Ftree%2Ftd_develop%2Fdocs%2Fapache-airflow-providers-teradata&data=05%7C02%7CSunil.KMallam%40teradata.com%7C2a3782d11fce4531fd9708dc044df8d8%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638389980147820074%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RKoEIa32dCmH3QMC6gOOy4vCQxH2NbOTZygtnJxNCz0%3D&reserved=0<https://github.com/Teradata/airflow/tree/td_develop/docs/apache-airflow-providers-teradata> > > Unit Tests: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTeradata%2Fairflow%2Ftree%2Ftd_develop%2Ftests%2Fproviders%2Fteradata&data=05%7C02%7CSunil.KMallam%40teradata.com%7C2a3782d11fce4531fd9708dc044df8d8%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638389980147820074%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HnLAKCB296kxJrP0D%2BBQwQKXDjVc8Nl%2B0ztXCWoxfi8%3D&reserved=0<https://github.com/Teradata/airflow/tree/td_develop/tests/providers/teradata> > > System Tests: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTeradata%2Fairflow%2Ftree%2Ftd_develop%2Ftests%2Fsystem%2Fproviders%2Fteradata&data=05%7C02%7CSunil.KMallam%40teradata.com%7C2a3782d11fce4531fd9708dc044df8d8%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638389980147820074%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KzUvjk5iulQ%2BkahYYOIReDq2HXSW%2FyCETZ9qB09BeGM%3D&reserved=0<https://github.com/Teradata/airflow/tree/td_develop/tests/system/providers/teradata> > > System Tests Dashboard: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fteradata.github.io%2Fairflow%2F&data=05%7C02%7CSunil.KMallam%40teradata.com%7C2a3782d11fce4531fd9708dc044df8d8%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638389980147820074%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5AtofepTW5Ck1y4jC0yHvIefKDXjYSrdC8iK8WQiXcM%3D&reserved=0<https://teradata.github.io/airflow/> > > > > > > Thanks, > > *Sunil K Mallam* > Staff Product Manager – 3rd Party Developer Tools > > [image: A black and orange logo Description automatically generated] > <https://www.teradata.com/> > > 17095 Via Del Campo Ct. > San Diego, CA 92127 > teradata.com <https://www.teradata.com/> > > This e-mail is from Teradata Corporation and may contain information that > is confidential or proprietary. If you are not the intended recipient, do > not read, copy, or distribute the e-mail or any attachments. Instead, > please notify the sender and delete the e-mail and any attachments. Thank > you. > > Please consider the environment before printing. > > > > > > >
