Hi,
I’m Sunil Mallam, a Staff Product Manager with Teradata, owning the Third-Party
Connectors and Integrations.
Here’s our proposal to be a Community Provider, as we see Airflow being a key
integration for Teradata’s cloud customers in terms of orchestrating data
pipelines and ML workflows -
Why Teradata wants to be a community provider?
* Teradata has over 700 customers (most of them are from Fortune 1000 list)
who run enterprise workloads.
* Being on Airflow’s documentation index provides better visibility for our
users and they can easily find Teradata’s provider package, without Teradata
having to put additional effort into promoting it.
* Also, our customers have confidence in the quality and stability of the
provider package if it’s managed and maintained by Teradata, while being
validated/approved by Airflow.
How is it a positive for the Airflow project/community and why should it be
accepted?
* We have a dedicated team to manage and maintain the provider package.
* Teradata will take responsibility for -
* end-to-end testing efforts, running system tests periodically in their
environment, and making the status available to Airflow.
* being available and responsive to any communication from the community
members.
* To support our dedication toward contributions, Teradata is a verified
adapter with dbt<https://docs.getdbt.com/docs/supported-data-platforms> and
we’re diligent in matching up to its releases (major and minor).
What is developed as part of the initial implementation (phase 1) and what’s
next (phase 2)?
Phase 1
1. Building a Hook, Base Operator, and Transfer Operator (Teradata to
Teradata).
2. Creating DAGs for a basic use case (data movement).
3. Creating Tests and match Test Coverage.
4. Create supporting documentation for users.
5. Create a dashboard with system tests status history.
Phase 2
1. Building a Transfer Operator (Cloud to Teradata and Teradata to Cloud).
2. Add SSL Support
3. Enhancements to match Airflow releases.
4. Enhancements to support Teradata VantageCloud use production use cases.
5. Support specific production use cases for Teradata Customers.
Initial Implementation Links
Implementation:
https://github.com/Teradata/airflow/tree/td_develop/airflow/providers/teradata
Documentation:
https://github.com/Teradata/airflow/tree/td_develop/docs/apache-airflow-providers-teradata
Unit Tests:
https://github.com/Teradata/airflow/tree/td_develop/tests/providers/teradata
System Tests:
https://github.com/Teradata/airflow/tree/td_develop/tests/system/providers/teradata
System Tests Dashboard: https://teradata.github.io/airflow/
Thanks,
Sunil K Mallam
Staff Product Manager – 3rd Party Developer Tools
[A black and orange logo Description automatically
generated]<https://www.teradata.com/>
17095 Via Del Campo Ct.
San Diego, CA 92127
teradata.com<https://www.teradata.com/>
This e-mail is from Teradata Corporation and may contain information that is
confidential or proprietary. If you are not the intended recipient, do not
read, copy, or distribute the e-mail or any attachments. Instead, please notify
the sender and delete the e-mail and any attachments. Thank you.
Please consider the environment before printing.
