FYI:
---------- Forwarded message ---------- From: Qubole Team <fieldeve...@qubole.com> Date: Tue, Jul 10, 2018 at 4:12 PM Subject: [Live Webinar Tomorrow] AIRFlow at Scale - Register Now To: sum...@qubole.com Big Data Your Way Any Cloud. Any Engine. Any Scale Visit Qubole at Spark Summit <http://visit.qubole.com/P000K08QAP0dY8030roZja2> Hi Sumit, The Data Team at Qubole collects usage and telemetry data from a million machines a month. We run many complex ETL workflows to process this data and provide reports, insights and recommendations to customers, analysts and data scientists. We use open source distribution of Apache Airflow to orchestrate our ETL and process more than 1 terabyte of data daily. Attend *tomorrow's webinar (3-4 pm IST)*, powered by *Digital Vidya*, and learn about *how we have extended Apache Airflow to manage the operational inefficiencies that arise when you manage data pipelines in a multi-tenant environment*. We will also be talking about how we have made the data pipelines robust by adding data quality checks using CheckOperators. *Key Takeaways:* - How we manage deploys and upgrades of data pipelines in a multi-tenant environment - How we manage configuration for data pipelines in a multi-tenant environment - Data quality issues we faced with data ingestion/transformation - Enhancements we had to make to Check operators - Challenges faced in developing the alerting framework - Lesson learned and best practices in using Apache Airflow for data quality checks *Register Now! <http://visit.qubole.com/P000K08QAP0dY8030roZja2>* Best Regards, Qubole Team Register Now for the Webinar→ <http://visit.qubole.com/P000K08QAP0dY8030roZja2> [image: logo-qubole-lp.png] Who we are Data Science Optimization Qubole cluster auto-scaling and automated lifecycle management eliminate barriers to data science. Spark Notebooks in One Click Combine code execution, rich text, mathematics, plots, and rich media for self-service data exploration. Become Data Driven Give your team access to Machine Learning, ETL with Spark SQL, and real-time processing of streaming data. www.qubole.com [image: facebook] <http://visit.qubole.com/X8A0dpP0K000Z38ra02jQZ0> [image: LinkedIn] <http://visit.qubole.com/GjP0Z420rr80Q08A0000dKa> [image: Twitter] <http://visit.qubole.com/A40s08QP20K8jda00A010rZ> Qubole, 469 El Camino Real, Suite 205, Santa Clara, CA 95050 <https://maps.google.com/?q=469+El+Camino+Real,+Suite+205,+Santa+Clara,+CA+95050&entry=gmail&source=g> <http://visit.qubole.com/Y08840tKd00ar002APQ02Zj> This email was sent to sum...@qubole.com. If you no longer wish to receive these emails you may unsubscribe <http://visit.qubole.com/u/yj4QradK080AZP508w00020> at any time.
