Airflow community news in April 2022 View this email in your browser (https://mailchi.mp/6c76b8b0780d/apache-airflow-newsletter-april-2022?e=9963ef1d25)
In April we released 2.2.5 and published a snapshot, 2.3.0beta1. In events news, the agenda for Airflow Summit 2022 is now available (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eae2a758d4&e=9963ef1d25) , featuring newly added speakers and in-person events. Check it out to see if a talk or watch party will be happening near you. The newsletter is moving to a new platform. Please subscribe to ensure you always receive the latest issue: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22e6f7e734&e=9963ef1d25. Notice anything different about the newsletter? Let us know what you think of the new design. Also, if there is other content you would like to see featured here, let us know that, too! The agenda (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6666157fb0&e=9963ef1d25) is now live! Scheduled talks include: * Ash Berlin-Taylor, “Dynamic DAGs – the New Horizon” * Jarek Potiuk & Mateusz Henc, “Multitenancy is Coming” * Maciej Obuchowski & Pawel Leszczynski, “OpenLineage and Airflow – Data Lineage has never been Easier” * Willy Lulciuc, “Automating Airflow Backfills with Marquez” * Daniel Imberman, “Introducing Astro Flow, the Next Generation of DAG Authoring” * Jed Cunningham, “Airflow/Kubernetes: Running on and Using k8s” * Evan Tahler & Marcos Marx, “`git push` Your Data Stack with Airbyte, Airflow and dbt” * Karthik Kadiyam, “Ingesting Game Telemetry in near Real-time Dynamically into Redshift with Airflow (WB Games)” * Rafael Pierre, “Automatic Speech Recognition at Scale Using Tensorflow, Kubernetes and Airflow” In-person events now include watch parties in India and Macedonia. For a complete list of local events and to register for the summit, visit the website (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=fb69c386c1&e=9963ef1d25) . On April 14, we published Airflow 2.3.0beta1. PyPI: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eda0654fec&e=9963ef1d25 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=84007ef2bd&e=9963ef1d25) Changelog: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b71e1b0b1f&e=9963ef1d25 Constraints: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a9cd94a3f2&e=9963ef1d25 This snapshot allows for build testing and early testing of 2.3.0. Jed Cunningham says, “Please test this beta and create GitHub issues wherever possible if you encounter bugs (use 2.3.0b1 in the version dropdown filter when creating the issue).” Added features include Dynamic Test Mapping (AIP-42). For more info, check out the dev docs (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=773f643d4d&e=9963ef1d25) . On April 4, we released Airflow 2.2.5. PyPI: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e64fcfa368&e=9963ef1d25 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c97a3052c1&e=9963ef1d25) Docs: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=83eb35bbb6&e=9963ef1d25 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=798eb075b0&e=9963ef1d25) Changelog: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=09e9b28a6c&e=9963ef1d25 Docker image: “docker pull apache/airflow:2.2.5” Constraints: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b01569a052&e=9963ef1d25 * April 20, “You Don’t Need an Orchestrator,” Benoit Pimpaud, https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1dbe13bdf7&e=9963ef1d25 * April 19, “Using Airflow as a Data Analyst”; Kenten Danas, Santona Tuli & Amir Zahreddine; https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=110c9b5b36&e=9963ef1d25 * April 5, “OpenLineage and Airflow, a Deeper Dive,” Ross Turk & Michael Collado, https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c6bba7021a&e=9963ef1d25 We can always use speakers for meetups. If you want to present your Airflow contribution, demo your project, or share your experience at a Community Meetup this year, let us know by filling out this form (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e5d7105aa9&e=9963ef1d25) . You could win some cool swag! * May 18: “Orchestrating Hybrid Workflows with Apache Airflow,” Richard Sueiras, QCon https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4f79fad7c2&e=9963ef1d25 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ae1cf4837f&e=9963ef1d25) * May 23-27: Airflow Summit 2022, https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b61046257b&e=9963ef1d25 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ee0fdfa984&e=9963ef1d25) * June 21: “Automating Airflow Backflows with Marquez,” Willy Lulciuc, Open Source Summit, https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=8728092cbe&e=9963ef1d25 Replace Breeze with Python-based implementation (#22880 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9edd88da47&e=9963ef1d25) ) Author: Jarek Potiuk (potiuk (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=638b718c47&e=9963ef1d25) ) This long-awaited change replaces the key functionality of the old Bash-based Breeze with a Python-based implementation. Planned: migration of maintainer tools and replacement of CI shell commands with Breeze equivalents. Kamil Bregula (mik-laj (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=36885cca5e&e=9963ef1d25) ) Where are you based? I live in Warsaw, Poland. Jarek Potiuk and the rest of the team are organizing the Airflow Summit here this year and I will definitely be there. What is your current role? I’m a Senior Software Engineer at Snowflake. I’m working on integrating the Snowflake platform with other solutions, and the experience in creating integrations for Apache Airflow is very helpful. I am also responsible for maintaining SnowflakeIO for Apache Beam. Why did you start contributing to the project? I started working on Apache Airflow because Polidea had partnered with a cloud provider that wanted us to add new integrations. I had free capacity and Python experience, so I was assigned to this project. What do you use it for? Honestly, I don't use Airflow in production. My experience in writing DAGs is limited to system tests used to test integration in Apache Airflow. When was your first PR and what was it? My first PR was created on Jan 14, 2019. This was a minor fix for URL format handling for connections. What was your latest? Recently, I have little time for contributions, because I only contribute as a hobby, not as part of my professional work. The last issue that I dealt with was improving the integration of Apache Airflow with Snowflake. What do you like about working on OSS projects? I liked the freedom I had on OSS work. As part of our work, we had general directions in which we wanted to go, but very often I dealt with topics important to the community. Every day something could happen that would cause me to abandon current tasks and deal with other more interesting or important work for the community. * Apache Airflow for Data Scientists (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e12505be51&e=9963ef1d25) by Julia Wrzosińska * 10 Best Practices for Modern Data Orchestration with Airflow (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9d09f3f804&e=9963ef1d25) by Steve Swoyer * Airflow and dbt, Hand in Hand (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22774f26a5&e=9963ef1d25) by Josh Fell, Sung Won Chung & Viraj Parekh * Data Discovery for ML Engineers (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d887c2009f&e=9963ef1d25) by German Osin * Data Lineage with Snowflake (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2e62b8cc3a&e=9963ef1d25) by Michael Robinson * Airflow at Scale... (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9230babb45&e=9963ef1d25) by Sven Balnojan The newsletter is moving to a new platform, Mailchimp. Please subscribe to ensure you always receive the latest issue: https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b2f7ee78b2&e=9963ef1d25. That’s all, Air-folks. See you in April! 👋 ============================================================ ** https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d621524a4&e=9963ef1d25 (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1c45ad63e1&e=9963ef1d25) ** Website (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=197b93cd55&e=9963ef1d25) ** GitHub (https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=fef3f39f09&e=9963ef1d25) Want to change how you receive these emails? You can ** update your preferences (https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=9963ef1d25&c=16fc4e9633) or ** unsubscribe from this list (https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=9963ef1d25&c=16fc4e9633) . This email was sent to [email protected] (mailto:[email protected]) why did I get this? (https://apache.us14.list-manage.com/about?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=9963ef1d25&c=16fc4e9633) unsubscribe from this list (https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=9963ef1d25&c=16fc4e9633) update subscription preferences (https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=9963ef1d25&c=16fc4e9633) Apache Airflow . 1000 N West Street . Suite 1200 . Wilmington, DE 19801 . USA Email Marketing Powered by Mailchimp http://www.mailchimp.com/email-referral/?utm_source=freemium_newsletter&utm_medium=email&utm_campaign=referral_marketing&aid=fe7ef7a8dbb32933f30a10466&afl=1
