IF YOU WOULD LIKE TO SEE THE FORMATTED VERSION OF THE NEWSLETTER, PLEASE GO HERE <https://www.astronomer.io/community-newsletter/january-2024/>
Hey Team, This month in the Airflow Community, we rang in the New Year by introducing new programs, growing existing ones, and as always, maintaining a spirit of authenticity and inclusion. In January, we: Launched our first Airflow Monthly Virtual Town Hall— sign-up for the 2nd here < https://astronomer.zoom.us/meeting/register/tZAqdu6qqz8jGdPaafmMbwdXkrgdhUBfdnRP > Released Airflow 2.8.1 <https://pypi.org/project/apache-airflow/> Brought in a new PMC Member- Andrey Anshin < https://lists.apache.org/thread/hflys9r2ctkx4019c6osfgy6cn4k1b9t> Announced the venue for Airflow Summit 2024— The Westin St. Francis in San Francisco! <https://airflowsummit.org/> As always, please reach out with items for the next issue to: < briana.oky...@astronomer.io> Thanks for reading! AIRFLOW SUMMIT 2024 SEPTEMBER 10TH-12TH, SAN FRANCISCO The venue for Airflow Summit 2024 has been announced! On September 10th-12th, join fellow Airflow enthusiasts at The Westin St. Francis in San Francisco! <https://airflowsummit.org/> Speaker submissions are also open. If you are interested in speaking at Airflow Summit, please submit your proposed session. < https://sessionize.com/airflow-summit-2024/> AIRFLOW 2.8.0 Airflow 2.8 was released on December 18th and includes a host of significant enhancements and new features that will greatly benefit our community. - PyPI <https://pypi.org/project/apache-airflow/> - Release Notes <https://pypi.org/project/apache-airflow/> - Docs <https://airflow.apache.org/blog/airflow-2.8.1/> - Sources < https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-sources.html > - Docker <https://hub.docker.com/r/apache/airflow/tags/?page=1&name=2.8.1> - Constraints <https://github.com/apache/airflow/tree/constraints-2.8.1> PROVIDERS New versions of 59 Airflow Providers packages were released. For all source releases, PyPI packages and docs, see: - Sources < https://airflow.apache.org/docs/apache-airflow-providers/installing-from-sources > - PyPI < https://airflow.apache.org/docs/apache-airflow-providers/installing-from-pypi > - Docs <https://airflow.apache.org/docs/> Ecosystem Initial releases: Astronomer Cosmos 1.3.2 <https://pypi.org/project/astronomer-cosmos/> OpenLineage 1.8.0 < https://github.com/OpenLineage/OpenLineage/releases/tag/1.8.0> UPCOMING EVENTS 2/7 Webinar: Airflow Monthly Town-Hall < https://astronomer.zoom.us/meeting/register/tZAqdu6qqz8jGdPaafmMbwdXkrgdhUBfdnRP#/registration > 2/8 Webinar: DAG writing for data engineers and data scientists < https://www.astronomer.io/events/webinars/dag-writing-for-data-engineers-and-data-scientists/ > 2/13 Meetup in Bangalore: Bangalore Airflow Meetup at Cloudera HQ < https://www.meetup.com/bangalore-apache-airflow-meetup/events/298355258/> 2/28 Meetup in San Francisco: Bay Area Airflow Meetup at Astronomer HQ! < https://www.meetup.com/bay-area-apache-airflow-incubating-meetup/events/298808510/ > 2/29 Meetup in NYC: NYC Airflow Meetup at the NEW Astronomer HQ! < https://www.meetup.com/nyc-apache-airflow-meetup/events/298611497/> 3/7 Meetup in São Paulo: First São Paulo Airflow Meetup at iFood! < https://www.meetup.com/sao-paulo-apache-airflow-meetup-group/events/298283822/ > 9/10-12 Conference in San Francisco: Airflow Summit 2024 < https://airflowsummit.org/> RECENT EVENTS 1/11 Webinar: Airflow Town Hall < https://astronomer.zoom.us/meeting/register/tZUpdOusrTItG90cw69WtvO2tOKfZbxxbQqH#/registration > 1/11 Webinar: What’s new in Airflow 2.8 < https://www.astronomer.io/events/webinars/whats-new-in-airflow-2-8/> 1/18 Webinar: How to manage connections in Airflow < https://www.astronomer.io/events/webinars/how-to-manage-connections-in-airflow/ > 1/25 Workshop: Free Apache Airflow Certification Workshop < https://www.astronomer.io/events/airflow-certification-workshop-jan-2024/?utm_source=clearfeed&utm_medium=organicsocial&utm_campaign=certification > 1/31/24 Webinar: Snowpark Day | Where Python Developers Can Learn The Latest Innovations for AI/ML Workflows in Snowflake < https://www.snowflake.com/webinar/virtual-hands-on-labs/snowpark-day-where-python-developers-can-learn-the-latest-innovations-for-ai-ml-workflows-in-snowflake-2024-01-31/?utm_source=astronomer&utm_medium=partner&utm_campaign=na--en-customers&utm_content=-wb-snowpark-day-2024-01-31 > 2/3-4 Conference in Brussels: Data workflows: translating dbt to Apache Airflow & `New` Workflow Orchestrator in town: "Apache Airflow 2.x" < https://fosdem.org/2024/schedule/event/fosdem-2024-1651-data-workflows-translating-dbt-to-apache-airflow/> < https://fosdem.org/2024/schedule/event/fosdem-2024-1652--new-workflow-orchestrator-in-town-apache-airflow-2-x-/ > PR OF THE MONTH PR #36537 Standardize airflow build process and switch to Hatchling build backend < https://github.com/apache/airflow/pull/36537> By @potiuk <https://github.com/potiuk> Hats off to Jarek Potiuk for winning PR of the Month! According to Aritra Basu “it's been a huge effort and it makes massive improvements in our packaging. Great to see it make it into airflow.” COMMUNICATIONS DIGEST Airflow Evolution at Snap by Yuri Desyatnik, Zhengyi Liu, Han Gan, Nanxi Chen, Jun & Gao on Airflow Medium < https://medium.com/apache-airflow/airflow-evolution-at-snap-c988cdd95abd> Standardizing your Astro projects with Cookiecutter and Cruft by Bas Harenslak on the Astronomer Blog < https://www.astronomer.io/blog/standardizing-astro-projects-with-cookiecutter-and-cruft/ > Bridging Gaps in Airflow: Unveiling the Journey from User to Contributor by Chris Hronek on Airflow Medium < https://medium.com/apache-airflow/bridging-gaps-in-airflow-unveiling-the-journey-from-user-to-contributor-54d361c75191 > Best practices for orchestrating MLOps pipelines with Airflow on Astronomer Learn <https://docs.astronomer.io/learn/airflow-mlops> Revolutionizing Data Orchestration and MLOps with Apache Airflow and Astronomer at Chiper by Jhoan Stiven Ruiz Arias on the Astronomer Blog < https://www.astronomer.io/blog/revolutionizing-data-orchestration-and-mlops-with-apache-airflow-astronomer-at-chiper/ > Introducing the Astronomer Champions Program for Apache Airflow by Briana Okyere on the Astronomer Blog < https://www.astronomer.io/blog/introducing-the-astronomer-champions-program-for-apache-airflow/ > Secrets Management in Airflow: Tailoring Connections for Dev, QA, and Prod Environments by Chris Hronek on Airflow Medium < https://medium.com/apache-airflow/secrets-management-in-airflow-tailoring-connections-for-dev-qa-and-prod-environments-374df481d506 > How to share code between multiple Airflow projects on Astronomer Learn < https://docs.astronomer.io/learn/sharing-code-multiple-projects?utm_source=linkedin&utm_medium=organicsocial&utm_campaign=learn-guide > GCP Data Engineering Project: Building and Orchestrating an ETL Pipeline for Online Food Delivery Industry with Apache Beam and Apache Airflow by Jana on Google Cloud Community Medium < https://medium.com/google-cloud/%EF%B8%8Fgcp-data-engineering-project-building-and-orchestrating-an-etl-pipeline-for-online-food-delivery-0fc8c532be14 > Create a Microsoft Entra Workload ID connection in Airflow on Astronomer Learn <https://docs.astronomer.io/learn/connections/entra-workload-identity> DEVLIST DIGEST Votes: [RESULT][VOTE] Airflow Providers - release of December 31, 2023 < https://lists.apache.org/thread/j51lh4b9q5y46clrd2xkwookjn9kw8pc> December PR of the Month < https://lists.apache.org/thread/5sn1rz67yhbz3wor75yzm3qobqw0r3g3> [RESULT][VOTE] December 2023 PR of the Month < https://lists.apache.org/thread/td6r3mq330foorkxhmsxpxj6ysgg8r7x> Airflow Providers prepared on January 07, 2024 < https://lists.apache.org/thread/4358xdt9jwqqws41jw13l6yzbng5mgsq> [RESULT][VOTE] Airflow Providers - release of January 07, 2024 < https://lists.apache.org/thread/4s0glylmhvwxxp3mdrcpx6p4123xw94g> Add Qdrant as a supported provider < https://lists.apache.org/thread/qbc046wfzj9j63vpqo8lnzc1gxyw37h0> New Airflow Community Provider: Teradata < https://lists.apache.org/thread/l08gf31p89dk8v9md0cdoyqgcglfxwgw> What goes into the next release < https://lists.apache.org/thread/031nj949q6wyldn02wc3txbm0wlxxk9s> Release Airflow 2.8.1 from 2.8.1rc1 < https://lists.apache.org/thread/kzxforo7rjvgjk2o64b98bpcv2kfj7lz> [RESULT][VOTE] Release Airflow 2.8.1 from 2.8.1rc1 < https://lists.apache.org/thread/3qj6ospj793f7o2rmcf9pnsfh8s3ybw1> [RESULT][VOTE] New Airflow Community Provider: Teradata < https://lists.apache.org/thread/0bfmw7t547lncw962v2oj0pq33zqpon8> Accept AIP-60 (Standard URI representation for Airflow Datasets) < https://lists.apache.org/thread/ycy3vh2h23t792zgrystv5k39nk9086n> Adding an Airflow provider for Qdrant < https://lists.apache.org/thread/7h7gsgy2k8yx91l9l52yy1g93f1rjdqx> January 2024 PR of the Month < https://lists.apache.org/thread/6j3ym5thqhm692zcvfmv39l1hp6759jd> Airflow Providers prepared on January 22, 2024 < https://lists.apache.org/thread/xw8v138dgwpml10wqbj28jtrvpszoblr> Airflow Providers prepared on January 26, 2024 < https://lists.apache.org/thread/rm2yjycl8snxftqvmsrfnzd6drs6q5go> [RESULT][VOTE] Airflow Providers - release of January 26, 2024 < https://lists.apache.org/thread/j1bfjnlbcgnvlf5wkcpwpbo82p0h1kld> [RESULT][VOTE] January 2024 PR of the Month < https://lists.apache.org/thread/ww7zslkfytrpgdkyj4jp118d5ykwjdzy> Airflow Providers prepared on January 30, 2024 < https://lists.apache.org/thread/rdlrxdtwr3h031xdmhj168yw19fh7hko> Discussed: AIP-60 Standard URI representation for Airflow Datasets < https://lists.apache.org/thread/rf6c80ljjkml0l15h2jys7k713q3os1d> Adding support for Qdrant provider < https://lists.apache.org/thread/ojtvbs6bymvcjl25ojp1rnp24gqppf5z> Enhanced Multi-Tenant Dataset Management in Airflow: Potential First Steps < https://lists.apache.org/thread/q4sd09xymvlskxdgcb1s6gfsqwnvqs2o> Announced: Apache Airflow Providers prepared on December 31, 2023 are released < https://lists.apache.org/thread/g907r6m1zco6nxl8ypdhbbmk92jnb8z7> Looking for potential new security team members < https://lists.apache.org/thread/2zl72252tzy457d773t3htcbdwjr1o2m> Apache Airflow Newsletter | December 2023 < https://lists.apache.org/thread/w3tpjwyqjftkljgb4xkjyz8cczgvfht2> Apache Airflow Providers prepared on January 07, 2024 are released < https://lists.apache.org/thread/d9nqotshcw7bz8hqqr2k9syyfk2fo22m> Modern Package management / Virtual environment management with Hatch < https://lists.apache.org/thread/31mkddnxxy4fn2p1djfvjzmty8sjy12f> January'23 Airflow Town Hall Recording, Meeting Summary and Deck < https://lists.apache.org/thread/gxk5hm1cbxvo84smmcqp9m3omko325p5> New PMC member: Andrey Anshin (taragolis) < https://lists.apache.org/thread/hflys9r2ctkx4019c6osfgy6cn4k1b9t> AIP-61 - Hybrid Executors < https://lists.apache.org/thread/94sg7l4m3qjk4b3vfq3lr94oc5fs9q4j> Apache Airflow 2.8.1 Released < https://lists.apache.org/thread/j2hfptn2n1bkoq1gk78f45jd798556lh> Airflow Community: We have some good news to share! < https://lists.apache.org/thread/rszwn1k32vo2nmwdm4vz3r9nncckthry> Apache Airflow Providers prepared on January 26, 2024 are released < https://lists.apache.org/thread/x57hh8swwbgtwgwld8t59h4nfw5g7ttq> Proposed: let's update the community content on the landing pages < https://lists.apache.org/thread/om0xm04rr14jxokcxqzbh6frplq09bf5> Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling) < https://lists.apache.org/thread/gw76y041rln5117ydg3nxxr36osbqlxt> Adding MSGraphSDK Async Operator to Airflow < https://lists.apache.org/thread/j2l7rf013f61fdrz66k9y7j80t9rklto> Add A Code of Conduct for Slack and Meetups < https://lists.apache.org/thread/jy16tb09qoc7lxb03pdj3dfnmrsmjq87> IN-DEVELOPMENT AIP-44: Airflow Internal API < https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-44+Airflow+Internal+API > AIP-49: Open Telemetry Support < https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-49+OpenTelemetry+Support+for+Apache+Airflow > AIP-51 Removing Executor Coupling from Core Airflow < https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-51+Removing+Executor+Coupling+from+Core+Airflow > AIP-56: Extensible user management < https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-56+Extensible+user+management > AIP-57: Refactor SLA feature < https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-57+Refactor+SLA+Feature > AIP-58 Airflow ObjectStore (AS) < https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263430565> AIP-59: Performance test framework < https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-59+Performance+tests+framework > AIP-60: Standard URI representation < https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-60+Standard+URI+representation+for+Airflow+Datasets > See the Wiki for more of the project’s approved AIPs. < https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals > AIRFLOW FACT OF THE MONTH Did you know that the bumblebee bat is the world’s smallest mammal? Weighing in at 0.05 to 0.07 ounces, with a head-to-body length of 1.14 to 1.29 inches and a wingspan of 5.1 to 5.7 inches, the bumblebee bat—also known as Kitti’s hog-nosed bat—is the smallest mammal in the world, according to the Guinness Book of World Records. To see this tiny bat for yourself, you’d have to visit one of a select few limestone caves on the Khwae Noi River in Kanchanaburi Province of southwest Thailand. Source: Reader’s Digest <https://www.rd.com/list/interesting-facts/> STAY TUNED Please subscribe to this newsletter to ensure you always receive the latest issue: <http://eepurl.com/hXUA3r> Know of an upcoming event or publication the community should know about? Let me know at briana.oky...@astronomer.io. -- *Briana Okyere* *Community Manager* *Astronomer*
