Let me know if there is a particular reason that this was scrapped? If I need to work around with my CFO on any pricing or other aspect I can push that to make it viable.
Just let me know the budget to get the approval from my CFO? From: Jarek Potiuk <[email protected]> Sent: Saturday, July 30, 2022 1:04 AM To: [email protected]; Michael Robinson <[email protected]> Cc: [email protected] Subject: Re: Apache Airflow Newsletter | July 2022 Nice! On Fri, Jul 29, 2022 at 9:32 PM Michael Robinson via users <[email protected]<mailto:[email protected]>> wrote: View this email in your browser<https://mailchi.mp/627a8910cc73/apache-airflow-newsletter-july-2022?e=ce16eef4ef> [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/940aaeeb-7d19-6946-b8e6-9c0d1f18a65c.png] [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/f09424c7-7b34-7252-53cb-e1ce49f98d4a.png] This month in the Airflow community, we released Airflow 2.3.3<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1bd27974f1&e=ce16eef4ef> and over 40 Airflow Providers packages<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aaec9074c7&e=ce16eef4ef>, including the new Common SQL shared provider and the new Tabular provider. We learned and networked at an Airflow Meetup in Tel Aviv<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=51d4c8cc3e&e=ce16eef4ef> and Google’s Airflow Day 2022<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2f955a2dfd&e=ce16eef4ef>, and over on the dev list we discussed Dynamic Yield’s “Magic Loop”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7cf95cdc1d&e=ce16eef4ef> for minimizing delays caused by many concurrent dynamic DAGs, among other topics. The newsletter is moving to a new platform, Mailchimp. Please subscribe<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=096e67c8c1&e=ce16eef4ef> to ensure you always receive the latest issue. [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/54e74767-a43b-478f-6d8b-4ccf389313cb.png] We can always use speakers for meetups. If you want to present your Airflow contribution, demo your project, or share your experience at a Community Meetup this year, let us know<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e7fc2b5028&e=ce16eef4ef>. You could win some excellent swag! * August 9: “Airflow 101: Essential Tips for Beginners”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=166096c917&e=ce16eef4ef> * August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great Expectations – with Interactivity”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7dbc8d0b3e&e=ce16eef4ef> * August 17: “Using Plugins for Webserver” with Faizan Qazi (details TBA on Clowder<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=88a3d50cba&e=ce16eef4ef>) [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/d3859580-85f5-1273-93e0-8e8ff1bdf4f8.png] We released Airflow 2.3.3. You can find the build artifacts using these links: * PyPI<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0d09446543&e=ce16eef4ef> | Notes<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e2c70b8cbd&e=ce16eef4ef> | Constraints<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=901d8a64b3&e=ce16eef4ef> * Docker image: apache/airflow:2.3.3 We released Airflow Providers packages. You can find them using these links: * Sources<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22c14f066e&e=ce16eef4ef> | Guide<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=63b31b2c5a&e=ce16eef4ef> [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/9ded1a56-059e-3e72-3e4e-844bc636848b.png] [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/c92ef16e-00a0-de14-2552-97dd9bb47832.png] @uranusjr <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5143e93b2b&e=ce16eef4ef> #24989 Implement expand_kwargs()<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=f4d747431b&e=ce16eef4ef> Implements a new ExpandInput type, "list-of-dicts", to support this new shape of mapped kwargs. To nominate a PR<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d9a79856a&e=ce16eef4ef> for a future issue, flag it by adding #protm to a comment. [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/7fc2ffcd-03ed-bcc8-b6d9-806670c45b90.png] If you missed any of these, you can catch a recap, watch on-demand, or dig into the slides: * July 28: “Apache Airflow TLV Meetup Summer Edition”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=48722ecbf9&e=ce16eef4ef> * July 27: “Airflow, Spark, Trinio and Data Engineering Best Practices”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa33f35f0&e=ce16eef4ef> * July 26: “Using Airflow with Tensorflow and MLFlow”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=31433f5b8d&e=ce16eef4ef> * July 15: “Data Engineering: Bulk Up Your Data Eng Skills using Airflow, Spark & DASK”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bc6157ff5d&e=ce16eef4ef> * July 14: “Apache Airflow – Future of Data Workflows”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b776db73d0&e=ce16eef4ef> * July 12: “Anatomy of an Operator”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4aa8403602&e=ce16eef4ef> * July 7: "Airflow Day 2022 on Google Open Source Live"<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=052c3897b0&e=ce16eef4ef> * July 6: “Introduction to Building Pipelines in GCP using Airflow”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ea6a2f2893&e=ce16eef4ef> [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/24fdcda5-566b-5d29-b4f0-ae039ef20802.png] * “Is Airflow the Right Choice for Machine Learning, Too?,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6213fd2ecd&e=ce16eef4ef> by Samhita Alla, BetterProgramming/Medium * “Deutsche Bank uses Cloud Composer Workload Automation,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9e495b03f8&e=ce16eef4ef>cnxtoday.com<http://cnxtoday.com/> * “Shopify’s Practical Guidelines from Running Airflow for ML and Data Workflows at Scale,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=143cd0347e&e=ce16eef4ef> by Reza Rahimi, infoq.com<http://infoq.com/> * “Magic Loop in Airflow – Reloaded,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c3257fbba3&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium * “Shared Volumes in Airflow – the Good, the Bad, and the Ugly,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=9046873bce&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium * “Running Airflow on ARM M1/M2? Hell Yes, but Upgrade to 2.3+,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a782285305&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium * “Generic Airflow Transfers Made Easy,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=eeb6b8d1cb&e=ce16eef4ef> by Jarek Potiuk, Apache Airflow/Medium * “Airflow’s Magic Loop,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cac6eafbf4&e=ce16eef4ef> by Itay Bittan, Towards Data Science/Medium * “Load Data from Postgres to BigQuery with Airflow,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7d83f3800a&e=ce16eef4ef> by Giorgos Myrianthous, Towards Data Science/Medium * “Automated Alerts for Airflow from Slack,”<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aa3832265c&e=ce16eef4ef> by Chris Young, Towards Data Science/Medium [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/a5ea0304-201d-0f05-c808-243b63e92a28.png] * Votes: * Airflow 2.3.3rc1 release (approved) * Airflow 2.3.3 release (approved) * Airflow Providers (RC1) (approved) * July’s PR of the Month (approved) * Release Airflow Python client 2.3.0 based on 2.3.0rc1 (pending) * Lazy consensus: * Deprecate “dependency detector” pluggability (passed) * Discussed: * “Magic Loop” as first-class citizen in Airflow * Airflow scheduling delay metric definition * Grouping of concerns * Migrate Airflow code to use PEP 563 (postponed evaluation of annotations) * Standard way of getting provider version? * Trimming down Airflow versions in issues * Deprecating the SLA feature? * Implementing a testing framework for Airflow * Consolidate DAG scheduling params * Proposed: * Provider’s mixed governance model * Simplification of WWW asset compilation for Breeze/dev env [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/7c616a13-898d-6342-a079-bd860959d6ac.png] * Airflow 2.4.0<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d0f3d39055&e=ce16eef4ef> This release will focus on new features. * Airflow Helm Chart 1.7.0<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cfff66719b&e=ce16eef4ef> The next release for the Airflow Helm Chart, fixing bugs and adding more features. For more info about upcoming releases and in-development projects, visit the current projects issue (#10176<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c64a96866&e=ce16eef4ef>). [https://mcusercontent.com/fe7ef7a8dbb32933f30a10466/images/671f8dfd-92e0-ed40-9b4c-1ace301b8c4f.png] The newsletter is moving to a new platform, Mailchimp. Please subscribe<https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05b5c570d8&e=ce16eef4ef> to ensure you always receive the latest issue. Is there an event or publication the community should know about? Let us know at [email protected]<mailto:[email protected]> or [email protected]<mailto:[email protected]>. Want to change how you receive these emails? You can update your preferences<https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4> or unsubscribe from this list<https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=ce16eef4ef&c=b4d78fe0f4>.
