Awesome!! Good work from the community, features are building like lego blocks :D!
Thanks & Regards, Amogh Desai On Fri, Feb 7, 2025 at 3:49β―PM Rahul Vats <rah.sharm...@gmail.com> wrote: > Awesome! We have started testing mapped tasks with Alpha2. > > Regards, > Rahul Vats > > On Fri, 7 Feb 2025 at 15:01, Ash Berlin-Taylor <a...@apache.org> wrote: > > > The big new part of AIP-72 (Task Execution Interface) that was included > in > > this Alpha was mapped tasks β so you should now be able to use and test > > mapped tasks (and task groups) in Celery and Local executor. > > > > -ash > > > > > On 6 Feb 2025, at 21:03, Vikram Koka <vik...@astronomer.io.INVALID> > > wrote: > > > > > > Awesome! > > > Great to see the consistent progress week over week. > > > > > > Thank you all for the wonderful teamwork, > > > > > > Vikram > > > > > > > > > On Thu, Feb 6, 2025 at 11:28β―AM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > >> wooohooo..... ! > > >> > > >> On Thu, Feb 6, 2025 at 8:20β―PM Utkarsh Sharma > > >> <utkarsh.sha...@astronomer.io.invalid> wrote: > > >> > > >>> Dear Airflow Community, > > >>> > > >>> I am thrilled to announce the availability of *Apache Airflow > > >> 3.0.0.alpha2* > > >>> for testing! Airflow 3.0 marks a significant milestone as the first > > major > > >>> release in over four years, introducing improvements that enhance > user > > >>> experience, task execution, and system scalability. > > >>> First, a few caveats: > > >>> > > >>> - This is an *alpha release*, so do not run it in production. It > may > > >>> contain significant issues, and you will likely need to reset your > > >>> database > > >>> between this and subsequent alpha, beta, or release candidate > > >>> versions. *(Consider > > >>> yourself warned!)* > > >>> - This release is intended for *Airflow developers and early > > adopters* > > >>> to test the build and start preparing for Airflow 3.0.0. This is > *not > > >> an > > >>> official release*βthat will happen when we create a release > candidate > > >>> and hold a vote. > > >>> - The *expected timeline* for the first release candidate is *the > > week > > >>> of 2025-03-31*, but we encourage early feedback to help stabilize > the > > >>> release. > > >>> > > >>> What's New in 3.0.0.alpha2? > > >>> > > >>> Apache Airflow 3.0.0 introduces significant enhancements and breaking > > >>> changes. > > >>> *Notable Features* *DAG Versioning & Bundles* > > >>> > > >>> - Airflow now tracks *DAG versions*, offering better visibility > into > > >>> historical DAG changes and execution states. > > >>> - *DAG Bundles* ensure tasks run with the correct code version, > even > > >> as > > >>> DAGs evolve. > > >>> > > >>> *Modern Web Application* > > >>> > > >>> - The UI has been *rebuilt using React* and a complete *API-driven > > >>> structure*, improving maintainability and extensibility. > > >>> - A new *React-based plugin system* supports custom widgets, > improved > > >>> workflow visibility, and integration with external tools. > > >>> > > >>> *Task Execution Interface* > > >>> > > >>> - Airflow 3.0 adopts a *client/server architecture*, decoupling > task > > >>> execution from the internal meta-database via API-based > interaction. > > >>> - This allows for *remote execution*, *multi-language support*, > > >> enhanced > > >>> security, and better dependency management. > > >>> - The *Edge Executor* enables seamless remote task execution > without > > >>> direct database connections. > > >>> > > >>> *Data Assets & Asset-Centric Syntax* > > >>> > > >>> - Airflow 3.0 enhances dataset management by introducing *Data > > >> Assets*, > > >>> expanding beyond tables and files to include ML models and more. > > >>> - Assets can be explicitly defined using the @asset decorator, > > >>> simplifying tracking and dependencies. > > >>> > > >>> *External Event-Driven Scheduling* > > >>> > > >>> - Airflow now supports *event-driven DAG triggers* from external > > >> sources > > >>> like message queues. > > >>> - This builds upon dataset scheduling and enhances integration with > > >> the > > >>> external data ecosystem. > > >>> > > >>> For a more comprehensive list of new features, see the 3.0.0.alpha2 > > >> release > > >>> notes > > >>> < > > >>> > > >> > > > https://github.com/apache/airflow/blob/3.0.0a2/RELEASE_NOTES.rst#airflow-300alpha2-2025-01-31 > > >>>> > > >>> . > > >>> > > >>> For a list of the breaking changes, visit: Apache Airflow Breaking > > >> Changes > > >>> <https://cwiki.apache.org/confluence/x/9pCMEw> > > >>> *Known Issues in 3.0.0.alpha2* *AIP-72 - Task Execution Interface* > > >>> > > >>> - *DB isolation in tasks* is not enforced yet. > > >>> - *KubernetesExecutor* does not yet use the Task Execution > Interface. > > >>> - The following do not work on *LocalExecutor* or *CeleryExecutor*: > > >>> - Skip-based tasks (e.g. branch or skip operators). > > >>> - Task callbacks. > > >>> - *OpenLineage listeners* are not yet functional in this alpha > > >> release. > > >>> > > >>> *AIP-38 - Modern Web Application* > > >>> > > >>> - The new UI has *limited functionality* at this time and is still > > >> being > > >>> enhanced until GA. > > >>> - *FastAPI API server* (including the new UI and public API) does > not > > >>> yet have authentication. > > >>> - UI performance may degrade with a *large number of DAGs* or *very > > >> big > > >>> DAGs*. > > >>> - Notable areas of functionality *not yet implemented* in the UI: > > >>> - DAG Versioning, Data Assets, Backfills, DAG Gantt View. > > >>> > > >>> *AIP-63 - DAG Versioning* > > >>> > > >>> - Callbacks are *not yet working* on the DAG Processor (e.g., > > >> DAG-level > > >>> callbacks). > > >>> - The concept of a separate --subdir in the DAG processor is being > > >>> replaced with *DAG Bundles*, but currently, the DAG processor > always > > >>> parses all bundles. > > >>> > > >>> *AIP-82 - Event-Based Scheduling* > > >>> > > >>> - Some triggers (e.g., *file-based triggers*) may lead to > *duplicate > > >> DAG > > >>> runs*. > > >>> > > >>> *Where to Get It?* > > >>> > > >>> The *alpha snapshot* is available at: Apache Airflow 3.0.0.alpha2 > > >>> Distribution < > https://dist.apache.org/repos/dist/dev/airflow/3.0.0a2/> > > >>> > > >>> - *Binary Python "sdist" snapshot:* > apache-airflow-3.0.0a2-bin.tar.gz > > >>> - *Binary Python wheel snapshot:* > > >>> apache_airflow-3.0.0a2-py3-none-any.whl > > >>> > > >>> This snapshot has *not been published to PyPI*. > > >>> > > >>> Also present are *alpha releases* for: > > >>> > > >>> - apache-airflow-task-sdk > > >>> - apache-airflow-providers-standard > > >>> - apache-airflow-providers-fab > > >>> - apache-airflow-providers-celery > > >>> > > >>> These *are also not published to PyPI*. > > >>> *Public Keys & Verification* > > >>> > > >>> Public keys for verification are available at: Apache Airflow KEYS > > >>> <https://www.apache.org/dist/airflow/KEYS> > > >>> *Installation Instructions* > > >>> > > >>> To build a virtualenv that installs this alpha release and required > > >>> packages: > > >>> > > >>> uv venv > > >>> > > >>> uv pip install --find-links > > >>> https://dist.apache.org/repos/dist/dev/airflow/3.0.0a2/ \ > > >>> apache-airflow==3.0.0a2 \ > > >>> apache-airflow-task-sdk==1.0.0a2 \ > > >>> apache-airflow-providers-standard==0.1.0a2 \ > > >>> apache-airflow-providers-fab==2.0.0a2 \ > > >>> apache-airflow-providers-celery==3.11.0a2 > > >>> > > >>> *Changes Youβll Need to Consider* > > >>> > > >>> - The *standalone DAG processor* is now required, start it with: > > >>> > > >>> airflow dag-processor > > >>> > > >>> - The new UI and public API start with: > > >>> > > >>> airflow fastapi-api > > >>> > > >>> UI is available at http://localhost:9091/webapp. > > >>> - Depending on your deployment setup, you may need to set the > > >> [workers] > > >>> execution_api_server_url config option. Default: > > >>> > > >>> http://localhost:9091/execution/ > > >>> > > >>> > > >>> *Get Involved* > > >>> > > >>> We encourage the community to *test this release* and report any > issues > > >> or > > >>> feedback. Your contributions help us ensure a *stable and reliable > > >> Airflow > > >>> 3.0.0 release*. > > >>> > > >>> A huge *thank you* to all the contributors who have worked on this > > >>> milestone release! > > >>> > > >>> *Best,* > > >>> Utkarsh and Jed > > >>> > > >> > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > >
