o-nikolas commented on code in PR #48223:
URL: https://github.com/apache/airflow/pull/48223#discussion_r2021431464
##########
airflow-core/docs/extra-packages-ref.rst:
##########
@@ -18,20 +18,61 @@
Reference for package extras
''''''''''''''''''''''''''''
-Airflow has a number of optional "extras" that you can use to add features to
your installation when you
-are installing Airflow. Those extras are a good way for the users to manage
their installation, but also
-they are useful for contributors to airflow when they want to contribute some
of the features - including
-optional integrations of Airflow - via providers.
-
-.. warning::
-
- Traditionally in Airflow some of the extras used `.` and `_` to separate
the parts of the extra name.
- This was not PEP-685 normalized name and we opted to change it to to `-`
for all our extras, Expecting that
- PEP-685 will be implemented in full by `pip` and other tools we change all
our extras to use `-` as
- separator even if in some cases it will introduce warnings (the warnings
are harmless). This is future
- proof approach. It's also fully backwards-compatible if you use `_` or `.`
in your extras, but we
- recommend using `-` as separator in the future.
-
+Airflow distribution packages
+-----------------------------
+
+With Airflow 3, Airflow is now split into several independent and isolated
distribution packages on top of
+already existing ``providers`` and the dependencies are isolated and
simplified across those distribution
+packages.
+
+While the original installation methods via ``apache-airflow`` distribution
package and extras still
+work as previously and it installs complete airflow installation ready to
serve as scheduler, webserver, triggerer
+and worker, the ``apache-airflow`` package is now a meta-package that installs
all the other distribution
+packages, it's also possible to install only the distribution packages that
are needed for a specific
+component you want to run airflow with.
+
+The following distribution packages are available:
+
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| Distribution package | Purpose
| Optional extras |
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| apache-airflow-core | This is the core distribution package that
contains | * Core extras that add optional functionality to
airflow |
+| | the airflow scheduler, webserver, triggerer
code. | core system - enhancing it's functionality across
|
+| |
| multiple providers. |
+| |
| |
+| |
| * Group ``all`` extra that install all optional |
Review Comment:
```suggestion
| | the airflow scheduler, webserver, triggerer
code. | core system - enhancing its functionality across |
| |
| multiple providers. |
| |
| |
| |
| * Group ``all`` extra that installs all optional |
```
##########
airflow-core/docs/extra-packages-ref.rst:
##########
@@ -18,20 +18,61 @@
Reference for package extras
''''''''''''''''''''''''''''
-Airflow has a number of optional "extras" that you can use to add features to
your installation when you
-are installing Airflow. Those extras are a good way for the users to manage
their installation, but also
-they are useful for contributors to airflow when they want to contribute some
of the features - including
-optional integrations of Airflow - via providers.
-
-.. warning::
-
- Traditionally in Airflow some of the extras used `.` and `_` to separate
the parts of the extra name.
- This was not PEP-685 normalized name and we opted to change it to to `-`
for all our extras, Expecting that
- PEP-685 will be implemented in full by `pip` and other tools we change all
our extras to use `-` as
- separator even if in some cases it will introduce warnings (the warnings
are harmless). This is future
- proof approach. It's also fully backwards-compatible if you use `_` or `.`
in your extras, but we
- recommend using `-` as separator in the future.
-
+Airflow distribution packages
+-----------------------------
+
+With Airflow 3, Airflow is now split into several independent and isolated
distribution packages on top of
+already existing ``providers`` and the dependencies are isolated and
simplified across those distribution
+packages.
+
+While the original installation methods via ``apache-airflow`` distribution
package and extras still
+work as previously and it installs complete airflow installation ready to
serve as scheduler, webserver, triggerer
+and worker, the ``apache-airflow`` package is now a meta-package that installs
all the other distribution
+packages, it's also possible to install only the distribution packages that
are needed for a specific
+component you want to run airflow with.
+
+The following distribution packages are available:
+
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| Distribution package | Purpose
| Optional extras |
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| apache-airflow-core | This is the core distribution package that
contains | * Core extras that add optional functionality to
airflow |
+| | the airflow scheduler, webserver, triggerer
code. | core system - enhancing it's functionality across
|
+| |
| multiple providers. |
+| |
| |
+| |
| * Group ``all`` extra that install all optional |
+| |
| functionalities together. |
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| apache-airflow-task-sdk | This is distribution package that is needed
| - |
+| | to run tasks in the worker
| |
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| apache-airflow-providers-* | Those are distribution packages that contain
| * Each provider distribution packages might have its |
+| | integrations of Airflow with external systems,
| own optional extras |
+| | 3rd-party software and services. Usually
provide |
|
Review Comment:
```suggestion
| | 3rd-party software and services. Usually they
provide |
|
```
##########
airflow-core/docs/extra-packages-ref.rst:
##########
@@ -18,20 +18,61 @@
Reference for package extras
''''''''''''''''''''''''''''
-Airflow has a number of optional "extras" that you can use to add features to
your installation when you
-are installing Airflow. Those extras are a good way for the users to manage
their installation, but also
-they are useful for contributors to airflow when they want to contribute some
of the features - including
-optional integrations of Airflow - via providers.
-
-.. warning::
-
- Traditionally in Airflow some of the extras used `.` and `_` to separate
the parts of the extra name.
- This was not PEP-685 normalized name and we opted to change it to to `-`
for all our extras, Expecting that
- PEP-685 will be implemented in full by `pip` and other tools we change all
our extras to use `-` as
- separator even if in some cases it will introduce warnings (the warnings
are harmless). This is future
- proof approach. It's also fully backwards-compatible if you use `_` or `.`
in your extras, but we
- recommend using `-` as separator in the future.
-
+Airflow distribution packages
+-----------------------------
+
+With Airflow 3, Airflow is now split into several independent and isolated
distribution packages on top of
+already existing ``providers`` and the dependencies are isolated and
simplified across those distribution
+packages.
+
+While the original installation methods via ``apache-airflow`` distribution
package and extras still
+work as previously and it installs complete airflow installation ready to
serve as scheduler, webserver, triggerer
+and worker, the ``apache-airflow`` package is now a meta-package that installs
all the other distribution
+packages, it's also possible to install only the distribution packages that
are needed for a specific
+component you want to run airflow with.
+
+The following distribution packages are available:
+
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| Distribution package | Purpose
| Optional extras |
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| apache-airflow-core | This is the core distribution package that
contains | * Core extras that add optional functionality to
airflow |
+| | the airflow scheduler, webserver, triggerer
code. | core system - enhancing it's functionality across
|
+| |
| multiple providers. |
+| |
| |
+| |
| * Group ``all`` extra that install all optional |
+| |
| functionalities together. |
++----------------------------+------------------------------------------------------------------+----------------------------------------------------------+
+| apache-airflow-task-sdk | This is distribution package that is needed
| - |
Review Comment:
```suggestion
| apache-airflow-task-sdk | This is the distribution package that is
needed | -
|
```
##########
airflow-core/docs/extra-packages-ref.rst:
##########
@@ -353,15 +400,16 @@ pre-installed when Airflow is installed.
| ssh | ``pip install 'apache-airflow[ssh]'`` |
SSH hooks and operators | |
+---------------------+-----------------------------------------------------+--------------------------------------+--------------+
-Doc extras
-==========
+Group extras
+------------
-Those are the extras that are needed to generated documentation for Airflow.
This is used for development time only
+The group extras are convenience extra that install all the extras in the
selected group.
Review Comment:
```suggestion
The group extras are a convenience extra that installs all the extras in the
selected group.
```
##########
dev/breeze/doc/08_ci_tasks.rst:
##########
@@ -114,23 +114,6 @@ These are all available flags of ``get-workflow-info``
command:
:width: 100%
:alt: Breeze ci get-workflow-info
-Finding backtracking candidates
Review Comment:
Very nice to get rid of this section :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
