vincbeck commented on code in PR #39513:
URL: https://github.com/apache/airflow/pull/39513#discussion_r1601723973
##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1108,6 +1108,133 @@ This prepares airflow .whl package in the dist folder.
breeze --use-airflow-version wheel --use-packages-from-dist
--mount-sources skip
+Compatibility Provider unit tests against older airflow releases
+................................................................
+
+Our CI runs provider tests for providers with previous compatible airflow
releases. This allows to check
+if the providers still work when installed for older airflow versions.
+
+.. note::
+
+ For now it's done for 2.9.1 version only.
+
+Those tests can be used to test compatibility of the providers with past and
future releases of airflow
Review Comment:
```suggestion
Those tests can be used to test compatibility of the providers with past and
future releases of airflow.
```
##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1108,6 +1108,133 @@ This prepares airflow .whl package in the dist folder.
breeze --use-airflow-version wheel --use-packages-from-dist
--mount-sources skip
+Compatibility Provider unit tests against older airflow releases
+................................................................
+
+Our CI runs provider tests for providers with previous compatible airflow
releases. This allows to check
+if the providers still work when installed for older airflow versions.
+
+.. note::
+
+ For now it's done for 2.9.1 version only.
+
+Those tests can be used to test compatibility of the providers with past and
future releases of airflow
+For example it could be used to run latest provider versions with released or
main
+Airflow 3 if they are developed independently.
+
+The tests use the current source version of ``tests`` folder - so care should
be taken that the tests
+implemented for providers in the sources allow to run it against previous
versions of Airflow and
+against Airflow installed from package rather than from the sources.
+
+This can be reproduced locally building providers from tag/commit of the
airflow repository.
+
+1. Make sure to build latest Breeze ci image
+
+.. code-block:: bash
+
+ breeze ci-image build --python 3.8
+
+2. Build providers from latest sources:
+
+.. code-block:: bash
+
+ rm dist/*
+ breeze release-management prepare-provider-packages
--include-not-ready-providers \
+ --version-suffix-for-pypi dev0 --package-format wheel
+
+3. Prepare provider constraints
+
+.. code-block:: bash
+
+ breeze release-management generate-constraints --airflow-constraints-mode
constraints-source-providers --answer yes
+
+3. Enter breeze environment, installing selected airflow version and the
provider packages prepared from main
+
+.. code-block::bash
+
+ breeze shell --use-packages-from-dist --package-format wheel\
+ --use-airflow-version 2.9.1 --airflow-constraints-reference
constraints-2.9.1 \
+ --install-airflow-with-constraints \
+ --providers-skip-constraints \
+ --mount-sources tests
+
+4. You can then run tests as usual:
+
+.. code-block::bash
+
+ pytest tests/providers/<provider>/test.py
+
+5. Iterate with the tests
+
+The tests are run using:
+
+* airflow installed from PyPI
+* tests coming from the current airflow sources (they are mounted inside the
breeze image)
+* provider packages built from the current airflow sources and placed in dist
+
+This means that you can modify and run tests and re-run them, but if you want
to modify provider code
+you need to exit breeze, rebuild the provider package and restart breeze using
the command above.
+
+Rebuilding single provider package can be done using this command:
+
+.. code-block::bash
+
+ breeze release-management prepare-provider-packages \
+ --version-suffix-for-pypi dev0 --package-format wheel <provider>
+
+
+Note that some of the tests if written without taking care about the
compatibility, might not work with older
+versions of Airflow - this is because of refactorings, renames, and tests
relying on internals of Airflow that
+are not part of the public API. We deal with it in one of the following ways:
+
+1) if the whole provider is supposed to only work for later airflow version,
we remove the whole provider
+ by excluding it from compatibility test configuration (see below)
+
+2) Some compatibility shims are defined in ``tests/test_utils/compat.py`` -
and they can be used to make the
+ tests compatible - for example importing ``ParseImportError`` after the
exception has been renamed from
+ ``ImportError`` and it would fail in Airflow 2.9, but we have a fallback
import in ``compat.py`` that
+ falls back to old import automatically, so all tests testing / expecting
``ParseImportError`` should import
+ it from the ``tests.tests_utils.compat`` module. There are few other
compatibility shims defined there and
+ you can add more if needed in a similar way.
+
+3) If only some tests are not compatible and use features that are available
only in newer airflow version,
+ we can mark those tests with appropriate ``AIRFLOW_V_2_X_PLUS`` boolean
constant defined in ``compat.py``
+ For example:
+
+.. code-block::python
+
+ from tests.test_utils.compat import AIRFLOW_V_2_7_PLUS
+
+ @pytest.mark.skip(not AIRFLOW_V_2_7_PLUS, reason="The tests should be
skipped for Airflow < 2.7")
+ def some_test_that_only_works_for_airflow_2_7_plus():
+ pass
+
+4) Sometimes, the tests should only be run when airflow is installed from the
sources. In this case you can
+ add conditional ``skipif`` markerfor
``RUNNING_TESTS_AGAINST_AIRFLOW_PACKAGES`` to the test. For example:
+
+.. code-block::python
+
+ @pytest.mark.skipif(RUNNING_TESTS_AGAINST_AIRFLOW_PACKAGES,
+ reason="Plugin initialization is done early in case of
packages")
+ def test_plugin():
+ pass
+
+
+How providers compatibility tests are run in CI ?
Review Comment:
```suggestion
How providers compatibility tests are run in CI?
```
##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1108,6 +1108,133 @@ This prepares airflow .whl package in the dist folder.
breeze --use-airflow-version wheel --use-packages-from-dist
--mount-sources skip
+Compatibility Provider unit tests against older airflow releases
+................................................................
+
+Our CI runs provider tests for providers with previous compatible airflow
releases. This allows to check
+if the providers still work when installed for older airflow versions.
+
+.. note::
+
+ For now it's done for 2.9.1 version only.
+
+Those tests can be used to test compatibility of the providers with past and
future releases of airflow
+For example it could be used to run latest provider versions with released or
main
+Airflow 3 if they are developed independently.
+
+The tests use the current source version of ``tests`` folder - so care should
be taken that the tests
+implemented for providers in the sources allow to run it against previous
versions of Airflow and
+against Airflow installed from package rather than from the sources.
+
+This can be reproduced locally building providers from tag/commit of the
airflow repository.
+
+1. Make sure to build latest Breeze ci image
+
+.. code-block:: bash
+
+ breeze ci-image build --python 3.8
+
+2. Build providers from latest sources:
+
+.. code-block:: bash
+
+ rm dist/*
+ breeze release-management prepare-provider-packages
--include-not-ready-providers \
+ --version-suffix-for-pypi dev0 --package-format wheel
+
+3. Prepare provider constraints
+
+.. code-block:: bash
+
+ breeze release-management generate-constraints --airflow-constraints-mode
constraints-source-providers --answer yes
+
+3. Enter breeze environment, installing selected airflow version and the
provider packages prepared from main
+
+.. code-block::bash
+
+ breeze shell --use-packages-from-dist --package-format wheel\
+ --use-airflow-version 2.9.1 --airflow-constraints-reference
constraints-2.9.1 \
+ --install-airflow-with-constraints \
+ --providers-skip-constraints \
+ --mount-sources tests
+
+4. You can then run tests as usual:
+
+.. code-block::bash
+
+ pytest tests/providers/<provider>/test.py
+
+5. Iterate with the tests
+
+The tests are run using:
+
+* airflow installed from PyPI
+* tests coming from the current airflow sources (they are mounted inside the
breeze image)
+* provider packages built from the current airflow sources and placed in dist
+
+This means that you can modify and run tests and re-run them, but if you want
to modify provider code
+you need to exit breeze, rebuild the provider package and restart breeze using
the command above.
+
+Rebuilding single provider package can be done using this command:
+
+.. code-block::bash
+
+ breeze release-management prepare-provider-packages \
+ --version-suffix-for-pypi dev0 --package-format wheel <provider>
+
+
+Note that some of the tests if written without taking care about the
compatibility, might not work with older
+versions of Airflow - this is because of refactorings, renames, and tests
relying on internals of Airflow that
+are not part of the public API. We deal with it in one of the following ways:
+
+1) if the whole provider is supposed to only work for later airflow version,
we remove the whole provider
Review Comment:
```suggestion
1) If the whole provider is supposed to only work for later airflow version,
we remove the whole provider
```
##########
tests/conftest.py:
##########
@@ -1292,6 +1296,20 @@ def _disable_redact(request: pytest.FixtureRequest,
mocker):
return
[email protected]
+def airflow_root_path() -> Path:
+ import airflow
+
+ return Path(airflow.__path__[0]).parent
+
+
+# This constant is set to True if tests are run with Airflow installed from
Packages rather than running
+# The tests within Airflow sources. While most tests in CI are run using
Airflow sources, there are
Review Comment:
```suggestion
# the tests within Airflow sources. While most tests in CI are run using
Airflow sources, there are
```
##########
contributing-docs/testing/unit_tests.rst:
##########
@@ -1108,6 +1108,133 @@ This prepares airflow .whl package in the dist folder.
breeze --use-airflow-version wheel --use-packages-from-dist
--mount-sources skip
+Compatibility Provider unit tests against older airflow releases
+................................................................
+
+Our CI runs provider tests for providers with previous compatible airflow
releases. This allows to check
+if the providers still work when installed for older airflow versions.
+
+.. note::
+
+ For now it's done for 2.9.1 version only.
+
+Those tests can be used to test compatibility of the providers with past and
future releases of airflow
+For example it could be used to run latest provider versions with released or
main
+Airflow 3 if they are developed independently.
+
+The tests use the current source version of ``tests`` folder - so care should
be taken that the tests
+implemented for providers in the sources allow to run it against previous
versions of Airflow and
+against Airflow installed from package rather than from the sources.
+
+This can be reproduced locally building providers from tag/commit of the
airflow repository.
+
+1. Make sure to build latest Breeze ci image
+
+.. code-block:: bash
+
+ breeze ci-image build --python 3.8
+
+2. Build providers from latest sources:
+
+.. code-block:: bash
+
+ rm dist/*
+ breeze release-management prepare-provider-packages
--include-not-ready-providers \
+ --version-suffix-for-pypi dev0 --package-format wheel
+
+3. Prepare provider constraints
+
+.. code-block:: bash
+
+ breeze release-management generate-constraints --airflow-constraints-mode
constraints-source-providers --answer yes
+
+3. Enter breeze environment, installing selected airflow version and the
provider packages prepared from main
+
+.. code-block::bash
+
+ breeze shell --use-packages-from-dist --package-format wheel\
+ --use-airflow-version 2.9.1 --airflow-constraints-reference
constraints-2.9.1 \
+ --install-airflow-with-constraints \
+ --providers-skip-constraints \
+ --mount-sources tests
+
+4. You can then run tests as usual:
+
+.. code-block::bash
+
+ pytest tests/providers/<provider>/test.py
+
+5. Iterate with the tests
+
+The tests are run using:
+
+* airflow installed from PyPI
+* tests coming from the current airflow sources (they are mounted inside the
breeze image)
+* provider packages built from the current airflow sources and placed in dist
+
+This means that you can modify and run tests and re-run them, but if you want
to modify provider code
+you need to exit breeze, rebuild the provider package and restart breeze using
the command above.
+
+Rebuilding single provider package can be done using this command:
+
+.. code-block::bash
+
+ breeze release-management prepare-provider-packages \
+ --version-suffix-for-pypi dev0 --package-format wheel <provider>
+
+
+Note that some of the tests if written without taking care about the
compatibility, might not work with older
+versions of Airflow - this is because of refactorings, renames, and tests
relying on internals of Airflow that
+are not part of the public API. We deal with it in one of the following ways:
+
+1) if the whole provider is supposed to only work for later airflow version,
we remove the whole provider
+ by excluding it from compatibility test configuration (see below)
+
+2) Some compatibility shims are defined in ``tests/test_utils/compat.py`` -
and they can be used to make the
+ tests compatible - for example importing ``ParseImportError`` after the
exception has been renamed from
+ ``ImportError`` and it would fail in Airflow 2.9, but we have a fallback
import in ``compat.py`` that
+ falls back to old import automatically, so all tests testing / expecting
``ParseImportError`` should import
+ it from the ``tests.tests_utils.compat`` module. There are few other
compatibility shims defined there and
+ you can add more if needed in a similar way.
+
+3) If only some tests are not compatible and use features that are available
only in newer airflow version,
+ we can mark those tests with appropriate ``AIRFLOW_V_2_X_PLUS`` boolean
constant defined in ``compat.py``
+ For example:
+
+.. code-block::python
+
+ from tests.test_utils.compat import AIRFLOW_V_2_7_PLUS
+
+ @pytest.mark.skip(not AIRFLOW_V_2_7_PLUS, reason="The tests should be
skipped for Airflow < 2.7")
+ def some_test_that_only_works_for_airflow_2_7_plus():
+ pass
+
+4) Sometimes, the tests should only be run when airflow is installed from the
sources. In this case you can
+ add conditional ``skipif`` markerfor
``RUNNING_TESTS_AGAINST_AIRFLOW_PACKAGES`` to the test. For example:
+
+.. code-block::python
+
+ @pytest.mark.skipif(RUNNING_TESTS_AGAINST_AIRFLOW_PACKAGES,
+ reason="Plugin initialization is done early in case of
packages")
+ def test_plugin():
+ pass
+
+
+How providers compatibility tests are run in CI ?
+-------------------------------------------------
+
+We run a set of back-compatibility tests based on the configuration specified
in the
+``BASE_PROVIDERS_COMPATIBILITY_CHECKS`` constant in the
``./dev/breeze/src/airflow_breeze/global_constants.py``
+file - where we specify:
+* python version
+* airflow version
+* which providers should be removed (exclusions)
+* whether to run tests
+
+We deal with it in the following way:
+
+* If the whole provider depends
Review Comment:
I guess it is missing something here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]