Nandann018-ux commented on code in PR #64133:
URL: https://github.com/apache/airflow/pull/64133#discussion_r3101722845
##########
airflow-core/tests/unit/cli/commands/test_task_command.py:
##########
@@ -305,6 +305,68 @@ def test_mapped_task_render(self):
assert "[3]" not in output
assert "property: op_args" in output
+ @pytest.mark.usefixtures("testing_dag_bundle")
+ def test_mapped_task_render_out_of_range_map_index(self):
+ """Raise ValueError when map_index exceeds the parse-time mapped
count."""
+ with pytest.raises(ValueError, match=r"map_index 5 is out of range.*3
mapped instance"):
Review Comment:
Yep, addressed. The test now validates the full message via
`exc_info.value.args` instead of using regex matching.
##########
airflow-core/docs/extra-packages-ref.rst:
##########
@@ -92,6 +92,8 @@ python dependencies for the provided package. The same extras
are available as `
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------------+
| kerberos | ``pip install 'apache-airflow[kerberos]'`` |
Kerberos integration for Kerberized services (Hadoop, Presto, Trino) |
+---------------------+-----------------------------------------------------+----------------------------------------------------------------------------+
+| memray | ``pip install 'apache-airflow[memray]'`` |
Required for memory profiling with memray |
Review Comment:
This one came from the earlier bad rebase that briefly pulled unrelated docs
and CI files into the PR. I rebuilt the branch cleanly on top of `main`, and
those files are no longer part of this change, so this thread no longer applies.
##########
airflow-core/docs/core-concepts/operators.rst:
##########
@@ -330,4 +343,38 @@ If you need to include a Jinja template expression (e.g.,
``{{ ds }}``) literall
dag=dag,
)
-This ensures the f-string processing results in a string containing the
literal double braces required by Jinja, which Airflow can then template
correctly before execution. Failure to do this is a common issue for beginners
and can lead to errors during Dag parsing or unexpected behavior at runtime
when the templating does not occur as expected.
+This ensures the f-string processing results in a string containing the
literal double braces required by Jinja, which Airflow can then template
correctly before execution. Failure to do this is a common issue for beginners
and can lead to errors during DAG parsing or unexpected behavior at runtime
when the templating does not occur as expected.
+
+Pre- and post-execute methods
+-----------------------------
+
+The ``pre_execute`` and ``post_execute`` methods are called before and after
the operator is executed, respectively.
+
+For example, you can use the ``pre_execute`` method to elegantly determine if
a task should be executed or not:
+
+.. code-block:: python
+
+ def _check_skipped(context: Any) -> None:
Review Comment:
This one came from the earlier bad rebase that briefly pulled unrelated docs
and CI files into the PR. I rebuilt the branch cleanly on top of `main`, and
those files are no longer part of this change, so this thread no longer applies.
##########
airflow-core/docs/core-concepts/operators.rst:
##########
@@ -330,4 +343,38 @@ If you need to include a Jinja template expression (e.g.,
``{{ ds }}``) literall
dag=dag,
)
-This ensures the f-string processing results in a string containing the
literal double braces required by Jinja, which Airflow can then template
correctly before execution. Failure to do this is a common issue for beginners
and can lead to errors during Dag parsing or unexpected behavior at runtime
when the templating does not occur as expected.
+This ensures the f-string processing results in a string containing the
literal double braces required by Jinja, which Airflow can then template
correctly before execution. Failure to do this is a common issue for beginners
and can lead to errors during DAG parsing or unexpected behavior at runtime
when the templating does not occur as expected.
+
+Pre- and post-execute methods
+-----------------------------
+
+The ``pre_execute`` and ``post_execute`` methods are called before and after
the operator is executed, respectively.
+
+For example, you can use the ``pre_execute`` method to elegantly determine if
a task should be executed or not:
+
+.. code-block:: python
+
+ def _check_skipped(context: Any) -> None:
+ """
+ Check if a given task instance should be skipped if the
`tasks_to_skip` Airflow Variable is a list that contains the task id.
+ """
+ tasks_to_skip = Variable.get("tasks_to_skip", deserialize_json=True)
+ if context["task"].task_id in on_ice:
Review Comment:
This one came from the earlier bad rebase that briefly pulled unrelated docs
and CI files into the PR. I rebuilt the branch cleanly on top of `main`, and
those files are no longer part of this change, so this thread no longer applies.
##########
airflow-core/docs/core-concepts/auth-manager/index.rst:
##########
@@ -161,35 +160,59 @@ cookie named ``_token`` before redirecting to the Airflow
UI. The Airflow UI wil
.. code-block:: python
+ from airflow.api_fastapi.app import get_cookie_path
from airflow.api_fastapi.auth.managers.base_auth_manager import
COOKIE_NAME_JWT_TOKEN
response = RedirectResponse(url="/")
secure = request.base_url.scheme == "https" or bool(conf.get("api",
"ssl_cert", fallback=""))
- response.set_cookie(COOKIE_NAME_JWT_TOKEN, token, secure=secure)
+ response.set_cookie(COOKIE_NAME_JWT_TOKEN, token, path=get_cookie_path(),
secure=secure, httponly=True)
return response
.. note::
- Do not set the cookie parameter ``httponly`` to ``True``. Airflow UI needs
to access the JWT token from the cookie.
+ Ensure that the cookie parameter ``httponly`` is set to ``True``. The UI
does not manage the token.
+Refreshing JWT Token
+''''''''''''''''''''
+Refreshing token is optional feature and its availability depends on the
specific implementation of the auth manager.
+The auth manager is responsible for refreshing the JWT token when it expires.
+The Airflow API uses middleware that intercepts every request and checks the
validity of the JWT token.
+Token communication is handled through ``httponly`` cookies to improve
security.
+When the token expires, the `JWTRefreshMiddleware
<https://github.com/apache/airflow/blob/3.1.5/airflow-core/src/airflow/api_fastapi/auth/middlewares/refresh_token.py>`_
middleware calls the auth manager's ``refresh_user`` method to obtain a new
token.
+
+
+To support token refresh operations, the auth manager must implement the
``refresh_user`` method.
+This method receives an expired token and must return a new valid token.
+User information is extracted from the expired token and used to generate a
fresh token.
+
+An example implementation of ``refresh_user`` could be:
+`KeycloakAuthManager::refresh_user
<https://github.com/apache/airflow/blob/3.1.5/providers/keycloak/src/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py#L113-L121>`_
+User information is derived from the ``BaseUser`` instance.
+It is important that the user object contains all the fields required to
refresh the token. An example user class could be:
+`KeycloakAuthManagerUser(BaseUser)
<https://github.com/apache/airflow/blob/3.1.5/providers/keycloak/src/airflow/providers/keycloak/auth_manager/user.pys>`_.
Review Comment:
This one came from the earlier bad rebase that briefly pulled unrelated docs
and CI files into the PR. I rebuilt the branch cleanly on top of `main`, and
those files are no longer part of this change, so this thread no longer applies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
