ramitkataria opened a new issue, #77273:
URL: https://github.com/apache/airflow/issues/77273
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
main (3.5.0 development)
### What happened and how to reproduce it?
**Issue description**
`AirflowProviderDeprecationWarning`s do not reach users when the deprecated
feature is used from a Dag file or other user code. They appear neither in
the
Dag processor's per-file log nor in the task log.
`airflow-core/src/airflow/configuration.py` enables `DeprecationWarning` only
for warnings attributed to a module whose name starts with `airflow`. A
provider
deprecation is attributed to the code using the deprecated feature:
- Operator `__init__` warnings are attributed to the Dag file, because
`fixup_decorator_warning_stack` shifts the stacklevel past the
`apply_defaults` wrapper. The Dag module is named
`unusual_prefix_<hash>_<file>`.
- Hook method warnings with `stacklevel=2` are attributed to the caller,
which
is user code when a `@task` function or a custom operator calls the hook.
Neither matches the `airflow` module filter, so Python's default
`ignore::DeprecationWarning` applies. The Dag importer records warnings
without
changing filters, so nothing is captured.
**Steps to reproduce**
1. Add this Dag file (Amazon provider used as the example):
```python
import warnings
from airflow.providers.amazon.aws.hooks.kinesis import FirehoseHook
from airflow.providers.amazon.aws.operators.glue_crawler import
GlueCrawlerOperator
from airflow.providers.amazon.aws.sensors.eks import EksClusterStateSensor
from airflow.sdk import DAG, task
warnings.warn("control UserWarning", UserWarning, stacklevel=1)
with DAG("deprecation_parse", schedule=None):
EksClusterStateSensor(task_id="eks", cluster_name="c",
region="us-east-1")
GlueCrawlerOperator(task_id="crawler", config={"Name": "x"})
with DAG("deprecation_runtime", schedule=None):
@task
def use_deprecated_hook():
FirehoseHook(delivery_stream="s")
use_deprecated_hook()
```
2. Run `airflow dag-processor --num-runs 1` and read the per-file log under
`logs/dag_processor/`. It contains the control `UserWarning` and neither
of
the two provider deprecations.
3. Run `airflow dags test deprecation_runtime`. The task output has no
`FirehoseHook` deprecation.
No `PYTHONWARNINGS` or `-W` option is set.
### What you think should happen instead?
The deprecation warnings should be logged like the control warning, pointing
at
the Dag file and line. Providers raise them so users can migrate before a
removal, and today users only find out when the feature is gone.
### Anything else?
- The module filter dates from 1.7.1 (2016). Operator `__init__` warnings
have
been attributed to the Dag file since 2.1.0 (#15667), so they have been
hidden
since then.
- `RemovedInAirflow4Warning` goes through the same filter. For example, the
`email` deprecation from `EmptyOperator(task_id="t",
email="[email protected]")`
in a Dag file is hidden as well.
- `airflow-core/src/airflow/dag_processing/dagbag.py` already special-cases
`TaskGroupCycleDeprecationWarning` for this reason.
- Tests do not catch this because pytest enables all deprecation warnings.
- Proposed fix: add a `default` filter for the
`AirflowProviderDeprecationWarning`
category with no module restriction. Whether `RemovedInAirflow4Warning`
should
get the same treatment is a question for maintainers.
### Operating System
Breeze on MacOS 26.7.3
### Deployment
Other
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
- Prior reports: #55875 reported the same symptom for deprecated imports.
#55881 traced it to the same filter and fixed only imports, by adding
`DeprecatedImportWarning` (a `FutureWarning`). #55889 discussed provider
warnings and left `AirflowProviderDeprecationWarning` unchanged, so
provider
deprecations are still hidden.
- #55881 avoided changing the filters so task logs would not show every
deprecation warning. A filter on the `AirflowProviderDeprecationWarning`
category alone does not do that. The other option, floated in #55889, is to
make it a `FutureWarning` subclass.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]