zachliu opened a new issue, #67133:
URL: https://github.com/apache/airflow/issues/67133
### Under which category would you file this issue?
Providers
### Apache Airflow version
3.2.1
### What happened and how to reproduce it?
`EcsRunTaskOperator` reads container stdout/stderr from CloudWatch and
writes all log lines to Airflow task logs at INFO level, regardless of the
original log level emitted by the application. This makes it
impossible to distinguish ERROR, WARNING, and DEBUG messages in the Airflow
UI.
For example, a container emits this JSON-structured log entry:
```json
{"asctime": "2026-05-18 08:06:04", "name": "__main__", "levelname": "ERROR",
"process": 21, "taskname": "my_task_name", "message": "During data transfer,
unable to find S3 key
'path/to/my/object' in the bucket 'my-bucket'", "exc_info": "Traceback (most
recent call last):\n File \"/home/prod/src/main.py\", line
141, in main\n s3.copy_file(\n File \"/home/prod/src/pkg/s3_utils.py\",
line 733, in copy_file\n raise RuntimeError(\nRuntimeError: ...", "source":
"application"}
```
Despite the clear `"levelname": "ERROR"` field, the Airflow UI displays it
as:
```
[2026-05-18 04:06:22] INFO - [2026-05-18 08:06:04,112] {...}
```
The level badge shows green "INFO" instead of orange "ERROR".
### What you think should happen instead?
`AwsTaskLogFetcher` should attempt to parse the log level from container
output and forward messages at the appropriate Python logging level. This would
make the UI display the correct level badge (e.g., ERROR in
orange, WARNING in yellow).
Use case/motivation
- ECS tasks emit structured JSON logs with explicit `levelname`/`level`
fields, but these are ignored by the log fetcher.
- Users scrolling through thousands of log lines cannot quickly spot errors
since everything appears as INFO.
- The log level filter in the UI becomes useless - filtering to "ERROR"
shows nothing, even when the task clearly failed with errors.
- Alerting pipelines that watch for ERROR-level entries in Airflow task logs
never trigger.
Root cause
In
https://github.com/apache/airflow/blob/providers-amazon/9.28.0/providers/amazon/src/airflow/providers/amazon/aws/utils/task_log_fetcher.py,
the `run()` method always writes at INFO:
https://github.com/apache/airflow/blob/077821050598e6807c63a497bcb70c420b529d8b/providers/amazon/src/airflow/providers/amazon/aws/utils/task_log_fetcher.py#L75
Similarly in
https://github.com/apache/airflow/blob/providers-amazon/9.28.0/providers/amazon/src/airflow/providers/amazon/aws/triggers/ecs.py,
the `_forward_logs` method:
https://github.com/apache/airflow/blob/077821050598e6807c63a497bcb70c420b529d8b/providers/amazon/src/airflow/providers/amazon/aws/triggers/ecs.py#L230-L231
No attempt is made to parse the log level from the message content.
Related issues
- #64820 - Identical issue for KubernetesPodOperator (fixed by #64829,
merged 2026-04-18)
- #53442 - Airflow Logging Configs are ignored / not used in Airflow 3.x
The fix in #64829 for KPO serves as a direct precedent for how this should
be implemented for the ECS/AWS provider.
### Operating System
LM 21.3 with docker python:3.12-slim-bookworm
### Deployment
Docker-Compose
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
[apache-airflow-providers-amazon==9.25.0](https://airflow.apache.org/docs/apache-airflow-providers-amazon/9.25.0)
[apache-airflow-providers-amazon==9.28.0](https://airflow.apache.org/docs/apache-airflow-providers-amazon/9.28.0)
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
