Echronix opened a new issue #18549: URL: https://github.com/apache/airflow/issues/18549
### Apache Airflow version 2.1.2 ### Operating System Ubuntu Docker ### Versions of Apache Airflow Providers apache-airflow-providers-amazon==2.0.0 apache-airflow-providers-celery==2.0.0 apache-airflow-providers-cncf-kubernetes==2.0.0 apache-airflow-providers-docker==2.0.0 apache-airflow-providers-elasticsearch==2.0.2 apache-airflow-providers-ftp==2.0.0 apache-airflow-providers-google==4.0.0 apache-airflow-providers-grpc==2.0.0 apache-airflow-providers-hashicorp==2.0.0 apache-airflow-providers-http==2.0.0 apache-airflow-providers-imap==2.0.0 apache-airflow-providers-microsoft-azure==3.0.0 apache-airflow-providers-mysql==2.0.0 apache-airflow-providers-neo4j==2.0.0 apache-airflow-providers-postgres==2.0.0 apache-airflow-providers-redis==2.0.0 apache-airflow-providers-sendgrid==2.0.0 apache-airflow-providers-sftp==2.0.0 apache-airflow-providers-slack==4.0.0 apache-airflow-providers-sqlite==2.0.0 apache-airflow-providers-ssh==2.0.0 ### Deployment Official Apache Airflow Helm Chart ### Deployment details ``` config: core: executor: 'CeleryExecutor' logging: remote_logging: 'True' remote_base_log_folder: 'cloudwatch://arn:aws:logs:eu-central-1:<redacted>:log-group:airflow' remote_log_conn_id: 'cloudwatch' ``` The log group is already created in Terraform. ### What happened ``` [2021-09-27 15:03:58,541: ERROR/ForkPoolWorker-15] Failed to execute task An error occurred (AccessDeniedException) when calling the CreateLogGroup operation: User: arn:aws:sts::<redacted>:assumed-role/airflow-service-account-role/botocore-session-1632755037 is not authorized to perform: logs:CreateLogGroup on resource: arn:aws:logs:eu-central-1:<redacted>:log-group:airflow:log-stream:. Traceback (most recent call last): File "/home/airflow/.local/lib/python3.8/site-packages/airflow/executors/celery_executor.py", line 117, in _execute_in_fork args.func(args) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command return func(*args, **kwargs) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 91, in wrapper return f(*args, **kwargs) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py", line 228, in task_run ti.init_run_context(raw=args.raw) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 2031, in init_run_context self._set_context(self) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py", line 54, in _set_context set_context(self.log, context) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py", line 173, in set_context handler.set_context(value) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/amazon/aws/log/cloudwatch_task_handler.py", line 81, in set_context self.handler = watchtower.CloudWatchLogHandler( File "/home/airflow/.local/lib/python3.8/site-packages/watchtower/__init__.py", line 148, in __init__ _idempotent_create(self.cwl_client, "create_log_group", logGroupName=self.log_group) File "/home/airflow/.local/lib/python3.8/site-packages/watchtower/__init__.py", line 15, in _idempotent_create method_callable(*args, **kwargs) File "/home/airflow/.local/lib/python3.8/site-packages/botocore/client.py", line 386, in _api_call return self._make_api_call(operation_name, kwargs) File "/home/airflow/.local/lib/python3.8/site-packages/botocore/client.py", line 705, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the CreateLogGroup operation: User: arn:aws:sts:<redacted>}:assumed-role/airflow-service-account-role/botocore-session-1632755037 is not authorized to perform: logs:CreateLogGroup on resource: arn:aws:logs:eu-central-1:<redacted>:log-group:airflow:log-stream: ``` ### What you expected to happen I would expect Airflow to create a stream and write the logs, not to create a log-group inside a log-group. ### How to reproduce Run any dag with the chart config. ### Anything else _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org