SEPURI-SAI-KRISHNA opened a new pull request, #74171: URL: https://github.com/apache/airflow/pull/74171
`AwsTaskLogFetcher` built its CloudWatch client from the connection id and region alone, so `verify` and `botocore_config` set on the operator never reached the client that streams task logs. A deployment pointing `verify` at a private CA bundle, or tuning `botocore_config`, got those settings honoured when talking to ECS or Batch and silently ignored when reading the logs. The clearest way to see it is inside `operators/ecs.py`, which has two CloudWatch paths. `execute_complete` builds `AwsLogsHook(aws_conn_id=..., region_name=..., verify=self.verify, config=self.botocore_config)` for the final log line. `_get_task_log_fetcher` built the streaming client with the first two arguments only. The same operator honoured both settings on one log path and dropped them on the other. `AwsTaskLogFetcher.__init__` now accepts `verify` and `botocore_config` and forwards them to `AwsLogsHook` as `verify=` and `config=`, which is what `AwsGenericHook` calls that argument. `EcsRunTaskOperator` and `BatchOperator` pass their own values. **Tests** One test per changed site: the fetcher forwards both settings to its hook, and each operator's log-fetcher factory passes its own. Reverting the three source edits fails all three, with `TypeError: AwsTaskLogFetcher.__init__() got an unexpected keyword argument 'verify'` at the fetcher and `-None +'/path/to/ca-bundle.pem'` at both call sites. Also run: the three full test files (151 passed, with 5 pre-existing `airflow_shared` collection errors that reproduce identically on `upstream/main`), mypy on all six files, and the `check-deferred-hook-configuration` prek hook. **Two scope decisions** I added a changelog warning because the change is user-actionable: a tight `botocore_config` read timeout now applies to log streaming where it previously did not. Happy to drop it if that reads as noise for a bugfix. I did not extend the `check-deferred-hook-configuration` prek hook to cover this. That hook sweeps hand-built hooks under `triggers/`, where the operator's settings are the only sensible source. Outside `triggers/`, 90 of 92 hook constructions in the provider pass fewer than all three parameters, because most are configured from `airflow.cfg` rather than from an operator: executors, the auth manager, bundles, the filesystem. An allowlist that size would guard nothing. Out of scope on purpose: `log/cloudwatch_task_handler.py` is remote logging configured from `airflow.cfg`, not from an operator. related: #72144 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes - Claude Code (Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
