vladimirbinshtok commented on issue #52943: URL: https://github.com/apache/airflow/issues/52943#issuecomment-3051816289
@fweilun According to AWS documentation (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/resource-initialization-error.html), most ResourceInitializationErrors are caused by misconfiguration in the Task Definition or Permissions; so, retry would not help in that case. Probably, there are some edge cases where retry should be implemented. Are there any contact points in AWS that can help define these cases? I'm using the ESC operator with the following flags: `deferrable=True` and `wait_for_completion=False`. Therefore, the function that should be used in the retry is `self._execute`, not `self._start_task`. The AwsBaseHook.retry mechanism currently doesn't support this mode, as the current retry attempt is not passed to the next task execution (which would lead to an infinite retry) and can only be performed in-process. Only ECS Operator use this retry logic; all other operators will fail in any case of AWS internal error. I suggest completely turning off the retry mechanism. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
