waterWang opened a new pull request, #72124: URL: https://github.com/apache/airflow/pull/72124
Closes: #72123 ## Problem When `deferrable=True`, `EmrServerlessDeleteApplicationOperator` never calls the `DeleteApplication` API. The task succeeds and logs `EMR serverless application <id> deleted successfully`, but the application is only *stopped*, never deleted. ## Root cause `EmrServerlessDeleteApplicationOperator` subclasses `EmrServerlessStopApplicationOperator`. In deferrable mode the parent's `execute()` defers with a hardcoded `method_name="execute_complete"`, which raises `TaskDeferred` before the subclass's `execute()` can reach the `delete_application()` call. On resume, the subclass's `execute_complete()` handler runs — but that handler only validates the **stop** trigger's event and logs success, so `DeleteApplication` is never issued. ## Fix Follow the chained-deferral pattern the stop operator already uses for the `force_stop` path (`execute` → `stop_application` → `execute_complete`): 1. Add a class attribute `stop_complete_method_name: str = "execute_complete"` to `EmrServerlessStopApplicationOperator`, used in place of both hardcoded `method_name="execute_complete"` occurrences. 2. `EmrServerlessDeleteApplicationOperator` sets `stop_complete_method_name = "stop_complete"` and implements `stop_complete()` to issue `DeleteApplication` and defer on `EmrServerlessDeleteApplicationTrigger`, leaving `execute_complete` to handle only the delete trigger's event. Backwards compatible: `execute_complete` still exists for tasks already deferred by an older version at upgrade time. ## Verification - New unit test `test_delete_application_deferrable_deletes_after_stop` proves the full chained flow: first defer is on the `StopApplicationTrigger` resuming at `stop_complete`, which calls `delete_application` and defers on the `DeleteApplicationTrigger` resuming at `execute_complete`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
