rt849122 opened a new issue, #76723: URL: https://github.com/apache/airflow/issues/76723
### Description Add an Airflow configuration option to control the default value of keep_task_state when clearing task instances. https://airflow.apache.org/docs/apache-airflow/3.4.0/release_notes.html#clearing-a-task-now-discards-its-task-state-store-entries-by-default-72100 Airflow 3.4 changed the default to discard task state on clear. The UI offers a browser-local preference, but administrators cannot configure a consistent default for their deployment. ### Use case/motivation Most tasks in our deployment use a custom operator that submits, attaches to, and monitors jobs running on an external compute platform. We use the task state store as a persistent execution history for each task instance, rather than only as a checkpoint for its current attempt. It records the external jobs associated with the task, including job identifiers, request identifiers, links, and the task attempts associated with those jobs. This history allows our operator to decide whether to reconnect to an existing job or submit a new one. Our workflow also supports manually attaching a task to an existing external job. The attachment is recorded in the task state store so subsequent attempts can continue monitoring that job without submitting another one. Clearing a task therefore does not necessarily mean starting its external work from scratch. We often clear a task to recover monitoring after an Airflow-side failure, reconnect to a job that is still running, or continue with a manually selected job. Discarding task state removes the execution history and attachment information needed for these workflows and can lead to duplicate job submissions. Because this operator accounts for most of our tasks, we explicitly want keep_task_state=True as our deployment-wide default. Preserving state is our normal operating mode; discarding it should be an explicit choice when a fresh execution is intended. We would like airflow to provide a deployment-wide configuration option that determines the default value of keep_task_state in the UI and when an API request omits the field. ### Related issues _No response_ ### Are you willing to submit a PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
