kaxil opened a new issue, #67236: URL: https://github.com/apache/airflow/issues/67236
## Problem Users browsing the [Airflow Registry](https://airflow.apache.org/registry/) cannot tell which operators support resumption -- i.e., operators that can pick up from where they left off after a worker crash, restart, or retry, rather than re-running from scratch. This matters for users planning long-running or expensive workloads (data transfers, ML training, large queries) where re-running from scratch on retry is wasteful or unsafe. ## Proposal Add a visible indicator on operator pages (and a filter on listing pages) in the Airflow Registry that marks operators as resumable. Open questions to resolve in this issue: - **Definition.** What qualifies as "resumable"? Candidates: deferrable operators (trigger-based resumption), operators with explicit checkpointing, idempotent transfer operators that can resume from offset/cursor, or some combination. Pick one definition or model it as multiple flags. - **Source of truth.** How do we get this signal? Options: a class attribute on the operator (e.g. `resumable: ClassVar[bool] = True`), inference from `BaseOperator` subclass traits (e.g. presence of `defer()` / `execute_complete()`), or curated metadata in `provider.yaml`. A class attribute is the most discoverable and authoritative. - **Registry surfacing.** Badge on the operator detail page + a filter chip on the operator list page. Tooltip explaining what "resumable" means with a link to docs. Implementation likely spans two repos: - `apache/airflow` -- metadata on the operators - `apache/airflow-site` -- rendering in `landing-pages/` (registry) ## Out of scope - Implementing resumability in operators that don't have it. This issue is about *surfacing* the capability, not adding it. - Changing how deferrable operators work. ## Project tracking https://github.com/orgs/apache/projects/612 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
