Hello Dheeraj & Shubham, Now that https://github.com/apache/airflow/pull/69148 is merged, isn't this a problem of calling "clear all downstream" tasks from the gate task? (which can be a custom operator based on ShortCircuitOperator)
In that case, isn't it simpler to just PR it (or internally solve it if it doesn't generalise well in open source) compared to having an AIP (since there is no fundamental architectural change to airflow, there hardly seems to be a need for an AIP here)? Thanks & Regards, Amogh Desai On Fri, Aug 7, 2026 at 3:41 AM Przemysław Mirowski <[email protected]> wrote: > Hi all, > > Thanks for writing the proposal. I've added one comment to the AIP > regarding alternative solutions. > > On 2026/07/31 08:12:47 Amogh Desai wrote: > > Thanks for writing this up, Dheeraj and Shubham. > > > > I have some thoughts on simplifying it and have added it in the AIP > > directly. > > > > Thanks & Regards, > > Amogh Desai > > > > > > On Fri, Jul 31, 2026 at 1:14 AM Constance Martineau via dev < > > [email protected]> wrote: > > > > > Hi Djeeraj, > > > > > > I left a comment in the AIP, but it would be great if you could include > > > some concrete examples of the workflow this serves. I understand why > the > > > workarounds are subpar and Airflow's limitations, but it's not obvious > to > > > me when I'd want to use this. A concrete workflow you run today would > make > > > the case much stronger. > > > > > > Thanks, > > > Constance > > > > > > On Thu, Jul 30, 2026 at 2:28 PM Dheeraj Turaga < > [email protected]> > > > wrote: > > > > > > > Dear Airflow Community, > > > > > > > > Id like to start a discussion on AIP-115: On-Demand Task Sections > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440305022/AIP-115+On-Demand+Task+Sections > > > > > > > > TL;DR - AIP-115: On-Demand Task Sections > > > > > > > > Some Dags contain work that is expensive, slow, risky, or only > > > > occasionally needed. Today an author can skip it (branching / > > > > short-circuit) and lose any first-class way to run it afterward, > > > > block the whole run on a human decision (HITL approval), or tell > > > > users to clear the right tasks by hand. GitLab's manual pipeline > > > > jobs are the closest existing analogue; Airflow has no Dag-level > > > > equivalent. > > > > > > > > AIP-115 proposes a ManualGateOperator that lets an author mark a > > > > section of a Dag as manual-only > > > > > > > > Scheduled runs skip past the gate and complete normally - nothing > > > > waits on a human. The gated tasks are shown distinctly in the grid > > > > and graph, and the gate offers a "Run manual section" action. > > > > Triggering it runs that section for the specific Dag run you chose, > > > > through the normal scheduler and executor path, with the usual logs, > > > > retries, XComs, callbacks, and trigger-rule behavior. There's an > > > > equivalent API endpoint for programmatic use. > > > > > > > > The result is an optional branch of a pipeline that's discoverable > > > > from the Dag definition, visible in the UI, auditable, and runnable > > > > per-run on demand - rather than a convention users have to know > > > > about and reproduce by clearing tasks manually. The change is > > > > additive: existing skipping, branching, and HITL behavior are > > > > unchanged. > > > > > > > > The proposal covers the section-boundary semantics, task states, API > > > > shape, and alternatives considered in detail > > > > > > > > ------ > > > > Would love to hear your thoughts & feedback > > > > > > > > Regards, > > > > Dheeraj > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
