+1 I often get requests for such a feature. Not in the Gen AI space specifically, but for simply having more human control over pipeline execution, to get a human review/approval before starting an important task.
There also have been prior discussions on Airflow’s GitHub: https://github.com/apache/airflow/issues/27162 https://github.com/apache/airflow/discussions/27400 Bas > On 21 May 2025, at 16:38, Buğra Öztürk <ozturkbugr...@gmail.com> wrote: > > +1 I like the idea. I think this can expand Apache Airflow usage in > different areas. It can enable lots of different possibilities via > orchestrating the business workflows. > > On Wed, 21 May 2025, 16:43 Jarek Potiuk, <ja...@potiuk.com> wrote: > >> +1. Sounds pretty useful to have it in Airflow. >> >> On Wed, May 21, 2025 at 4:24 PM Vincent Beck <vincb...@apache.org> wrote: >> >>> +1 as well. I can see this feature useful in many different cases (and >> not >>> only AI/ML related in my opinion). Happy to read the AIP and give >> feedbacks >>> :) >>> >>> On 2025/05/21 07:37:01 Tom Rutter wrote: >>>> +1 here!. As part of our Airflow deployment, we have built a layer on >> top >>>> to do review/approve and branching. It would be great for more of this >> to >>>> be standardised (and managed in the community by developers who are >> more >>>> skilled than me!). >>>> >>>> On Wed, 21 May 2025 at 09:12, Ankit Chaurasia <sunank...@gmail.com> >>> wrote: >>>> >>>>> +1 for this. From an ML/LLM-Ops standpoint, the introduction of a >>> "human in >>>>> the loop" operator appears exceptionally valuable. I can envision >> this >>>>> operator being effectively integrated into ML pipelines, allowing for >>>>> dynamic adjustments to model training parameters based on user >> feedback >>>>> received during the workflow execution. >>>>> >>>>> Furthermore, the utility of such an operator extends beyond ML >>>>> applications. There are numerous use cases within standard data and >>>>> infrastructure workflows where this functionality could be >> beneficial, >>> such >>>>> as implementing data quality overrides, securing compliance >> sign-offs, >>> or >>>>> even pausing incident response pipelines for human intervention. >>>>> >>>>> Best regards, >>>>> *Ankit Chaurasia* >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wed, May 21, 2025 at 12:42 PM Aritra Basu < >> aritrabasu1...@gmail.com >>>> >>>>> wrote: >>>>> >>>>>> I'm +0 for it, I like the broad strokes of the idea but actual >>> experience >>>>>> will very much depend on how we implement it. >>>>>> >>>>>> I do very much agree for it to have api support and if it would be >>> good >>>>> if >>>>>> we can design it with some thought to what future extensions might >> be >>>>>> needed on this feature. >>>>>> -- >>>>>> Regards, >>>>>> Aritra Basu >>>>>> >>>>>> On Wed, 21 May 2025, 11:51 am Amogh Desai, < >> amoghdesai....@gmail.com >>>> >>>>>> wrote: >>>>>> >>>>>>> I really like the idea and see it to be a window for some AI use >>> cases >>>>> to >>>>>>> be onboarded with >>>>>>> the orchestration capabilities of Airflow. >>>>>>> >>>>>>> I also concur with what Jens mentioned, it should be an opt-in / >>>>> opt-out >>>>>>> feature without >>>>>>> impacting how Airflow works with schedule today. >>>>>>> >>>>>>> As I see it, it's a special case of "event" driven scheduling >>> where the >>>>>>> "event" is a human input. >>>>>>> We should clearly define what happens to the task or the dag run >>> if the >>>>>>> human responsible for >>>>>>> the input doesn't respond in a timely manner. Either escalate, >>> fail, or >>>>>>> make an educated guess >>>>>>> would be some of the thoughts I have in mind. >>>>>>> >>>>>>> I really like the idea and the inputs from others here and I >> would >>> love >>>>>> to >>>>>>> join the party for this. >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> Amogh Desai >>>>>>> >>>>>>> >>>>>>> On Wed, May 21, 2025 at 10:07 AM Avi <a...@astronomer.io.invalid> >>>>> wrote: >>>>>>> >>>>>>>> +1 for the opt-in feature. Can clearly see the need for this to >>>>> exist. >>>>>>> And >>>>>>>> for both UI and APIs. +1 with Julian's idea of Dags waiting for >>>>> input. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> - Avi >>>>>>>> >>>>>>>> On Wed, May 21, 2025 at 3:42 AM Kalyan Reddy < >> kaly...@apache.org >>>> >>>>>> wrote: >>>>>>>> >>>>>>>>> I'm +1 on the idea. There should probably be notifications >> when >>>>>>> approval >>>>>>>>> is needed to continue a task. Additionally, a dashboard would >>> be >>>>>>> helpful >>>>>>>>> for viewing the tasks awaiting my approval. It kind of makes >> me >>>>>>> remember >>>>>>>>> approval gates on Azure DevOps pipelines. Even integrations >> to >>>>>>>>> Slack/PagerDuty to get alerts and approve/deny externally >> sound >>>>>>>>> interesting. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Kalyan R >>>>>>>>> >>>>>>>>> On 2025/05/21 02:07:55 Wei Lee wrote: >>>>>>>>>> I like this idea. As Jens mentioned, this should be an >> opt-in >>>>>>> feature, >>>>>>>>> and starting with providers seems reasonable. After that, we >>> could >>>>>>>> probably >>>>>>>>> get feedback from users and see how we could improve UI/UX. >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Wei >>>>>>>>>> >>>>>>>>>>> On May 21, 2025, at 5:36 AM, Jens Scheffler >>>>>>>> <j_scheff...@gmx.de.INVALID> >>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> oh I am a bit "envy" on receiving the email because I am >>>>> thinking >>>>>>>>> about EXACTLY the same propsal but did not file an AIP to >>> discuss >>>>>> about >>>>>>>>> this... as a matter of personal capacity and I wanted to >>> finish off >>>>>>>>> existing obligations before dropping new ideas. >>>>>>>>>>> >>>>>>>>>>> The very same idea for me was called "Human Interaction" >>>>> provider >>>>>>>>> package (maybe shot name "apache-airflow-providers-human" >> with >>> a >>>>> set >>>>>> of >>>>>>>>> operators/sensors to be integrated in a DAG (if needed) to >>>>>>> Approve/Rejct >>>>>>>>> ==ShortCircuit Operator, an Operator to select by Human as a >>>>>>>>> "HumanBranchOprator" and also if data is missing allowing to >>>>> request >>>>>>> more >>>>>>>>> data with the TriggerForm component. >>>>>>>>>>> >>>>>>>>>>> So nobody is forcing to have this but it can be easily a >>>>> provider >>>>>>>>> package w/o need to change main parts in core. Everybod is >>> free to >>>>>>> decide >>>>>>>>> whether it is a good thing to "block" (with a sensor) a >>> workflow to >>>>>>>> approve >>>>>>>>> - for example you make a Terraform Plan and sombody need to >>> double >>>>>>> check >>>>>>>>> before you Terraform Apply. Maybe you run an AI traning set >>> and if >>>>>> the >>>>>>>>> threshold of data is exceeded somebody need to approve >> budget. >>> Or >>>>> you >>>>>>>> build >>>>>>>>> an AI model and based on the metrics somebody need to approve >>>>>>> deployment. >>>>>>>>> Many cases possible. >>>>>>>>>>> >>>>>>>>>>> I was also a bit reluctant to push this topic now >>> (myself... >>>>> but >>>>>>> now >>>>>>>>> the idea is un-bottled :-D) bcause I assume AIP-68 would need >>> to be >>>>>>>>> implemented first with allowing the UI to extnd with UI >>> widgets in >>>>>>> React >>>>>>>>> allowing the human forms being seamlessly integrated in the >>> new UI. >>>>>> So >>>>>>> I >>>>>>>>> see AIP-68 as a dependency. >>>>>>>>>>> >>>>>>>>>>> I think it might be good to document details on an AIP to >>> align >>>>>>> idas >>>>>>>>> and scope. Would like to join the party. (with my currentl >>> limited >>>>>>>> capacity >>>>>>>>> that aims to complete more tasks than opening more) >>>>>>>>>>> >>>>>>>>>>> Jens >>>>>>>>>>> >>>>>>>>>>> On 20.05.25 22:02, Constance Martineau wrote: >>>>>>>>>>>> I like the idea! We built something similar where a task >>> would >>>>>>> send >>>>>>>> an >>>>>>>>>>>> email with a report attached (Excel, of course). The >> user >>> had >>>>> to >>>>>>>>> either >>>>>>>>>>>> approve the report or flag an issue. If they didn't >>> respond >>>>> in 2 >>>>>>>>> hours, it >>>>>>>>>>>> escalated to a backup, then to the department lead. If >> no >>> one >>>>>>>>> responded, >>>>>>>>>>>> the DAG failed. It was pretty effective for >>> business-critical >>>>>>>>> approvals, >>>>>>>>>>>> and we had clear timeouts and fallback logic. >>>>>>>>>>>> >>>>>>>>>>>> That said, I think this pattern works best when it's >> used >>>>>>> sparingly >>>>>>>>> and >>>>>>>>>>>> with good defaults. Echoing Alexander, human input is >>>>> inherently >>>>>>>>>>>> unreliable, but sometimes you really do it, whether it's >>> for >>>>>>>>> validating a >>>>>>>>>>>> data quality issue, resolving a content flag, or signing >>> off >>>>> on >>>>>>>>> something >>>>>>>>>>>> before moving forward. I don't think that breaks the >>> Airflow >>>>>> model >>>>>>>> as >>>>>>>>> long >>>>>>>>>>>> as: >>>>>>>>>>>> >>>>>>>>>>>> - You can define what happens if no one responds >>> (timeout, >>>>>>> fail, >>>>>>>>>>>> escalate, etc.) >>>>>>>>>>>> - There's an API so you're not forcing people into >> the >>> UI >>>>>>>>>>>> - There's a centralized view of all "waiting for >> input" >>>>> tasks >>>>>>> to >>>>>>>>> avoid >>>>>>>>>>>> hunting them down >>>>>>>>>>>> >>>>>>>>>>>> It shouldn't become a crutch for manual processes, but >> I'd >>>>> love >>>>>> to >>>>>>>> see >>>>>>>>>>>> first-class support for this kind of interaction. It >>>>> definitely >>>>>>>> comes >>>>>>>>> up in >>>>>>>>>>>> the wild. >>>>>>>>>>>> >>>>>>>>>>>> On Tue, May 20, 2025 at 3:46 PM Alexander Shorin < >>>>>>> kxe...@apache.org >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> While this is an understandable idea, I think it's >> wrong >>> to >>>>>>> involve >>>>>>>>> humans >>>>>>>>>>>>> to take a part into Airflow workflow. >>>>>>>>>>>>> Run with parameters. Read the logs. Retry. >>>>>>>>>>>>> Blocking the whole pipeline could not be acceptable: >>> humans >>>>> are >>>>>>> not >>>>>>>>> stable >>>>>>>>>>>>> resource for input, they may be sick or on vacation and >>> who >>>>>> will >>>>>>> be >>>>>>>>>>>>> operator for this branch? >>>>>>>>>>>>> Better strategy to collect metrics and logs and do >>>>> post-review >>>>>> of >>>>>>>>> tasks >>>>>>>>>>>>> results. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> ,,,^..^,,, >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, May 20, 2025 at 10:15 PM Vikram Koka >>>>>>>>> <vik...@astronomer.io.invalid >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hey everyone, >>>>>>>>>>>>>> >>>>>>>>>>>>>> This is a small technical change, but conceptually it >> is >>>>>>>>> significant and >>>>>>>>>>>>>> therefore bringing it to the devlist. >>>>>>>>>>>>>> >>>>>>>>>>>>>> As we have been having conversations with Airflow >>> users, who >>>>>> are >>>>>>>>> using >>>>>>>>>>>>>> Airflow for Gen AI applications, there are a couple of >>>>>> features >>>>>>>>> that have >>>>>>>>>>>>>> been brought up as being desirable. As context, based >>> on the >>>>>>> most >>>>>>>>> recent >>>>>>>>>>>>>> Airflow survey, over 8% of Airflow users are now using >>>>> Airflow >>>>>>> for >>>>>>>>> Gen AI >>>>>>>>>>>>>> use cases. >>>>>>>>>>>>>> >>>>>>>>>>>>>> One of the desired features is to have "human >>> interaction" >>>>>>> within >>>>>>>>> the >>>>>>>>>>>>>> context of the overall orchestration flow. Here are >> the >>> most >>>>>>>> common >>>>>>>>>>>>> flows: >>>>>>>>>>>>>> 1. Choose a branch in the workflow: This is like a >>> "human >>>>>>>> branch >>>>>>>>>>>>>> operator", where the human gets to pick one of the >>> paths >>>>> to >>>>>>>> move >>>>>>>>>>>>>> forward. >>>>>>>>>>>>>> This is a common pattern for business use cases >> such >>> as >>>>>>> content >>>>>>>>>>>>>> moderation, >>>>>>>>>>>>>> when the automated sentiment analysis is unsure if >>> the >>>>>>> content >>>>>>>> is >>>>>>>>>>>>>> "appropriate" and requires human judgement. >>>>>>>>>>>>>> 2. Approve / Reject: This is a specialized form of >>> the >>>>>> above, >>>>>>>>> where >>>>>>>>>>>>> the >>>>>>>>>>>>>> human approves or rejects the output from the prior >>>>> tasks, >>>>>>>>> thereby >>>>>>>>>>>>>> triggering a different branch in the flow. >>>>>>>>>>>>>> 3. Provide input: The human is expected to provide >>>>> textual >>>>>>>> input, >>>>>>>>>>>>> which >>>>>>>>>>>>>> can then be used for subsequent steps. This is >>> intended >>>>> to >>>>>> be >>>>>>>>> used >>>>>>>>>>>>>> within >>>>>>>>>>>>>> LLM workflows, where a human supplied context / >>> guidance >>>>>> may >>>>>>> be >>>>>>>>>>>>>> critical. >>>>>>>>>>>>>> >>>>>>>>>>>>>> All of these above operations are expected to be >>> performed >>>>>>> through >>>>>>>>> the >>>>>>>>>>>>>> Airflow UI. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 4. A distinct variation of the above could be an API >>>>>> interaction >>>>>>>>> for the >>>>>>>>>>>>>> above workflows without using the UI. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Technical notes: >>>>>>>>>>>>>> a. While the above tasks are being run (or rather >>> waiting >>>>> for >>>>>>>> human >>>>>>>>>>>>> input), >>>>>>>>>>>>>> the DAG will be treated as a continued DAG run. >>>>>>>>>>>>>> >>>>>>>>>>>>>> b. The Airflow UI for these tasks would be a standard >> UI >>>>> form >>>>>>>>> similar to >>>>>>>>>>>>>> the Trigger DAG form. >>>>>>>>>>>>>> For flows (1) and (2) above, this would be populated >>> based >>>>> on >>>>>>> the >>>>>>>>>>>>> follow-on >>>>>>>>>>>>>> task options. >>>>>>>>>>>>>> For flow (3) above, this would be a UI form similar to >>> DAG >>>>>>> params. >>>>>>>>>>>>>> >>>>>>>>>>>>>> c. Each of the above operations will be treated as a >>> "task". >>>>>>>>>>>>>> The task state while waiting for human input will >>> probably >>>>> be >>>>>> a >>>>>>>>> variation >>>>>>>>>>>>>> of the "deferred task state". TBD as we get closer to >>>>>>>>> implementation. >>>>>>>>>>>>>> For flow (3), the input data will be passed using XCom >>> to >>>>>>>> subsequent >>>>>>>>>>>>> tasks. >>>>>>>>>>>>>> >>>>>>>>>>>>>> As detailed above, the technical changes are fairly >>> small. >>>>> Any >>>>>>>>> feedback >>>>>>>>>>>>> is >>>>>>>>>>>>>> welcome. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best regards, >>>>>>>>>>>>>> Vikram >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> >>>>>>>>>>>>>> Vikram Koka >>>>>>>>>>>>>> Chief Strategy Officer >>>>>>>>>>>>>> Email: vik...@astronomer.io >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> <https://www.astronomer.io/> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>> >>> --------------------------------------------------------------------- >>>>>>>>>>> To unsubscribe, e-mail: >> dev-unsubscr...@airflow.apache.org >>>>>>>>>>> For additional commands, e-mail: >>> dev-h...@airflow.apache.org >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>> >> --------------------------------------------------------------------- >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>>>>> For additional commands, e-mail: >> dev-h...@airflow.apache.org >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>> For additional commands, e-mail: dev-h...@airflow.apache.org >>> >>> >>
