I am convinced about using HITL too. I agree that this should probably be a separate package and should not be part of the standard one if possible due to plenty of reasons mentioned by Jens and others.
"common.io" sounds to be a very interesting place to start, as this HITL operator might not be the only one we will implement in the long run. "human" operator sounds weird to me. Thanks & Regards, Amogh Desai On Tue, Jun 24, 2025 at 1:43 PM Kaxil Naik <kaxiln...@gmail.com> wrote: > Thanks Wei. > > 1) "Human in the Loop": +1 on the naming. Standard names. HITL acronym is > also pretty standard (https://en.wikipedia.org/wiki/Human-in-the-loop | > https://cloud.google.com/discover/human-in-the-loop). "interactive" is a > loaded term and be pretty vague. > 2) re: Standard vs Separate Provider: Fine with either. But if it is in a > separate - the name "human" provider seems odd :) HITL as a functionality > makes sense but a "human" provider seems odd to me. If it is separate and > becomes part of "common.ai" - I am fine with that. I am equally happy with > keeping it in the Standard provider. Seems like a "core" functionality > compared to Control+M, and other legacy tools as well as new AI tools. > > On Tue, 24 Jun 2025 at 10:57, Jarek Potiuk <ja...@potiuk.com> wrote: > > > I like HITL as an acronym as well - it's well recognized. > > > > Just to add a bit of stir this is an interesting article when someone > tried > > to also distinct: > > * HITL (Human In the Loop) > > * with HOTL (Human On the Loop) > > * and HATL (Human Above the Loop) > > * and HBTL (Human Behind the Loop) > > > > Interesting concepts worth understanding the different roles humans can > > play here - But that's more of an interesting side/related read :) . > > > > > > > https://medium.com/@pawel.rzeszucinski_55101/ai-humans-and-loops-04ee67ac820b > > > > :) > > > > > > > > On Tue, Jun 24, 2025 at 4:35 AM Wei Lee <weilee...@gmail.com> wrote: > > > > > Hi everyone, > > > > > > Thank you for joining this discussion! At this point, it seems we have > > > reached some consensus. > > > > > > For naming, we now agree to use human centric term. To embed the whole > > > idea into operators, I will use HITL (apologies for the previous typo) > > and > > > mention "Human in the Loop" in the documentation and docstrings. > > > > > > Regarding whether it should be a standalone feature or not, it seems > more > > > like while it wouldn’t hurt to add it to the standard, it might be > better > > > to keep it separate. I’d like to gather more opinions on this. If we > > don’t > > > have a strong opinion about adding it to the standard, I think we > should > > > consider separating it. In the meantime, I will use the same PR to > > develop > > > the major functionality in the standard provider for easier development > > and > > > move it to a separate one if we reach a clear consensus. > > > > > > Thanks! > > > > > > Best, > > > Wei > > > > > > > On Jun 24, 2025, at 4:58 AM, Jens Scheffler > <j_scheff...@gmx.de.INVALID > > > > > > wrote: > > > > > > > > Hi, > > > > > > > > Thanks Wei for taking the lead in starting to implement! Hope I can > > > review the next days. > > > > > > > > as I was writing the AIP together with Vikram I was and still am for > > > (=+1) to keep it "human" centric. Also adding an API such that somebody > > > else is able to roll their whatever UI and not being locked into > Airflow > > UI > > > but still with the aim to loop-in humans. > > > > > > > > For the provider question I am for a separate provider because (1) it > > > was as such in the AIP, (2) I see it is optional and we should not > force > > > every install to have this (as it has not been there the last 10 years > I > > > assume there are many installs not needing it actually and some > > objections > > > were raised in the discussion that it is accepted if it is an optional > > > feature which it would not be if in standard provider) as well as (3) > we > > > need to adjust the DB and slightly extend this for the human response > > data > > > storage - and I would feel uncomfortable to force this DB extension > with > > > every install... then we could also directly package this into core - > but > > > as (2) it should be an optional feature. > > > > > > > > As well as (4) other common things like http, ftp, git, sftp, smtp > are > > > also pretty like stdnard but are also separate providers. From point of > > > security (5) every additional thing adds a bit of complexity and if you > > > want to make your setup secure you want to slim it down to the > functions > > > you need. Even though minimal if no human interaction is needed then I > > > think we should not force every install to have this. > > > > > > > > TLDR I'd therefore favor (+1) a separate "human" provider; not > favoring > > > (but +0) adding this to standard. > > > > > > > > Jens > > > > > > > > On 23.06.25 08:02, Amogh Desai wrote: > > > >> I was not strongly against using "human" -- it just felt a little > odd > > > and > > > >> confusing to me at first. > > > >> > > > >> Jarek's email has convinced me that having HITH is contextual in the > > AI > > > >> space and it is kinda what > > > >> we are doing with this AIP - 90. In fact, using "interactive" now > > seems > > > odd > > > >> that it is not descriptive enough > > > >> or doesn't highlight the intention of the operator enough. > > > >> > > > >> I do not have concerns with whatever we decide to name it :D > > > >> > > > >> > > > >> Thanks & Regards, > > > >> Amogh Desai > > > >> > > > >> > > > >> On Mon, Jun 23, 2025 at 9:42 AM Jarek Potiuk <ja...@potiuk.com> > > wrote: > > > >> > > > >>> In standard Provider, yes > > > >>> > > > >>> Re: name: I changed my opinion. Previously I raised concerns about > > it, > > > but > > > >>> they are gone. The name is IMHO perfect. > > > >>> > > > >>> Why do I think "Human-In-The-Loop" is the **right** name. It's a > very > > > >>> popular term in AI workflows, and used to interact with the "real" > > > human, > > > >>> and it has a very concrete meaning. Also I think it's really, > really > > > worth > > > >>> looking at the talk by Andrey Karpathy published a few days ago > > > >>> https://www.youtube.com/watch?v=LCEmiRjPEtQ - I think it is very > > > >>> insightful. Andrey coined the term "Vibe Coding", and I think he is > > > one of > > > >>> the smartest people in the AI space who is not hype-driven - i.e. > he > > > seems > > > >>> to genuinely think that AI is another technology change that is > > > reinventing > > > >>> how we do software. Unlike many of others he is not "selling" their > > > product > > > >>> in AI, he seems to be focused on one thing that I believe also is > > > important > > > >>> i.e. "Keeping Human in the Loop". > > > >>> > > > >>> One of the very interesting things I've learned from that talk is > how > > > >>> important it is to provide a good User Interface to AI. I.e. that > the > > > chat > > > >>> interface is cool, and everything but the crucial part of the AI > > > >>> interaction is to wrap the AI results into actionable, quick and > > > "nice" way > > > >>> of interacting with various aspects of AI by Human(s), when the > input > > > is > > > >>> not only important, but crucial to get the real value of AI. > > > >>> > > > >>> In this context I think we should focus to make sure that our > "Human > > In > > > >>> the Loop" is indeed designed for the Human - not for LLM imitating > > > Human, > > > >>> not for Agents. It should have a nice, pleasant and efficient > > > >>> UI, that should allow surfacing all the information that is > necessary > > > for > > > >>> the Human to make the decision. That information should be > > > >>> nicely formattable, and you should be able to use the typical way > > > >>> that people interact with it - with controls and everything they > are > > > >>> used to. A good interface example of a UI is when you use Copilot > in > > > your > > > >>> IDE for the translation. For example, the information you get (as > > > human) is > > > >>> targeted for humans and is very actionable. It is put in context, > you > > > can > > > >>> interact with it individually by accepting individual suggestions > (or > > > >>> rejecting them) or accept/reject things in bulk. > > > >>> > > > >>> Here is an example: (for those who do not see embedded picture - > link > > > here > > > >>> https://ibb.co/3Y03xN06) > > > >>> > > > >>> [image: Screenshot 2025-06-23 at 06.05.38.png] > > > >>> > > > >>> We should design "Human In the Loop" of ours in a very similar way > - > > > i.e. > > > >>> give the author of the "HIL" interaction capability of adding UI > > > >>> components, surfacing the right information - and having rich > > > interaction. > > > >>> Maybe not all the bells and whistles initially (for example it's ok > > > for now > > > >>> to just have bulk decisions on the whole interaction, but I think > > this > > > >>> should be our long-term design goal to allow for richer > interactions. > > > And - > > > >>> in this context - "Human in the loop" is a very appropriate name. > > > >>> > > > >>> BTW. Slightly related - there is a blog post coming about it from a > > > few of > > > >>> us about AI with internationalisation and how we made it to follow > > that > > > >>> (pretty naturally) with Open-Source spirit - by making sure that we > > > keep > > > >>> Human In the Loop and that it is designed to follow the Open Source > > > Spirit, > > > >>> Foster collaboration. > > > >>> > > > >>> > > > >>> On Mon, Jun 23, 2025 at 4:42 AM Wei Lee <weilee...@gmail.com> > wrote: > > > >>> > > > >>>> Got it! Yes, it makes sense to keep the phrase widely used. > Thanks a > > > lot! > > > >>>> As a compromise, I will try something like `HITHOperator`, which > may > > > >>>> address some of the concerns. We can always rename it to whatever > we > > > decide > > > >>>> before the release. I will also send a follow-up email to this > > thread > > > once > > > >>>> it's ready for review, so that anyone interested can take a look. > > > >>>> > > > >>>> Best, > > > >>>> Wei > > > >>>> > > > >>>>> On Jun 23, 2025, at 10:27 AM, Vikram Koka > > > <vik...@astronomer.io.INVALID> > > > >>>> wrote: > > > >>>>> I agree with the standard provider approach. > > > >>>>> > > > >>>>> > > > >>>>> On Sun, Jun 22, 2025 at 7:26 PM Vikram Koka < > vik...@astronomer.io> > > > >>>> wrote: > > > >>>>>> Thanks Wei, I really appreciate the work, and will review it as > > > soon as > > > >>>>>> possible. > > > >>>>>> > > > >>>>>> With respect to the naming, I believe the Human-in-the-loop is > the > > > >>>> right > > > >>>>>> phrase, because that is recognized as such both in older > "legacy" > > > >>>> systems, > > > >>>>>> as well as the new AI solutions. I agree that it may be less > than > > > ideal > > > >>>>>> from a technical perspective, but from a user perspective, I > > believe > > > >>>> it is > > > >>>>>> better to stick with a known term, rather than to invent our > own. > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> On Sun, Jun 22, 2025 at 7:07 PM Wei Lee <weilee...@gmail.com> > > > wrote: > > > >>>>>> > > > >>>>>>> Hi fellow Airflower, > > > >>>>>>> > > > >>>>>>> I am currently working on a PoC for AIP-90. I've incorporated > > some > > > >>>>>>> suggestions based on comments in the voting thread and Jira > page. > > > >>>> Since > > > >>>>>>> they have not yet been included in the AIP, I want to confirm > > with > > > >>>> everyone > > > >>>>>>> to ensure I'm on the right track. > > > >>>>>>> > > > >>>>>>> 1. Many have expressed concerns about the term “Human,” so I'm > > now > > > >>>> using > > > >>>>>>> the term “Interactive” as suggested by Among. For example, I > > change > > > >>>>>>> "HumanOperator" to “InteractiveOperator". > > > >>>>>>> 2. This functionality is now part of the standard provider > rather > > > than > > > >>>>>>> being a separate provider as suggested by Bas and Ash. > > > >>>>>>> > > > >>>>>>> Here is the PoC PR. > https://github.com/apache/airflow/pull/52053 > > > >>>>>>> It's not ready to be reviewed yet, but I'll try to wrap it up > > over > > > the > > > >>>>>>> next few days. Several features are still missing and will be > > > >>>> implemented > > > >>>>>>> in the following pull requests. Thanks! > > > >>>>>>> > > > >>>>>>> Best regards, > > > >>>>>>> Wei Lee > > > >>>>>> > > > >>>> > > > >>>> > > --------------------------------------------------------------------- > > > >>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > >>>> For additional commands, e-mail: dev-h...@airflow.apache.org > > > >>>> > > > >>>> > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > >
