GitHub user Ashfaqbs created a discussion: Human-in-the-loop approval: pluggable approver (decision model first, human fallback) - design discussion for #1067
Follow-up discussion for #1067, as requested there. ## Problem A common agentic pattern is pausing a workflow for a human to approve a consequential step (send this email, execute this trade, merge this change) before continuing. Flink Agents already has the durable, checkpointed, keyed state that makes this pattern natural to support well, but the runtime questions raised on #1067 need answering before any implementation: - An unfinished `ActionTask` keeps the key in flight, and later events for the same key are buffered. An approval delivered as an ordinary keyed input event could be queued behind the Action it needs to resume. - A long-lived pending key may affect watermark progression and end-of-input handling. - Checkpoints persist operator-managed task state, but not an in-memory Java stack or Python coroutine. Recovery needs an explicit persisted approval state machine and deterministic re-entry. - #1016 aligns part of the Java/Python recovery model, but doesn't yet define durable Action suspension and external resumption. ## A narrower first version: pluggable approver Rather than treating every proposal as needing the full durable-suspend-resume path, treat the **approver as pluggable**, with a decision model as the first tier and a human as the fallback (credit to @dangzitou for this framing). **Use case.** In cross-border e-commerce, for example, an agent proposes actions such as raising an ad bid or pausing a listing. Most proposals are routine and could be cleared automatically; only ambiguous or high-impact ones need a person. Sending every proposal to a human makes approval the bottleneck, and every one of them would take the hard suspend/resume path. **What already works today, with no new runtime semantics**, now that DecisionModel resources have landed (#1243 / #1264): 1. The proposing Action emits a `DecisionRequestEvent` with a `PredicateQuestion` (e.g. "Is this change within policy?") and application-supplied criteria. 2. The `DecisionResponseEvent` handler applies application-owned thresholds: - `probability_true >= T_approve` -> continue to the next step (e.g. emit the event that triggers the tool call) - `probability_true <= T_reject` -> reject and emit an output - in between, a `RefusalAnswer`, or a failed response -> escalate to a human Because the call goes through the built-in Decision Action, its outcome is recorded via durable execution when an Action State Store is configured, so recovery replays the verdict instead of asking the model again. **What this means for scope:** - Only the escalation branch needs durable suspension. The questions above still need answers, but they'd apply to a much smaller share of records, which may make a narrower first version acceptable. - If the pending-approval state records *who* resolved it (a decision model with its probabilities, or a human with an identity), the audit trail stays uniform across both paths. - Thresholds and the escalation policy should stay in application code, consistent with the DecisionModel design, where the business Action "chooses thresholds, and decides what to do with the answers." ## Open questions for maintainers 1. Is a built-in `await_approval`-style Action type the right shape, or should this stay a documented pattern built from existing primitives (Decision Action + a resource/tool call for the human-escalation leg)? 2. For the escalation leg specifically, how should the external "approve" signal be delivered - a resource/tool call, an API, or a keyed Kafka topic tied to the pending approval id? 3. Does narrowing scope to "only the escalation branch needs durable suspension" change the answers to the in-flight-key-buffering / watermark / recovery questions raised on #1067, or do they need full answers regardless of how small the escalation share is? Related: #1067, #1016, #1243, #1264 GitHub link: https://github.com/apache/flink-agents/discussions/1304 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
