Mark, https://github.com/apache/nifi/pull/11754 if you have some time.
Thanks, Mike On Fri, Sep 18, 2026 at 9:40 AM Mike Thomsen <[email protected]> wrote: > Understood. I just wanted to pass that along because my gut sense was that > it's not necessary for a MVP of the tool I'm hacking out, but I wanted to > run it by you instead of assuming. > > Here's the branch I created off your suggestion. It looks clean, and my > tool was able to be refactored around it and run my test flows with more or > less the same capability so it seems like a fit. > > https://github.com/MikeThomsen/nifi/tree/mp-nip46 > > Thanks, > > Mike > > On Thu, Sep 17, 2026 at 1:14 PM Mark Payne <[email protected]> wrote: > >> Thanks for the follow-up Mike. >> >> I think I’d need more details to really understand the concern there. A >> general exception does not necessarily mean that there is a failure. If >> there are true failures that somehow get swallowed and not propagated back >> up to the caller, that sounds like a bug that we’d need to dig into. >> >> Thanks >> -Mark >> >> >> > On Sep 17, 2026, at 8:34 AM, Mike Thomsen <[email protected]> >> wrote: >> > >> > To clarify… where an exception happened during an asynchronous commit. >> > >> > Sent from my iPhone >> > >> >> On Sep 17, 2026, at 8:33 AM, Mike Thomsen <[email protected]> >> wrote: >> >> >> >> Mark, >> >> >> >> I am working on an integration test runner for flows, and that is the >> core use case that made me look into this. I had Claude work out my >> proposal, David’s and yours. I think yours is probably the cleanest for >> capturing exceptions that escape onTrigger, but Claude warned that it >> didn’t capture cases where an asynchronous commit happened nor did it >> reliably capture exceptions inside callbacks in processsession read/write. >> What are your thoughts on that? >> >> >> >> Sent from my iPhone >> >> >> >>> On Sep 16, 2026, at 10:22 AM, Mark Payne <[email protected]> >> wrote: >> >>> >> >>> Thanks, David, I’ve been meaning to take a minute to reply here so >> good that you pinged. >> >>> >> >>> Currently the API has: >> >>> >> >>> ``` >> >>> interface StatelessDataflow { >> >>> … >> >>> DataflowTrigger trigger(DataflowTriggerContext context); >> >>> … >> >>> } >> >>> ``` >> >>> >> >>> And DataFlowTrigger has: >> >>> ``` >> >>> Optional<TriggerResult> getResult(long maxWaitTime, TimeUnit >> timeUnit) throws InterruptedException; >> >>> ``` >> >>> >> >>> And TriggerResult contains: >> >>> ``` >> >>> boolean isSuccessful(); >> >>> Optional<Throwable> getFailureCause(); >> >>> ``` >> >>> >> >>> So at present, the API does expose whether or not the run was >> successful and if not, the Throwable that was the cause. >> >>> In terms of the information that is available, compared to what is >> proposed in NIP-46, I would say the differences are: >> >>> >> >>> - The API does NOT expose which component caused the failure. The >> type is likely exposed in the stack trace, but not made available in a way >> that is accessible programmatically, and it doesn’t expose the ID or the >> name of the component. >> >>> - The API is not a callback but rather this information is surfaced >> to the caller. >> >>> >> >>> I do believe that surfacing this information to the caller via >> TriggerResult is the more desirable API, rather than introducer a new >> callback / listener type of API. >> >>> >> >>> If there’s a concrete use case where we think it makes sense to >> return something like a `FailingComponent` object that contains the >> component id, name, type, etc. I’m not opposed to surfacing that >> information. But IMO it should be surfaced on the TriggerResult, such as a >> new `Optional<FailingComponent> getFailingComponent()` method, rather than >> via listeners. >> >>> >> >>> Would that scratch the itch here? >> >>> >> >>> Thanks >> >>> -Mark >> >>> >> >>> >> >>> >> >>> On Sep 16, 2026, at 9:09 AM, Mike Thomsen <[email protected]> >> wrote: >> >>> >> >>> David, >> >>> >> >>> That might actually be a better approach. I'll look into those two >> >>> repositories and see if I can achieve a similar outcome without >> messing >> >>> with stateless NiFi (or doing nothing more than forking the current >> >>> repository code to make it more observable). >> >>> >> >>> Thanks, >> >>> >> >>> Mike >> >>> >> >>> On Tue, Sep 15, 2026 at 4:51 PM David Handermann < >> >>> [email protected]> wrote: >> >>> >> >>> Mike, >> >>> >> >>> Thanks for proposing this addition to NiFi Stateless behavior. >> >>> >> >>> The concept of observing lower-level details in the Stateless flow >> >>> makes sense at one level, but also raises a few more motivation >> >>> questions. >> >>> >> >>> The CI/CD use case is a good one in general, and I can see why it >> >>> could be useful to get more details on whether the flow might have >> >>> failed. Although it would be a bit more indirect, did you consider >> >>> some alternative type of Bulletin Repository or Provenance Repository >> >>> that could surface this information? The Bulletins provide the >> >>> component-level message and Provenance keeps track of processing >> >>> outcomes, so building something in that direction seems to align more >> >>> with the general approach of NiFi itself. >> >>> >> >>> The new listener surface seems focused enough to avoid significant >> >>> changes, but I raise the question of alternatives to evaluate whether >> >>> this approach aligns with Stateless in general, or whether it starts >> >>> observability going in a different direction. >> >>> >> >>> I would also look for input from Mark Payne given his extensive work >> >>> with Stateless and NiFi itself. >> >>> >> >>> Regards, >> >>> David Handermann >> >>> >> >>> On Sat, Sep 12, 2026 at 5:31 AM Mike Thomsen <[email protected]> >> >>> wrote: >> >>> >> >>> I have created a NIP to add some additional capabilities to Stateless >> >>> NiFi >> >>> that would make it much easier for calling applications to track >> failure >> >>> signals in NiFi components. >> >>> >> >>> https://issues.apache.org/jira/browse/NIP-46 >> >>> >> >>> The need for this NIP came about while I was working on a tool to run >> >>> NiFi >> >>> flows inside of a test harness, but I think it could also be useful in >> >>> general for other applications of Stateless NiFi such as launching it >> >>> from >> >>> a stateful flow. >> >>> >> >>> Assuming no objections, I plan to start a lazy-consensus vote soon. >> >>> >> >>> Thanks, >> >>> >> >>> Mike >> >>> >> >>> >> >>
