Alexis Yep we totally understand.
What scenarios are you trying to plan/design for that require restarts? Thanks On Sat, Aug 17, 2024 at 8:24 AM Alexis Sarda-Espinosa < [email protected]> wrote: > Hi Joe, > > I agree the system can recover after the restart, but we want to deliver > data with as little latency as possible, so if FlowFiles stay queued during > the restart time window, by the time the node is up and running again the > data will be very late when time it finally leaves the NiFi cluster. > > Regards, > Alexis. > > On Sat, 17 Aug 2024, 16:59 Joe Witt, <[email protected]> wrote: > >> Alexis >> >> >> If you are simply restarting nodes you dont need to ensure everything is >> processed first. It will recover. >> >> If you want to scale down by removing a node there is an api to invoke >> offloading so you could scale down to a single node even if needed. >> >> Thanks >> >> On Sat, Aug 17, 2024 at 2:02 AM Alexis Sarda-Espinosa < >> [email protected]> wrote: >> >>> Hello, >>> >>> We use NiFi in cluster mode in Kubernetes via a StatefulSet. Sometimes, >>> we need to restart the cluster, so the StatefulSet restarts one pod at a >>> time. In this scenario, will a given node attempt to process all >>> "in-flight" FlowFiles to drain the queues as much as possible before >>> shutting down? >>> >>> I know it's probably impossible to have a completely graceful offload in >>> these cases, but since the process still receives a termination signal and >>> has some time before it must stop, it could try to prevent FlowFiles from >>> staying in the system for too long, but I don't know if there are any >>> mechanisms to handle this in NiFi. >>> >>> Regards, >>> Alexis. >>> >>
