Scott Check out this from Pierre: https://pierrevillard.com/tag/reporting-task/
We monitor all parameters from NiFi via Reporting Tasks. We send all parameters via MQTT to InfluxDB and monitor that via Grafana. There we can then start alerts when the levels reach a critical value. If that doesn't help you, then describe your problem specifically. Maybe we can help you. Mit freundlichen Grüßen / best regards Kay-Uwe Moosheimer > Am 22.07.2021 um 00:04 schrieb Joe Witt <joe.w...@gmail.com>: > > > Scott > > Nifi supports both push and pull. Push via reporting tasks and pull via rest > api. > > Are you needing a particular impl of a reporting task? > > You are right this is a common need. Solved using one of these methods. > > Thanks > >> On Wed, Jul 21, 2021 at 2:58 PM scott <tcots8...@gmail.com> wrote: >> Great comments all. I agree with the architecture comment about push >> monitoring. I've been monitoring applications for more than 2 decades now, >> but sometimes you have to work around the limitations of the situation. It >> would be really nice if NiFi had this logic built-in, and frankly I'm >> surprised it is not yet. I can't be the only one who has had to deal with >> queues filling up, causing problems downstream. NiFi certainly knows that >> the queues fill up, they change color and execute back-pressure logic. If it >> would just do something simple like write a log/error message to a log file >> when this happens, I would be good. >> I have looked at the new metrics and reporting tasks but still haven't found >> the right thing to do to get notified when any queue in my instance fills >> up. Are there any examples of using them for a similar task you can share? >> >> Thanks, >> Scott >> >>> On Wed, Jul 21, 2021 at 11:29 AM u...@moosheimer.com <u...@moosheimer.com> >>> wrote: >>> In general, it is a bad architecture to do monitoring via pull request. You >>> should always push. I recommend a look at the book "The Art of Monitoring" >>> by James Turnbull. >>> >>> I also recommend the very good articles by Pierre Villard on the subject of >>> NiFi monitoring at >>> https://pierrevillard.com/2017/05/11/monitoring-nifi-introduction/. >>> >>> Hope this helps. >>> >>> Mit freundlichen Grüßen / best regards >>> Kay-Uwe Moosheimer >>> >>>>> Am 21.07.2021 um 16:45 schrieb Andrew Grande <apere...@gmail.com>: >>>>> >>>> >>>> Can't you leverage some of the recent nifi features and basically run sql >>>> queries over NiFi metrics directly as part of the flow? Then act on it >>>> with a full flexibility of the flow. Kinda like a push design. >>>> >>>> Andrew >>>> >>>>> On Tue, Jul 20, 2021, 2:31 PM scott <tcots8...@gmail.com> wrote: >>>>> Hi all, >>>>> I'm trying to setup some monitoring of all queues in my NiFi instance, to >>>>> catch before queues become full. One solution I am looking at is to use >>>>> the API, but because I have a secure NiFi that uses LDAP, it seems to >>>>> require a token that expires in 24 hours or so. I need this to be an >>>>> automated solution, so that is not going to work. Has anyone else tackled >>>>> this problem with a secure LDAP enabled cluster? >>>>> >>>>> Thanks, >>>>> Scott
