Scott Nifi supports both push and pull. Push via reporting tasks and pull via rest api.
Are you needing a particular impl of a reporting task? You are right this is a common need. Solved using one of these methods. Thanks On Wed, Jul 21, 2021 at 2:58 PM scott <tcots8...@gmail.com> wrote: > Great comments all. I agree with the architecture comment about push > monitoring. I've been monitoring applications for more than 2 decades now, > but sometimes you have to work around the limitations of the situation. It > would be really nice if NiFi had this logic built-in, and frankly I'm > surprised it is not yet. I can't be the only one who has had to deal with > queues filling up, causing problems downstream. NiFi certainly knows that > the queues fill up, they change color and execute back-pressure logic. If > it would just do something simple like write a log/error message to a log > file when this happens, I would be good. > I have looked at the new metrics and reporting tasks but still haven't > found the right thing to do to get notified when any queue in my > instance fills up. Are there any examples of using them for a similar task > you can share? > > Thanks, > Scott > > On Wed, Jul 21, 2021 at 11:29 AM u...@moosheimer.com <u...@moosheimer.com> > wrote: > >> In general, it is a bad architecture to do monitoring via pull request. >> You should always push. I recommend a look at the book "The Art of >> Monitoring" by James Turnbull. >> >> I also recommend the very good articles by Pierre Villard on the subject >> of NiFi monitoring at >> https://pierrevillard.com/2017/05/11/monitoring-nifi-introduction/. >> >> Hope this helps. >> >> Mit freundlichen Grüßen / best regards >> Kay-Uwe Moosheimer >> >> Am 21.07.2021 um 16:45 schrieb Andrew Grande <apere...@gmail.com>: >> >> >> Can't you leverage some of the recent nifi features and basically run sql >> queries over NiFi metrics directly as part of the flow? Then act on it with >> a full flexibility of the flow. Kinda like a push design. >> >> Andrew >> >> On Tue, Jul 20, 2021, 2:31 PM scott <tcots8...@gmail.com> wrote: >> >>> Hi all, >>> I'm trying to setup some monitoring of all queues in my NiFi instance, >>> to catch before queues become full. One solution I am looking at is to use >>> the API, but because I have a secure NiFi that uses LDAP, it seems to >>> require a token that expires in 24 hours or so. I need this to be an >>> automated solution, so that is not going to work. Has anyone else tackled >>> this problem with a secure LDAP enabled cluster? >>> >>> Thanks, >>> Scott >>> >>
