Joining two or more flow files and merging the content

2020-12-08 Thread nathan.english
Hi All, I've got a case where I need to join two or more separate flow files, based on one or more 's ID within the flow (Multi Record flow file, each record with a unique ID). The content of the second flow file could also contain one or more records each again with a matching ID to the

Re: 1 minutes vs 5 minutes Metrics via Prometheus

2020-12-08 Thread jmkofoed
Hi Now I have checked all metrics and the following metrics have values going from 0 up to X and back to 0 within 3 scrapings: nifi_amount_items_queued nifi_amount_threads_active nifi_percent_used_bytes nifi_percent_used_count nifi_size_content_queued_total nifi_transmitting The following

Re: 1 minutes vs 5 minutes Metrics via Prometheus

2020-12-08 Thread jmkofoed
Hi again, This is a bit more complex because I see some metrics change every time Prometheus is scraping (15 sec.) The following metrics is updated frequently: nifi_size_content_queued_total nifi_amount_items_queued nifi_amount_items_output nifi_amount_bytes_received But if the rate of flowfiles

1 minutes vs 5 minutes Metrics via Prometheus

2020-12-08 Thread Jens M. Kofoed
We are looking into getting metrics from NIFI into grafana via Prometheus. It seems the metrics reported are the 5 minutes stats and not the 1 minutes snap. If we look at the status history of a processor we can see that the history is shown in a 1 minutes interval. I know the stat on the

NiFi Missing Provenance Data

2020-12-08 Thread Leon Yu
Hello, This is the first time I’ve emailed here so please bear with me. We are using nifi-1.12.1-RC2 and for one of our tests, we are seeing missing data provenance in NiFi despite the DB showing the flow being processed. A simplified explanation of how the system works is NiFi ingests a text

Utilizing NetflowV9 in NiFi v1.12.1

2020-12-08 Thread Thane Durey
Hello all, Has anyone found a solution to utilize NetflowV9? I know NiFi v1.12.1 has a processor to parse Netflow version 5, but I haven't found any information on Netflow version 9. >From all the browsing I've done, I've only found a few tickets/issues that discuss it in the