Uwe, However you feel comfortable sharing it. If you feel more comfortable describing the settings in text, we'll try our best to work with whatever you can provide. A GitHub Gist would probably work without signing up for anything extra.
Thanks, James On Fri, Mar 24, 2017 at 12:50 PM, Uwe Geercken <[email protected]> wrote: > James, > > where should I put the screenshot? This mail group does not allow to send > graphics. > > Rgds, > > Uwe > > *Gesendet:* Freitag, 24. März 2017 um 18:27 Uhr > *Von:* "James Wing" <[email protected]> > *An:* [email protected] > *Betreff:* Re: Nifi - Data provenance not reporting anymore > Uwe, > > Can you share a screenshot of the provenance search criteria? If you try > the search the other way around, starting with all provenance events and > then filtering down to the time and processor, does that change anything? > > Thanks, > > James > > On Fri, Mar 24, 2017 at 8:39 AM, Uwe Geercken <[email protected]> > wrote: >> >> Hello, >> >> I have a production flow in place (nifi-1.1.1-RC1). It runs on a virtual >> machine running redhat. The reads multiple Json files (small) from a >> windows share/mountpoint. Various shell scripts write single lines (json) >> into files. The files are read, split and put into MongoDb. >> >> The flow goes like this: >> >> TailFile >> Split Text >> PutMongo >> >> Today is 2017-03-24. >> >> I have a webpage running which displays the data from the MongoDb. So I >> know when data arrives. And I can see that data arrived throughout the last >> days and also today >> >> I have detected something I don't understand: TailFile shows me in the >> flow that it has processed flow file in the last 5 minutes. But when I show >> the provenance events then Nifi shows: "Oldest event available: 03/16/2017 >> 18:02:34 CET". The list of events below it shows the most recent event as >> "03/24/2017 11:00:35.290 CET". But where are the events since then? I see >> that I have data in MongoDb after that. >> >> My current nifi-app.log looks like this: >> >> 2017-03-24 16:28:38,472 INFO [Write-Ahead Local State Provider >> Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali. >> MinimalLockingWriteAheadLog@2212e291 checkpointed with 91 Records and 0 >> Swap Files in 27 milliseconds (Stop-the-world time = 1 milliseconds, Clear >> Edit Logs time = 12 millis), max Transaction ID 12245824 >> 2017-03-24 16:30:38,506 INFO [Write-Ahead Local State Provider >> Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali. >> MinimalLockingWriteAheadLog@2212e291 checkpointed with 91 Records and 0 >> Swap Files in 34 milliseconds (Stop-the-world time = 2 milliseconds, Clear >> Edit Logs time = 15 millis), max Transaction ID 12247564 >> 2017-03-24 16:32:38,540 INFO [Write-Ahead Local State Provider >> Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali. >> MinimalLockingWriteAheadLog@2212e291 checkpointed with 91 Records and 0 >> Swap Files in 33 milliseconds (Stop-the-world time = 1 milliseconds, Clear >> Edit Logs time = 11 millis), max Transaction ID 12248871 >> >> There are no errors in the log and when I look back around "03/24/2017 >> 11:00:35.290 CET" there is nothing special either: >> >> 2017-03-24 10:56:32,547 INFO [Write-Ahead Local State Provider >> Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali. >> MinimalLockingWriteAheadLog@2212e291 checkpointed with 91 Records and 0 >> Swap Files in 34 milliseconds (Stop-the-world time = 2 milliseconds, Clear >> Edit Logs time = 17 millis), max Transaction ID 12002746 >> 2017-03-24 10:58:32,580 INFO [Write-Ahead Local State Provider >> Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali. >> MinimalLockingWriteAheadLog@2212e291 checkpointed with 91 Records and 0 >> Swap Files in 33 milliseconds (Stop-the-world time = 2 milliseconds, Clear >> Edit Logs time = 16 millis), max Transaction ID 12004378 >> 2017-03-24 11:00:32,624 INFO [Write-Ahead Local State Provider >> Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali. >> MinimalLockingWriteAheadLog@2212e291 checkpointed with 91 Records and 0 >> Swap Files in 43 milliseconds (Stop-the-world time = 2 milliseconds, Clear >> Edit Logs time = 20 millis), max Transaction ID 12005794 >> 2017-03-24 11:00:35,339 INFO [Timer-Driven Process Thread-7] >> o.a.n.p.PersistentProvenanceRepository >> Created new Provenance Event Writers for events starting with ID 2237386 >> 2017-03-24 11:00:50,975 INFO [Provenance Repository Rollover Thread-1] >> o.a.n.p.PersistentProvenanceRepository Successfully merged 16 journal >> files (111869 records) into single Provenance Log File >> ./provenance_repository/2125517.prov in 15677 milliseconds >> 2017-03-24 11:00:50,976 INFO [Provenance Repository Rollover Thread-1] >> o.a.n.p.PersistentProvenanceRepository Successfully Rolled over >> Provenance Event file containing 185608 records. In the past 5 minutes, 50 >> events have been written to the Provenance Repository, totaling 37.05 KB >> 2017-03-24 11:02:32,658 INFO [Write-Ahead Local State Provider >> Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali. >> MinimalLockingWriteAheadLog@2212e291 checkpointed with 91 Records and 0 >> Swap Files in 33 milliseconds (Stop-the-world time = 1 milliseconds, Clear >> Edit Logs time = 16 millis), max Transaction ID 12007317 >> 2017-03-24 11:04:32,687 INFO [Write-Ahead Local State Provider >> Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali. >> MinimalLockingWriteAheadLog@2212e291 checkpointed with 91 Records and 0 >> Swap Files in 28 milliseconds (Stop-the-world time = 1 milliseconds, Clear >> Edit Logs time = 14 millis), max Transaction ID 12008733 >> >> >> So the flow is working but I don't have provenance events. Can somebody >> help? >> >> Rgds, >> >> Uwe >> >> >
