I am also trying out a simple example. I connected a Generate Flowfile processor to a LogAttribute processor and even though I could see flowfiles moving through I didn't see any events for either of the component ids nor did I see any events in the overall view of all provenance events.
I did some grepping around the "provenance_repositoy" directory and I was able to find the component ids and the filenames I was setting in some of the files there. But nothing showed up in the UI. I am not sure of the conditions needed to reproduce this, but I do know it happens often enough to be a problem. Thanks, Eric On Wed, Dec 9, 2020 at 10:41 AM Eric Secules <esecu...@gmail.com> wrote: > Hi Leon, > > I want to try to rule out something. Are there any logs we can search for > that would help us find when provenance events are cleared out? I want to > rule out the CREATE event getting created and getting immediately deleted > due to some race condition. > > Some more detail is that the initial processor in our flow is a ListSFTP. > > Leon, could you confirm whether there were any provenance events for that > file? > > Thanks, > Eric > > > > On Tue, Dec 8, 2020 at 3:32 PM Leon Yu <leon...@macrohealth.com> wrote: > >> Hello, >> >> >> >> This is the first time I’ve emailed here so please bear with me. >> >> >> >> We are using nifi-1.12.1-RC2 and for one of our tests, we are seeing >> missing data provenance in NiFi despite the DB showing the flow being >> processed. >> >> A simplified explanation of how the system works is NiFi ingests a text >> file, creates a flow file from text file, and some data is written to the >> DB. >> >> The tests I was running ran two files through NiFi about 20 minutes >> apart. The DB has shown both files having successfully been processed. No >> other process writes to the DB other than NiFi. NiFi’s data provenance >> only showed data for the 1st file, there was no trace of the 2nd file >> despite success flow. >> >> >> >> Thank you, >> >> >> >> >> >> [image: A close up of a sign Description automatically generated] >> >> * Leon Yu* >> >> SDET, Macrohealth >> >> >> >> [image: A close up of a sign Description automatically generated] >> <http://www.macrohealth.com/> [image: A picture containing drawing >> Description automatically generated] >> <https://www.linkedin.com/company/macrohealth/> >> >> >> >
