Nope, already checked that. On Fri, Apr 10, 2020 at 8:23 PM Patrick Timmins <ptimm...@cox.net> wrote:
> No issues here. Sounds like a timezone / system clock / clock drift issue > (in a cluster). > On 4/10/2020 11:59 AM, Joe Witt wrote: > > The provenance repo is in large scale use by many many users so > fundamentally it does work. There are conditions that apparently need > improving. In the past couple days these items have been flagged by folks > on this list, JIRAs and PRs raised and merged, all good. If you can help by > creating a build of the latest and confirm it fixes your case then please > do so. > > Thanks > > On Fri, Apr 10, 2020 at 12:48 PM Darren Govoni <dar...@ontrenet.com> > wrote: > >> It would seem the feature is either broken completely or only works in >> specific conditions. >> >> Can the Nifi team put a fix on their road map for this? >> Its a rather central feature to Nifi. >> >> Sent from my Verizon, Samsung Galaxy smartphone >> >> ------------------------------ >> *From:* Wyllys Ingersoll <wyllys.ingers...@keepertech.com> >> *Sent:* Friday, April 10, 2020 11:17:42 AM >> *To:* users@nifi.apache.org <users@nifi.apache.org> >> *Subject:* Re: Not Seeing Provenance data >> >> I have a similar problem with viewing provenance. I have a 3-node >> cluster in a kubernetes environment, the provenance_repository directory >> for each node is on a persistent data store so it is not deleted or lost >> between container restarts (which are not very common). My >> nifi.provenance.repository.max.storage.time is 24 hours. >> >> Whenever I try to view any provenance, nothing is ever shown. If I >> manually inspect the provenance_repository directory, there is a lucene >> index and TOC being created. >> >> I see log messages like these: >> >> Submitting query +processorId:882133fe-b684-148b-ad88-7850437ca591 with >> identifier 64a703fe-0171-1000-0000-000065abd91a against index directories >> [./provenance_repository/lucene-8-index-1560864819888] >> Returning the following list of index locations because they were >> finished being written to before 1586531601311: [] >> Found no events in the Provenance Repository. In order to perform >> maintenace of the indices, will assume that the first event time is now >> (1586531601311) >> >> >> Any suggestions? >> >> -Wyllys Ingersoll >> >> >> >> On Thu, Apr 9, 2020 at 11:25 AM Dobbernack, Harald (Key-Work) < >> harald.dobbern...@key-work.de> wrote: >> >> Hey Mark, >> >> >> >> great news and thank you very much! >> >> >> >> Happy Holidays! >> >> Harald >> >> >> >> *Von:* Mark Payne <marka...@hotmail.com> >> *Gesendet:* Donnerstag, 9. April 2020 17:18 >> *An:* users@nifi.apache.org >> *Betreff:* Re: Not Seeing Provenance data >> >> >> >> Thanks Harald, >> >> >> >> I have created a Jira [1] for this. There’s currently a PR up for it as >> well. >> >> >> >> Thanks >> >> -Mark >> >> >> >> [1] https://issues.apache.org/jira/browse/NIFI-7346 >> >> >> >> On Apr 9, 2020, at 11:14 AM, Dobbernack, Harald (Key-Work) < >> harald.dobbern...@key-work.de> wrote: >> >> >> >> Hi Mark, >> >> >> >> I can confirm after testing that if no provenance event has been >> generated in a time greater than the set >> nifi.provenance.repository.max.storage.time >> then as expected the last recorded provenance events don’t exist anymore >> but also from then on any new provenance events are also not searchable, >> the provenance Search remains completely empty regardless of how many flows >> are active. As described also *.prov file is then missing in provenance >> repository. After restart of Nifi new prov File will be generated and >> provenance will work again, but only showing stuff generated since last >> NiFi Start. >> >> >> >> So yes, I’d say your Idea >> >> ‘If so, then I think that would understand why it deleted the data. >> It’s trying to age off old data >> >> but unfortunately it doesn’t perform a check to first determine >> whether or not the “old file” >> >> that it’s about to delete is also the “active file”.’ >> >> fits very nicely to my test. >> >> >> >> As a workaround we’re going to set a greater >> nifi.provenance.repository.max.storage.time >> until this can be resolved. >> >> >> >> Thanks again for looking into this. >> >> Harald >> >> >> >> >> >> *Von:* Dobbernack, Harald (Key-Work) >> *Gesendet:* Donnerstag, 9. April 2020 15:22 >> *An:* users@nifi.apache.org >> *Betreff:* AW: Not Seeing Provenance data >> >> >> >> Hi Mark, >> >> >> >> thank you for looking into this. >> >> >> >> The nifi.provenance.repository.max.storage.time setting might explain why >> I haven’t been experiencing the effect so often since changing from the >> default to 120 hours a few months ago 😉 >> >> >> >> But I believe provenance stopped working last time although there was an >> ‘active’ flows in wait Processor, expiring every hour, going on to ‘send a >> message’ before being rerouted to the same wait processor. I would have >> expected this generates provenance entries? As I am not actually 100% sure >> if that wait processor was in use when last provenance got lost I will >> check with a testing system to see if I can reproduce provenance breakage >> when no active flows are around for a time greater >> nifi.provenance.repository.max.storage.time and I will get back to you. >> >> >> >> Thank you! >> >> Harald >> >> >> >> >> >> *Von:* Mark Payne <marka...@hotmail.com> >> *Gesendet:* Donnerstag, 9. April 2020 14:41 >> *An:* users@nifi.apache.org >> *Betreff:* Re: Not Seeing Provenance data >> >> >> >> Hey Daren, Herald, >> >> >> >> Thanks for the note. I have seen this once before but couldn’t figure out >> what caused it. Restarting addressed the issue. >> >> >> >> I think I may understand the problem, now, though, after looking at it >> again. >> >> >> >> In nifi.properties, there are a couple of property named >> “nifi.provenance.repository.max.storage.time” that defaults to “24 hours" >> >> Is it possible that you went 24 hours (or whatever value is set for that >> property) without generating any Provenance events? >> >> >> >> If so, then I think that would understand why it deleted the data. It’s >> trying to age off old data but unfortunately it doesn’t perform a check to >> first determine whether or not the “old file” that it’s about to delete is >> also the “active file”. >> >> >> >> Can you confirm whether or not you would expect to see 24 hours pass >> without any provenance data? >> >> >> >> Thanks >> >> -Mark >> >> >> >> >> >> >> >> On Apr 9, 2020, at 4:32 AM, Dobbernack, Harald (Key-Work) < >> harald.dobbern...@key-work.de> wrote: >> >> >> >> What I noticed is that as long as provenance is working there will be >> *.prov files in the directory. When Provenance isn’t working these files >> are not to be seen. Maybe some Cleaning Process deletes those files >> prematurely or the process building them doesn’t work any more? >> >> >> >> *Von:* Dobbernack, Harald (Key-Work) <harald.dobbern...@key-work.de> >> *Gesendet:* Donnerstag, 9. April 2020 10:27 >> *An:* users@nifi.apache.org >> *Betreff:* AW: Not Seeing Provenance data >> >> >> >> This is something I experience too from time to time. My quick and dirty >> workaround is stop nifi, delete everything in the provenance directory, >> restart…. Then Provenance is usable again (of course only with data since >> the delete) . I’m hoping very much there is a better way, someone can show >> us better settings or a potential bug can be discovered… >> >> >> >> *Von:* Darren Govoni <dar...@ontrenet.com> >> *Gesendet:* Mittwoch, 8. April 2020 20:31 >> *An:* users@nifi.apache.org >> *Betreff:* Not Seeing Provenance data >> >> >> >> Hi, >> >> When I go to "View data provenance" in Nifi, I never see any logs for >> my flow. Am I missing some configuration setting somewhere? >> >> >> >> thanks, >> >> Darren >> >> >> >> >> >> *Harald Dobbernack* >> Key-Work Consulting GmbH | Kriegsstr. 100 | 76133 | Karlsruhe | Germany | >> https://www.key-work.de | Datenschutz >> <https://www.key-work.de/de/footer/datenschutz.html> >> Fon: +49-721-78203-264 | E-Mail: harald.dobbern...@key-work.de | Fax: >> +49-721-78203-10 >> >> Key-Work Consulting GmbH, Karlsruhe, HRB 108695, HRG Mannheim >> Geschäftsführer: Andreas Stappert, Tobin Wotring >> >> >> >>
