Which I hope to have RC1 of 1.12 up today to start the review/vote process. If you two (Shawn, Wyll) could test it against what you're seeing it would be very helpful!
Thanks On Mon, Aug 10, 2020 at 10:43 AM Mark Payne <marka...@hotmail.com> wrote: > OK, so that would be indicative of NIFI-7346. So the data is being written > into the ether, and then it’s being indexed by Lucene (which is why you see > the data being written to that Lucene subdirectory). But even when you > search, it can’t find the event files themselves. So this should all be > addressed in the upcoming release. > > Thanks > -Mark > > > On Aug 10, 2020, at 1:40 PM, Wyll Ingersoll < > wyllys.ingers...@keepertech.com> wrote: > > Great, thanks! Looking forward to trying it out on 1.12 > > I can't see any provenance data with either method (right-click on > processor or hamburger menu). > > When I login to the server, I do see data being written to the > provenance_repository directory in a sub-dir that looks like > "lucene-8-index-1560864167766" and which appears to have current data, I > just cant view anything from the UI. > > > ------------------------------ > *From:* Mark Payne <marka...@hotmail.com> > *Sent:* Monday, August 10, 2020 1:37 PM > *To:* users@nifi.apache.org <users@nifi.apache.org> > *Subject:* Re: Data Provenance Stops Working > > Shawn / Wyll, > > I think you’re probably running into NIFI-7346 [1], which basically says > there’s a case in which NiFi may “age off” old data even when it’s still > the file that’s being actively written to. In Linux/OSX this results in > simply deleting the file, and then anything else written to it disappears > into the ether. Of course, now the file never exceeds the max size, since > it’ll be 0 bytes forever, os it never rolls over again. So when this > happens, no more provenance data gets created until NiFi is restarted. > > It’s also possible that you’re hitting NIFI-7375 [2]. This Jira only > affects you if you get to provenance by right-clicking on a Processor and > clicking View Provenance (i.e., not if you go to the Hamburger Menu in the > top-right corner and go to Provenance from there and search that way). If > this is the problem, once you right-click and go to View Provenance, you > can actually click the Search icon (magnifying glass) in that empty > Provenance Results panel and click Search and then it will actually bring > back the results. So that’s obnoxious but it’s a workaround that may help. > > The good news is that both of these have been addressed for 1.12.0, which > sounds like it should be coming out very soon! > > Thanks > -Mark > > [1] https://issues.apache.org/jira/browse/NIFI-7346 > [2] https://issues.apache.org/jira/browse/NIFI-7375 > > > On Aug 10, 2020, at 1:26 PM, Joe Witt <joe.w...@gmail.com> wrote: > > shawn - i believe it is related to our default settings and have phoned a > friend to jump in here when able. but default retention and default > sharding i *think* can result in this. You can generate a thread dump > before and after the locked state to see what it is stuck/sitting on. That > will help here > > Thanks > > On Mon, Aug 10, 2020 at 10:24 AM Shawn Weeks <swe...@weeksconsulting.us> > wrote: > > Out of the box even the initial admin user has to be granted permission I > think, mine worked fine for several months since 1.11.1 was released and > just started having an issues a couple of weeks ago. I’ve increasing the > retention time a bit to see if that improves the situation a bit. > > Thanks > Shawn Weeks > > *From: *Wyll Ingersoll <wyllys.ingers...@keepertech.com> > *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Date: *Monday, August 10, 2020 at 12:22 PM > *To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Subject: *Re: Data Provenance Stops Working > > I run 1.11.4 in a cluster on AWS also and have a similar issue with the > provenance data, I can't ever view it. It's probably somehow misconfigured > but I haven't been able to figure it out. > ------------------------------ > *From:* Andy LoPresto <alopre...@apache.org> > *Sent:* Monday, August 10, 2020 1:11 PM > *To:* users@nifi.apache.org <users@nifi.apache.org> > *Subject:* Re: Data Provenance Stops Working > > Shawn, > > I don’t know if this is specifically related, but there were a number of > critical issues discovered in the 1.11.x release line that have been fixed > in 1.11.4. I would not recommend running any prior version on that release > line. > > 1.12.0 should be coming imminently, so if you are going to upgrade anyway, > you may want to wait a week or so and get the newest bits with hundreds of > new features, but for stability alone, I would strongly recommend 1.11.4. > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes > > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > He/Him > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > On Aug 10, 2020, at 10:00 AM, Shawn Weeks <swe...@weeksconsulting.us> > wrote: > > I’m running a three node NiFi Cluster on AWS EC2s using integrated > Zookeeper and SSL Enabled. Version is 1.11.1, OS is RedHat 7.7, Java is > 1.8.0_242. For some reason after a period of time Data Provenance goes > blank, old records are no longer queryable and new data provenance doesn’t > appear to get written. No Lucene or other exceptions are logged and > restarting the node causes data provenance to go back to being written > however old data provenance does not re-appear. No exceptions appear when > querying data provenance. All tests have been run as the initial admin user > and roles and permissions appear to be correct. I’ve also checked selinux > audit logs to see if something is being blocked but nothing appears. > > WriteAheadProvenanceRepository, max storage is set to 24 hours, 1GB, 30 > seconds for rollover, 100mb rollover size, 2 query threads, 2 index > threads, compress on rollover, and don’t sync always. > > Thanks > Shawn Weeks > > >
